135mm SHOOTOUT! Detailed Analysis of the Hand-Holdability and General Sharpness of the RF135/1.8, the RF100-500/4.5-7.1, and the EF135/2

SwissFrank

1N 3 1V 1Ds I II III R R5
Dec 9, 2018
573
395
Background

I want to understand how long I can hand-hold my lenses. Traditionally we used a rule of thumb: the "reciprocal rule," stating that you should use a shutter speed at least as high as one divided by the focal length. For instance, a 90mm lens would call for a 1/90th second exposure or better. But it was never that simple: it's not a question of "sharp" vs. "not sharp," but rather "how sharp" and even "what percentage would be how good or how bad." Further, this advice came out when even 400-speed film was grainy, focus was never perfect, and lenses weren't that sharp anyway. Quality of the image just wasn't great, and even a fair amount of unsharpness from a moving camera would be hidden by these other greater factors that are now gone. It's possible that the reciprocal rule is now way too optimistic, given the high-megapixel, low-noise sensors with today's excellent lenses and perfect autofocus. On the other hand, the advent of image stabilization changes everything too, with makers claiming unbelievably great-sounding improvements in hand-holdability, that I was frankly dubious of.

I also bought the new RF135/1.8IS, to replace my beloved but ancient EF135/2.0, and want to characterize what my money's buying in terms of image stabilization as well as general sharpness. (Other benefits are slightly increased bokeh, and weatherproofing.) My camera, as most high-end cameras now, has in-body image stabilization via a sensor that the camera can move slightly to offset detected movement, and Canon especially has put additional stabilization in most cameras that can move lenses to compensate.

Finally, I'd like to know how my various lenses compare: if I have three lenses that can shoot at 100mm or 135mm, which should I use on a shot? Am I safe leaving lens X or Y home, or should I really take it to get best results? Until 1995 you could be sure that any prime lens would be sharper than any zoom, but that started changing, and by 2020 the more moderately-apertured pro-quality zooms seem outstanding. But have primes continued to improve too?

The Test

For these reasons I wanted to do a comparison. I have multiple lenses to choose from at focal lengths 16mm, 28mm, 50mm, 100mm, and 135mm, so these are the focal lengths I'm personally keen to test. However the 135 was my most recent acquisition so was on my mind first.

Here are four comparison charts I've made, each based on shooting ten trials at each of ten or so shutter speeds. The target shown was from the center or close to it in each shot. Each line of each chart has a benchmark tripod image from the 135/1.8 at the beginning and end of the line. The ten trials at each shutter speed are sorted from sharpest to least-sharp. Software attempts to score the sharpness to automate this, though it's only a guideline. The score is written on each target again, it's just a guideline; let your eyes be the judge. The squares are 225 pixels wide, which is basically 1mm on the sensor (227 pixels per mm I think). The data is 1:1 scale and meant to be viewed shrunken to fit, to get an overview, and at 1:1 to study individual results. Some lenses don't show ten images: the missing ones were too blurry to process.

The test target has 55lp/mm, the lines of which are about 2.05 pixels tall on the sensor of a Canon R5. I chose this as thinner lines cannot be depended to produce a black pixel. (Say, a 1.5 pixel wide line could fill two adjacent pixels 3/4 black, so even if the lens were perfect you wouldn't have black. In contrast 2 pixels tall will always leave one pixel seeing nothing but black, and possibly two if it just happens to be aligned.)

Findings

This is a severe test conducted to differentiate even between perfect and "merely" excellent.
The main point is that, of the IS images, even the worst are fantastic in an absolute sense, even the worst of the 1 second images on the worst lens here. The image quality obviously sucks at 1 second for the IS images compared to what they can do at faster speeds. And yet, each black and white line is 2 pixels wide. If you reduce size by 4 (say, to 2000 pixels width), the circles will literally be uniform gray and NONE of the blur shown here will even be visible, even looking pixel by pixel at such an image. Your two takeaways from that are:
  • Even the 135/2's apparently bad results will look nearly identical to the 135/1.8 if you're using <=2000 pixel-wide images.
  • Even 1 second handheld exposures will look nearly identical to tripod benchmarks on any of these lenses if you're using <=2000 pixel-wide images.

At 1/30, even the worst trial from the RF135/1.8L IS or RF100-500/4.5-7.1L IS is sharper than the very best image at any speed from the EF135/2.0L. When I got the EF135 in 1997, it seemed to be by far the sharpest lens I had, definitely the sharpest non-white lens until the 180/3.5Macro came out. So, I'm saddened yet amazed to see it's being out-shot by a zoom lens hand-held a relative eternity.

At 1/60, even the worst trial from the RF135/1.8L IS is sharper than the very best image at any speed from the RF100-500/4.5-7.1L IS.

At 1/4, the RF135/1.8L IS with IS on outperforms the same lens with no IS at any speed including 1/1000. I would therefore say it is quite accurate to say IS really is giving 8 stops of stabilization. Likewise 1/2 with IS outperforms 1/500 no IS (again 8 stops). 1 sec with IS outperformed 1/250 no IS in 6 out of 10 samples or so, so yet again 8 stops.

While the EF135/2.0 lacks in-lens stabilization, it doesn't seem to be held back too much: it's maybe 0.5-1.5 stops "less hand-holdable" than the RF, perhaps, if you compare the really long exposures. So don't worry about the lack of in-lens IS if your budget only stretches to the EF135.

Future Directions

I need to now compare 100, 50, 28, and 16mm, but I'd like to incorporate any feedback I get from this test.

I want to adjust color temp on the white balance to make the images more white. I think this probably doesn't affect the results, but it was an oversight.

I do need to turn off in-body sharpening: these were taken from default JPG's which have a "sharpening" parameter set to "2". I don't see how this would necessarily give a bias, but I wonder if I'm overlooking something, and I certainly don't see how it could help.

Maybe I should use raw output instead of the high-quality large JPG's, but it'd be time-consuming. My guess is that JPG probably gives a small but fairly equal injury to all of the trials.

Arguably the 1 sec exposures are so sharp that I should test 2-4 sec even at 135mm.

All images are tested only in the center, which was reasonable for testing hand-holdability. I need corner tests and multiple apertures to compare lenses.

Not mentioned above but I think I saw a very marked difference between using the 2-sec timer and the regular drive mode on tripod (a Gitzo G1228 Mountaineer from 1998, with a Arca Swiss B1 ball head). I should do a specific study on how best to shoot from the tripod (IS on or off, shutter button vs. self-timer vs. using smart phone for a remote trigger).

Sometimes the fastest shutters counterintuitively seemed worse than slower. This may be due to the high ISO's used (up to 10k) but could be a subject for another study.

Images

I attach, in order: EF135, RF135, RF100-500, and finally RF135 with IS turned off. I trialed this lens down to 1 sec with ten trials at each speed, but there are fewer images as it literally couldn't be determined where the center of the chart was.
 

Attachments

  • EF135WithIS.jpg
    EF135WithIS.jpg
    2.1 MB · Views: 29
  • RF135WithIS.jpg
    RF135WithIS.jpg
    2.3 MB · Views: 32
  • RF100-500At135WithIS.jpg
    RF100-500At135WithIS.jpg
    2.1 MB · Views: 27
  • RF135NoIS.jpg
    RF135NoIS.jpg
    1.4 MB · Views: 28
Last edited:
  • Like
Reactions: 4 users
Nice test! Thanks for putting this up. It must have taken a long time to produce!

Some comments/questions
- What software did you use to estimate sharpness?
- I agree that adjusting the white balance would help visualizing, but didn't find it essential to this particular study
- I would turn off all in-camera adjustments and use jpegs. Using RAW and then processing in the computer will just generate a different systematic way of comparing, but I don't think it would be superior. I think what is important is to remove corrections / adjustments done by the camera (or the RAW processing in the computer) since these tend to make all lenses look similar as they attempt to "correct" imperfections.
- How did you determine the distance to the target for each lens? Just pick a size of the circle on the image sensor and then adjust the distance to the target so that all the lenses would, at least nominally through specs, produce the same image size on the sensor? What about MFD, does that play a role?
- Ideally, you should try to control ISO more carefully. That is, have a variable light source so you can keep ISO to a narrower range. 10K seems high to me and could potentially impact the results. 100-800 would be awesome, though 1600 should be fine. While higher ISOs should be ok for modern sensors, it is always good to have less variability.
 
Upvote 0
Nice test! Thanks for putting this up. It must have taken a long time to produce!

Some comments/questions
- What software did you use to estimate sharpness?
- I agree that adjusting the white balance would help visualizing, but didn't find it essential to this particular study
- I would turn off all in-camera adjustments and use jpegs. Using RAW and then processing in the computer will just generate a different systematic way of comparing, but I don't think it would be superior. I think what is important is to remove corrections / adjustments done by the camera (or the RAW processing in the computer) since these tend to make all lenses look similar as they attempt to "correct" imperfections.
- How did you determine the distance to the target for each lens? Just pick a size of the circle on the image sensor and then adjust the distance to the target so that all the lenses would, at least nominally through specs, produce the same image size on the sensor? What about MFD, does that play a role?
- Ideally, you should try to control ISO more carefully. That is, have a variable light source so you can keep ISO to a narrower range. 10K seems high to me and could potentially impact the results. 100-800 would be awesome, though 1600 should be fine. While higher ISOs should be ok for modern sensors, it is always good to have less variability.

- What software did you use to estimate sharpness?

I wrote some!

- white balance, turn off all in-camera adjustments and use jpegs

Thanks, that is my thought too!

- How did you determine the distance to the target for each lens?

I guessed the distance and printed a target and took a photo. It was not the right size so I adjusted the target and reprinted! Now I know how big a target is needed at that distance for 135mm (at "the end of my hall") I can make a 100mm target for the next test. For 50 28 16 I can't print big enough so will just make something closer, that still has 55lp/mm.

- Ideally, you should try to control ISO more carefully.

OK, I'll work on that! But a problem is that in a home environment, it's easy to change ISO but very hard to change the brightness in a 100% repeatable way. Still, if you look very closely at the diagrams, in 1:1, the ISO doesn't seem to be a huge problem. Compare 1/500 with the 100-500, probably f/5.6 and high ISO, with the bottom rows which were ISO100. Yes, more noise, but not enough to affect my sharpness algorithm I think. And, the main reason I wrote the sharpness algorithm was to sort the images from sharp to unsharp, but the number seems much better than I expected for comparing between shutter speeds, between ISOs, and between lenses.

Thanks for taking a look!
 
Upvote 0
One other thing to keep in mind is that the 1/FL rule of thumb was intended for enlargements no greater than 8x10 or 8x12 inches.

When one is pixel peeping at 100% on a 96ppi monitor (like a typical 23" FHD), looking at a 22MP file at 100% is like looking at a portion of a roughly 40x60 inch enlargement. Looking at a 45Mp file at 100% on the same monitor is like looking at a piece of a roughly 85x57 inch enlargement!
 
  • Like
Reactions: 3 users
Upvote 0
One other thing to keep in mind is that the 1/FL rule of thumb was intended for enlargements no greater than 8x10 or 8x12 inches.

When one is pixel peeping at 100% on a 96ppi monitor (like a typical 23" FHD), looking at a 22MP file at 100% is like looking at a portion of a roughly 40x60 inch enlargement. Looking at a 45Mp file at 100% on the same monitor is like looking at a piece of a roughly 85x57 inch enlargement!
When the high resolution sensors first came out we had posts recommending to modify the rule of thumb by a factor of 2 or so. And if you are cropping, the rule is hopeless. For much of what I do, say using a 4 mpx crop from 45 mpx, I need a shutter speed of faster of 1/250s even with 5-stops of IS for tack sharpness. Although, Topaz can work wonders.
 
Upvote 0
When the high resolution sensors first came out we had posts recommending to modify the rule of thumb by a factor of 2 or so. And if you are cropping, the rule is hopeless. For much of what I do, say using a 4 mpx crop from 45 mpx, I need a shutter speed of faster of 1/250s even with 5-stops of IS for tack sharpness. Although, Topaz can work wonders.
If you have time, study the charts I'm posting pretty carefully. On one hand I'm finding that the R5 is really giving 8 stops of IS with IS lenses, not a lie! Even with the EF135/2, it seems like more than 6 stops. But yes, modern lenses and sensors really can be used even when throwing away 90% of the pixels. In that case, sure, 1/250 with IS is far better than no IS.
 
Upvote 0
If you have time, study the charts I'm posting pretty carefully. On one hand I'm finding that the R5 is really giving 8 stops of IS with IS lenses, not a lie! Even with the EF135/2, it seems like more than 6 stops. But yes, modern lenses and sensors really can be used even when throwing away 90% of the pixels. In that case, sure, 1/250 with IS is far better than no IS.
I do appreciate your charts and have studied them. I have done many myself with my lenses and different conditions appropriate for what I do and for my own degree of hand twitch and shake. I am using focal lengths of 400-1000mm at longer distances than you do, usually 20m. Here, https://www.canon-europe.com/pro/infobank/image-stabilisation-lenses/ Canon points out specifically that you get up to 8 stops only with wider angle lenses that have a large image circle. They say that for longer lenses it's pitch and yaw that are giving the blurring and the IBIS is not going to add much to the lens IS and you won't get 8 stops, the experience is maybe a stop added on.
 
Last edited:
  • Like
Reactions: 2 users
Upvote 0
I do appreciate your charts and have studied them. I have done many myself with my lenses and different conditions appropriate for what I do and for my own degree of hand twitch and shake. I am using focal lengths of 400-1000mm at longer distances than you do, usually 20m.
I've done similar back around 1999-2000, shooting on film using a loupe or later scanner. I've done one or two attempts at testing on digital bodies but by and large I already knew the lenses well enough, and I didn't hit on the right idea for testing. But I used the EF outfit I put together in the 90s pretty much unchanged for 20+ years and knew it all very well. (14, 24TS, 35/1.4 MkI, 50/1, 1.2, 1.4, and the 1.8 MkI !, 85/1.2 MkI, 135/2, 180Mac, 600/4, trinity zooms and the EF24-105 MkI)

I got the RF24-105 with an R in 2019, but the rest of the lenses only in the last year and haven't been able to shoot much at all for various reasons. So I've got a dream outfit here and realized I need to really study its capabilities to know how to use it best. Step 1 was realizing that probably everything I know is wrong.

One fun fact I'll publish evidence for here later is that tripod shots with immediate shutter release look a LOT worse than with the 2-second timer. I never would have realized that without these tests.

Here, https://www.canon-europe.com/pro/infobank/image-stabilisation-lenses/ Canon points out specifically that you get up to 8 stops only with wider angle lenses that have a large image circle. They say that for longer lenses it's pitch and yaw that are giving the blurring and the IBIS is not going to add much to the lens IS and you won't get 8 stops, the experience is maybe a stop added on.
Interesting. Well, I'm not a birder, but up to 135mm at least my tests show a true 8 stops:

1 sec IS has about the same success rate as 1/250 no-IS
1/2 sec IS has about the same success rate as 1/500 no-IS
1/4 sec IS has about the same success rate as 1/1000 no-IS

I have only done the tests on 100mm and 135mm so can't yet speak for other focal lengths from experience or tests.

However, the narrower the field of view, the more that the effect of pitch and yaw resembles the simple translation the moving sensor can do. In contrast say 14mm makes the apparent angle of objects away from the center change drastically, and that's not something that can be addressed in-lens OR with IBIS. So while I'm happy to take Canon's word for this, I also don't know, as an engineer, how what they're saying could be quite true.
 
Last edited:
Upvote 0
[...]One fun fact I'll publish evidence for here later is that tripod shots with immediate shutter release look a LOT worse than with the 2-second timer. I never would have realized that without these tests.
I use my phone as bluetooth remote when I use the camera on a tripod, especially when the camera is at a funny angle. The Camera Connect update from a few weeks ago made the process almost frictionless, I highly recommend giving it a try if you are doing a lot of finicky tripod work.

Using the phone over wifi to get a live view is still an exercise in frustration, though.
 
  • Like
Reactions: 1 user
Upvote 0
Using the phone over wifi to get a live view is still an exercise in frustration, though.
Funny, this worked great at home but I tried to use it with my Camera way up in the air on a monopod, to shoot the huge mass of commuters going through Shinagawa Station. And I think it was the fact that there were about 25+ people a second entering wifi range with their cell phones, and a like number leaving, that made the connection nearly impossible. That was with the R though.
I use my phone as bluetooth remote when I use the camera on a tripod, especially when the camera is at a funny angle.
Yes, I've done that in the past but have gotten out of the habit. Or more accurately, habits formed over decades, of using the 2-sec timer, kind of took over. I still have the angle finder viewer for the EOS-1N. Arguably it still made sense even for 1Ds MkIII (with a real viewfinder, and without a tilting display) but I might as well toss it at this point.
 
Upvote 0
Funny, this worked great at home but I tried to use it with my Camera way up in the air on a monopod, to shoot the huge mass of commuters going through Shinagawa Station. And I think it was the fact that there were about 25+ people a second entering wifi range with their cell phones, and a like number leaving, that made the connection nearly impossible. That was with the R though.[...]
The combination of using a low powered radio in the camera to talk to another low powered radio in your phone limits the range to about 4 meters with a direct line of sight. I used the wifi remote this weekend to take a family picture and it worked great till I tried hiding the phone behind my back, it instantly stopped responding.
The R5 seems to have a larger usable range than my M6II, but not much larger.

This live-view-to-phone feature is one of the things I hope Canon will improve on, it is currently much easier and faster to mount my phone on the tripod and use my watch for live view and triggering than using a 'proper' camera and phone. Having the Camera Connect app support smart watches for triggering would be a big improvement.
 
  • Like
Reactions: 1 user
Upvote 0
One fun fact I'll publish evidence for here later is that tripod shots with immediate shutter release look a LOT worse than with the 2-second timer. I never would have realized that without these tests.
One important thing to know about IS systems is that they takes ~0.5 s to achieve full stabilization, and during that period between IS activation (e.g. with a half shutter button press) and full stabilization, the IS system actually makes the resulting image worse. So in a situation where the lens is pre-focused and you're just pushing the shutter fully down in one motion to take the shot, the IS system is working against you. If you're going to 'mash the shutter button' to take a shot, you're better off with IS turned off.
 
  • Like
Reactions: 1 user
Upvote 0
This live-view-to-phone feature is one of the things I hope Canon will improve on, it is currently much easier and faster to mount my phone on the tripod and use my watch for live view and triggering than using a 'proper' camera and phone. Having the Camera Connect app support smart watches for triggering would be a big improvement.
Being able to trigger a Canon camera with my Apple Watch, as I can currently trigger my iPhone, would be great. If it was reliable, that is. I haven't bothered with the Camera Connect app, mainly because in my experience Canon software is all kludgy and unreliable. I'm looking at you, DPP (and also the UI on my little Selphy printer, where I basically gave up and just copy pics to an SD card to print them, and even then half the time the preview images don't show on the little LCD even though the pics print fine).

Am I missing out on something good with the Canon iPhone app?
 
Upvote 0
Being able to trigger a Canon camera with my Apple Watch, as I can currently trigger my iPhone, would be great. If it was reliable, that is. I haven't bothered with the Camera Connect app, mainly because in my experience Canon software is all kludgy and unreliable. I'm looking at you, DPP (and also the UI on my little Selphy printer, where I basically gave up and just copy pics to an SD card to print them, and even then half the time the preview images don't show on the little LCD even though the pics print fine).

Am I missing out on something good with the Canon iPhone app?
The app has been improved immensely with the last update, but it is still kludgy and unreliable. But less bad than before, which is a very low bar to clear.
 
  • Like
Reactions: 2 users
Upvote 0
Funny, this worked great at home but I tried to use it with my Camera way up in the air on a monopod, to shoot the huge mass of commuters going through Shinagawa Station. And I think it was the fact that there were about 25+ people a second entering wifi range with their cell phones, and a like number leaving, that made the connection nearly impossible. That was with the R though.

Yes, I've done that in the past but have gotten out of the habit. Or more accurately, habits formed over decades, of using the 2-sec timer, kind of took over. I still have the angle finder viewer for the EOS-1N. Arguably it still made sense even for 1Ds MkIII (with a real viewfinder, and without a tilting display) but I might as well toss it at this point.

I've also found that even with the phone sitting right next to the camera they have trouble connecting in an area with a high density of networks/devices. Walk 20-30 feet outside where there is no one standing around and everything connects as it should.
 
  • Like
Reactions: 1 user
Upvote 0
One important thing to know about IS systems is that they takes ~0.5 s to achieve full stabilization, and during that period between IS activation (e.g. with a half shutter button press) and full stabilization, the IS system actually makes the resulting image worse. So in a situation where the lens is pre-focused and you're just pushing the shutter fully down in one motion to take the shot, the IS system is working against you. If you're going to 'mash the shutter button' to take a shot, you're better off with IS turned off.
How does mode 3 of IS whereby IS comes on only when you press the shutter manage to work?
 
Upvote 0
For general sharpness and LOADS of other specs I’ve used Reikan Focal for many years, that is very easy to determine sharpness , the sharpest aperture, focus consistency and accuracy etc. I stopped using it when I went mirrorless, since I don’t need it for AFMA. Would be interesting to compare the results with your test too.
 
  • Like
Reactions: 1 users
Upvote 0
For general sharpness and LOADS of other specs I’ve used Reikan Focal for many years, that is very easy to determine sharpness , the sharpest aperture, focus consistency and accuracy etc. I stopped using it when I went mirrorless, since I don’t need it for AFMA. Would be interesting to compare the results with your test too.
FoCal basically uses edge sharpness rather than resolution to measure whether focus is correct. For lenses it would be more the equivalent of measuring the 10 lp/mm MTF for contrast than the 30 lp/mm for resolution. I used to use it routinely and frequently for AFMA with DSLRs, where it is excellent, and continued to use it for a while for mirrorless with downloaded jpegs to see how say, for example, edge sharpness compared with centre or for reproducibility of focus.
 
  • Like
Reactions: 1 user
Upvote 0
For general sharpness and LOADS of other specs I’ve used Reikan Focal for many years, that is very easy to determine sharpness , the sharpest aperture, focus consistency and accuracy etc. I stopped using it when I went mirrorless, since I don’t need it for AFMA. Would be interesting to compare the results with your test too.
Reikan Focal looks interesting... I'll bookmark it. What's AFMA?

From my 90s lens collection, sharpest aperture always seemed to be two stops down. But now, the 135/1.8 and 100-500 (at least at 100-135) are so sharp wide-open that I think a 45MP sensor can't even make use of any more resolution. That is to say, they're already delivering just about as good a result as the sensor can sense wide-open. Maybe on 65MP or 100MP you could detect them getting sharper yet, but on an R5 possibly not.
 
Upvote 0