Background
I want to understand how long I can hand-hold my lenses. Traditionally we used a rule of thumb: the "reciprocal rule," stating that you should use a shutter speed at least as high as one divided by the focal length. For instance, a 90mm lens would call for a 1/90th second exposure or better. But it was never that simple: it's not a question of "sharp" vs. "not sharp," but rather "how sharp" and even "what percentage would be how good or how bad." Further, this advice came out when even 400-speed film was grainy, focus was never perfect, and lenses weren't that sharp anyway. Quality of the image just wasn't great, and even a fair amount of unsharpness from a moving camera would be hidden by these other greater factors that are now gone. It's possible that the reciprocal rule is now way too optimistic, given the high-megapixel, low-noise sensors with today's excellent lenses and perfect autofocus. On the other hand, the advent of image stabilization changes everything too, with makers claiming unbelievably great-sounding improvements in hand-holdability, that I was frankly dubious of.
I also bought the new RF135/1.8IS, to replace my beloved but ancient EF135/2.0, and want to characterize what my money's buying in terms of image stabilization as well as general sharpness. (Other benefits are slightly increased bokeh, and weatherproofing.) My camera, as most high-end cameras now, has in-body image stabilization via a sensor that the camera can move slightly to offset detected movement, and Canon especially has put additional stabilization in most cameras that can move lenses to compensate.
Finally, I'd like to know how my various lenses compare: if I have three lenses that can shoot at 100mm or 135mm, which should I use on a shot? Am I safe leaving lens X or Y home, or should I really take it to get best results? Until 1995 you could be sure that any prime lens would be sharper than any zoom, but that started changing, and by 2020 the more moderately-apertured pro-quality zooms seem outstanding. But have primes continued to improve too?
The Test
For these reasons I wanted to do a comparison. I have multiple lenses to choose from at focal lengths 16mm, 28mm, 50mm, 100mm, and 135mm, so these are the focal lengths I'm personally keen to test. However the 135 was my most recent acquisition so was on my mind first.
Here are four comparison charts I've made, each based on shooting ten trials at each of ten or so shutter speeds. The target shown was from the center or close to it in each shot. Each line of each chart has a benchmark tripod image from the 135/1.8 at the beginning and end of the line. The ten trials at each shutter speed are sorted from sharpest to least-sharp. Software attempts to score the sharpness to automate this, though it's only a guideline. The score is written on each target again, it's just a guideline; let your eyes be the judge. The squares are 225 pixels wide, which is basically 1mm on the sensor (227 pixels per mm I think). The data is 1:1 scale and meant to be viewed shrunken to fit, to get an overview, and at 1:1 to study individual results. Some lenses don't show ten images: the missing ones were too blurry to process.
The test target has 55lp/mm, the lines of which are about 2.05 pixels tall on the sensor of a Canon R5. I chose this as thinner lines cannot be depended to produce a black pixel. (Say, a 1.5 pixel wide line could fill two adjacent pixels 3/4 black, so even if the lens were perfect you wouldn't have black. In contrast 2 pixels tall will always leave one pixel seeing nothing but black, and possibly two if it just happens to be aligned.)
Findings
This is a severe test conducted to differentiate even between perfect and "merely" excellent. The main point is that, of the IS images,
even the worst are fantastic in an absolute sense, even the worst of the 1 second images on the worst lens here. The image quality obviously sucks at 1 second for the IS images compared to what they can do at faster speeds. And yet, each black and white line is 2 pixels wide. If you reduce size by 4 (say, to 2000 pixels width), the circles will literally be uniform gray and NONE of the blur shown here will even be visible, even looking pixel by pixel at such an image. Your two takeaways from that are:
- Even the 135/2's apparently bad results will look nearly identical to the 135/1.8 if you're using <=2000 pixel-wide images.
- Even 1 second handheld exposures will look nearly identical to tripod benchmarks on any of these lenses if you're using <=2000 pixel-wide images.
At 1/30, even the worst trial from the RF135/1.8L IS or RF100-500/4.5-7.1L IS is sharper than the very best image at any speed from the EF135/2.0L. When I got the EF135 in 1997, it seemed to be by far the sharpest lens I had, definitely the sharpest non-white lens until the 180/3.5Macro came out. So, I'm saddened yet amazed to see it's being out-shot by a zoom lens hand-held a relative eternity.
At 1/60, even the worst trial from the RF135/1.8L IS is sharper than the very best image at any speed from the RF100-500/4.5-7.1L IS.
At 1/4, the RF135/1.8L IS with IS on outperforms the same lens with no IS at any speed including 1/1000. I would therefore say it is quite accurate to say IS really is giving 8 stops of stabilization. Likewise 1/2 with IS outperforms 1/500 no IS (again 8 stops). 1 sec with IS outperformed 1/250 no IS in 6 out of 10 samples or so, so yet again 8 stops.
While the EF135/2.0 lacks in-lens stabilization, it doesn't seem to be held back too much: it's maybe 0.5-1.5 stops "less hand-holdable" than the RF, perhaps, if you compare the really long exposures. So don't worry about the lack of in-lens IS if your budget only stretches to the EF135.
Future Directions
I need to now compare 100, 50, 28, and 16mm, but I'd like to incorporate any feedback I get from this test.
I want to adjust color temp on the white balance to make the images more white. I think this probably doesn't affect the results, but it was an oversight.
I do need to turn off in-body sharpening: these were taken from default JPG's which have a "sharpening" parameter set to "2". I don't see how this would necessarily give a bias, but I wonder if I'm overlooking something, and I certainly don't see how it could help.
Maybe I should use raw output instead of the high-quality large JPG's, but it'd be time-consuming. My guess is that JPG probably gives a small but fairly equal injury to all of the trials.
Arguably the 1 sec exposures are so sharp that I should test 2-4 sec even at 135mm.
All images are tested only in the center, which was reasonable for testing hand-holdability. I need corner tests and multiple apertures to compare lenses.
Not mentioned above but I think I saw a very marked difference between using the 2-sec timer and the regular drive mode on tripod (a Gitzo G1228 Mountaineer from 1998, with a Arca Swiss B1 ball head). I should do a specific study on how best to shoot from the tripod (IS on or off, shutter button vs. self-timer vs. using smart phone for a remote trigger).
Sometimes the fastest shutters counterintuitively seemed worse than slower. This may be due to the high ISO's used (up to 10k) but could be a subject for another study.
Images
I attach, in order: EF135, RF135, RF100-500, and finally RF135 with IS turned off. I trialed this lens down to 1 sec with ten trials at each speed, but there are fewer images as it literally couldn't be determined where the center of the chart was.
www.dpreview.com
