….and other Myths
Welcome aboard Roger Cicala from LensRentals.com. He has been kind enough to allow us to republish some of his fine technical articles. You’ll find his writing easy to read, often humorous and generally right on the ball.
I chose this article first because I am constantly asked about this phenomenon. It’s also talked about quite frequently on message boards.
So sit back and enjoy the read.
Part 1 – Written in 2008
One of the most common examples of anti-logic we see at LensRentals is the statement The lens is soft/frontfocuses/backfocuses. Now don’t get me wrong, there are bad copies of lenses out there, as best we can tell ranging from 3% to 7% of lenses. And we know, despite our checkout procedures, that 1 of 400 lenses or so will be damaged in shipping and arrive not functioning. Sometimes there’s actual damage or misalignment of an element in the lens, although the vast majority of the time that’s not the case. Usually the subject of the photograph is soft because the lens is not focusing precisely.
Three to 4 times a week we have the following conversation:
“The lens you sent me frontfocuses, its not good.”
“OK, we’ll overnight you a replacement.”
Then the first lens comes back and its perfectly fine when we check it out. But the customer is very happy with the replacement lens, it worked great even thought the first one didn’t. So what has happened? Its rather simple, actually, and like most examples of anti-logic it stems from a wrong assumption: the customer knows his/her camera is ‘fine’ because it works with fine with their other lenses—none of them front focus or back focus.
The key to the puzzle is the definition of ‘fine’. Most people assume that ‘fine’ means ‘perfectly calibrated’. In reality cameras are like any other manufactured item, calibration is within a given tolerance range. We don’t have privvy to what the actual tolerance range Canon, Nikon, or the other manufacturers (except Zeiss and Leica) consider acceptable, so lets arbitrarily say the manufacturer will consider a camera or lens to be ‘in specifications’ if its + or – 3 ‘focus units’ from perfect. We can assume they reached this number because anything within + or – 3 focus units will be within the depth of field of a wide aperture (probably f/2.8) lens.
Lets consider that I have a camera body that is -2 focus units from perfect, and a lens that is +2 focus units from perfect. Both are considered ‘fine’ according to the manufacturers definition, although they certainly aren’t perfect. However, the combination of a +2 lens on my -2 camera will be absolutely perfect, I’ll love the lens on my camera . After my experience with this one lens on one camera, I will write Sonnets on the various online forums about how great it is, and will tell anyone who doesn’t like it that they must be a bad photographer. I will have become the most dreaded online lifeform, a FLAO (Fanboy with Loss of All Objectivity).
But what if the lens was -2 focus units from the theoretical perfect and I put it on my -2 focus units from perfect camera? Well it depends. If the lens is say an f/4 maximum aperture, probably not much: the depth of field from an f/4 aperture lens may well mask a bit of front focusing or back focusing. You might notice the lens frontocuses 3 feet in front of the subject at 20 feet if you pixel peep, but since the depth of field is 10 feet the subject is still in focus and the lens seems fine. I will probably describe the lens as very good, but not descend to complete FLAOdom.
But if its an f/1.4 lens with a very shallow depth of field, the front focusing will be noticeable: the subject will be out of focus and soft. If I know how to do a front/backfocus test I may have figured out the problem, but here’s the kicker: if I sent the lens in to the manufacturer to fix the problem they would check the lens out, say it was fine (because it is fine, its within specifications) and send it back. Ony if I send the camera and lens together to be calibrated would the fact that the two together are out of focus be apparent, and then the manufacturer would be able to fix the calibration.
Ah, but there’s no free lunch. If the camera calibration was adjusted as part of the fix, I might find that another lens in my kit that used to be great, now backfocuses a bit. In the past, many full time pros who were aware of these issues, would send their entire collection of cameras and lenses to the manufacturer to be calibrated together. This was one of the original reasons Canon and Nikon formed their Professional Services groups. Most of the rest of us just made do, or sent copy after copy of a given lens back until we got one that was sharp ON OUR CAMERA.
The bad thing is many, many people who did this then hopped on their online camera forum and made blanket statements like “I had to try 3 copies before I found one that was calibrated right”. In reality what they should have said was “I had to try 3 copies before I found one that was calibrated right FOR MY CAMERA”. Those other two copies might well have been fine on someone else’s camera.
When you have a few dozen copies of each lens and each camera like we do, you quickly find out this is just a fact of camera reality. And the funny part of all this is the more expensive wide aperture lenses are the ones most likely to show the problem, because their depth of field is so narrow and the in-focus portion of the picture is so sharp compared to the out of focus portion. That $200 f/5.6 zoom is not going to show a minor front focus problem because the depth of field is about half a mile. The $2,000 f/1.4 prime has a depth of field of a few inches and any problems are immediately evident (and the owner 10 times more invested in wanting a perfect lens).
The good news is newer cameras have taken all this into account and the fix is right at your fingertips. The following cameras all have a “lens microcalibration” feature: Canon 1DMkIII, 1DsMkIII, 5DMkII, 50D; Nikon D3, D3x, D300, D700; the Pentax K20D, the Olympus E-30 and E-620, and the Sony A900. I’m surprised at how many people don’t take advantage of this feature – its a bit time consuming to do, but once done each of your lenses is locked in the camera’s memory and it will automatically compensate so that each lens is at a nearly perfect focusing plane whenever you mount it on the camera. I find the feature makes such a huge difference for most of my better lenses that I consider this feature alone makes the upgrade to one of the above bodies worthwhile.
Bad lenses (and cameras) will still exist, but the vast majority of front and backfocus issues will be a thing of the past. And for those of you who don’t have this feature, we will continue, as we always have, to do our best to get you a lens that works great on your camera, even when it means sending a replacement.
Addendum I recently saw the greatest real life example of this ever, in an online forum where the poster states ’Canon’s New XX camera sucks’ (I’m eliminating names so the bots don’t pick this up and repeat it.) He goes on to say he had a body for several years, and a hand picked collection of lenses that he knew were perfect because he’d gone through several copies of each to get the sharpest one. Now he bought a new body and all his lenses sucked, and he’d now exchanged bodies twice and they still all sucked. So here is the perfect example of a person starting with a camera at the edge of tolerance, choosing through multiple selection a set of edge-of-tolerance lenses, and now generalizing that all the new bodies suck. The sad part is the new body has microfocus adjustment and he never even tried it. Just sent copy after copy back to the store.
Part 2 – Written in 2010
So what is different since 2008?
Camera sensors, as they always do, have increased in resolution, magnifying any flaws in the lens. What might not be noticeable on an 8 Mpix crop sensor can be frighteningly obvious on a 21 Mpix full frame camera.
Additionally, since 2008 a number of world-class wide and ultra-wide angle lenses have become available: The Zeiss 21, Canon 14 f/2.8, Canon 17 TS-E, and Nikon 14-24 f/2.8 among others. A telephoto lens has a narrow angle of view—maybe 5 or 10 degrees—which means the light rays don’t have to be bent a lot before they get to the sensor. The angle of view of a wide angle lens can be 80 degrees or more so the light rays, especially from the sides and edges have to be bent a great deal before they get to the sensor. A slight difference in the way refracts light from one side to the other is much more noticeable in a wide angle compared to a telephoto lens. With a full-frame sensor any problems are even more apparent.
I’m not trying to say that wide angle lenses are more prone to problems than other lenses. It’s simply that some recent observations were first made with wide angle lenses because they were more obvious with them. Until recently, wide angle lenses tended to be soft in the edges and corners and people just accepted it. Recent lenses are so much better that differences from one side to the other became noticeable (the difference between sharp and soft is more obvious than the difference between soft and softer, apparently).
Finally, people have started to accept that copy-to-copy variation occurs in both bodies and lenses, to understand that manufacturing tolerances are just that: a range of acceptable values, not an exact point. In other words, what is specified as a 1/4 inch diameter screw may be anything between 0.247 inches and 0.253 inches in diameter (1, 2 ). The machines that make them can’t be more accurate than that. Additionally, the screw diameter will vary slightly with temperature, etc. Nothing can be made exact at reasonable cost.
Given that people accepted that there must be some variation, a few lens testers and reviewers started looking for it. We all expected to find some random variation: a group of copies would all be a little different, in different ways. Some early results, though, were surprising in regards to how and where variation might occur.
Some outside information
We have the privilege of working fairly closely with some excellent reviewers and testers, and get to discuss their results with them. But everything I’ll be mentioning here is available for you to read from the original source and I’ll link to their articles. But full disclosure first: Lensrentals furnishes some lenses for testing to both SLRGear and to Lloyd Chambers and have advertised on Diglloyd, SLRGear and TheDigitalPicture We’re rather picky in who we tend to associate with and have, in all three cases, chosen to affiliate with these guys because we respect the work they do. But I think the internet these days is way too full of business relationships that aren’t mentioned by bloggers and writers, so I want to make sure I mention these. Now on to their observations:
Lens variation isn’t always random
I was totally shocked by this. I expected that variation between copies of lenses would be totally random. If someone tested a hundred copies of a lens, I would expect equal numbers would front or back focus. Equal numbers would be softer on the left side and the right side. And most would focus properly and not be soft on either side. But some early evidence is suggesting this isn’t so. SLR gear recently tested 5 copies of the Nikon 50 f/1.4 and found some minor, random variation. When they tested 5 copies of the Canon 50mm f/1.4, all fairly new from our stock however, they found all 5, to varying degrees, were softer on the right side compared to the left side. Five out of five softer on the right isn’t random variation.
While not studied, there has long been a lot of talk that certain lenses had “bad batches”, or that manufacturer’s made “silent upgrades” to certain lenses, improving quality. Certain Canon date codes on a few lenses are unpopular because people have reported lots of trouble with just that date code (no slam on Canon, at least they put a date code on their lenses). Other lenses are widely believed better if you get a later copy, the early ones had problems.
Our own experience supports this. With some hesitation we recently tried a few copies of the Sigma 50-150 f/2.8, tested them, rented them for 6 months, and decided we would stock them in quantity, since they seemed great. We ordered 6 more copies and all 6 were horrible, incredibly soft, and went straight back. The Digital Picture found exactly the same thing. We assume (and assumptions are dangerous) that the second set of six came from a different assembly line run than the first few we bought. We’ve had similar experiences with other lenses we bought in batches. For example, of the initial dozen Sigma 150-500s we bought, almost all failed very early and we stopped carrying them. Lots of people at that time reported similar things, but there’s a lot of folks saying that more recent copies don’t have those problems. The Canon 300 f/4 IS was one of our most trouble-free lenses, but we purchased 6 copies last December, of which 4 developed electrical problems. None of about 30 other copies bought earlier or later had that problem, we only saw it in that one batch of lenses.
There’s no science here, but the phenomenon has been reported so often that I do think where there’s smoke, there’s fire. I think getting 4 of 4 good copies in one shipment and 6 of 6 bad copies of the same lens in another isn’t coincidence. I don’t believe that 4 of 6 in one batch developing electrical problems, while none of nearly 30 other copies did, is a coincidence, either. Call me paranoid if you want to.
Camera body variation also occurs
Take a second and look at the heavy, industrial-strength lens mount on your camera, and the one on your lens. The lens, mounted to the camera, is supposed to be at exact right angles to the sensor. When Lloyd Chambers was testing multiple copies of a very sharp, wide-angle lens on a very high resolution camera he found that the right side was consistently blurred on all but one copy. On a different camera, the left side was consistently blurred (all but two copies). Further testing with different cameras and lenses showed similar findings. The end result is the discovery that on high resolution cameras, with high-quality wide-angle lens (because side to side variation is much more apparent with a wide angle lens) a very small difference in alignment between the lens mount and the sensor will cause this kind of problem.
How small of a variation? One of his sources said as little as 20 microns (0.02mm or 0.0008 inches) is sufficient to cause side to side variation. You can only detect that amount of variation with laboratory grade laser equipment, I’m told. Medical-grade machine parts (used in arthroscopes, etc.) are expected to have tolerances of about 50 microns, (3) and it would seem unlikely that a camera lens mount would be made more than twice as accurately as a medical arthroscope. In other words, with a top quality wide-angle lens on a high resolution sensor, we can perceive a 20 micron difference, but the manufacturer probably can’t make the part more accurately than +/- 50 microns at a reasonable cost.
It’s not just major manufacturer’s lenses
Before the fanboys among you gear up (pun intended again) to use one sentence out of this to claim your brand is better than their brand (I actually read a psychological study on why people feel the need to do that, perhaps another article some day, but the summary is: it says a lot more about the fanboy than it does the product in question) there’s plenty of evidence that there is variation among every brand. Joseph Holmes wrote a nice article about variation in expensive Medium Format lenses. They seem to have at least as significant a degree of sample-to-sample variation as do SLR lenses, despite costing a lot more. Even Zeiss (who’s quality has been clearly excellent, although perhaps somewhat because manually focused lenses are simpler to assemble) apparently will use hand select optics for their new Compact Prime II Cine lenses compared to the standard ZE lenses (source Vincent Laforte ) which I assume means tighter tolerances in these more expensive ($6k) lenses.
Let’s take a logical look at the manufacturing process
Have you ever watched Canon’s very nice video of how a lens is made ? Or read Zeiss’s article on making lenses? Did you notice how about 90% of it is about the glass elements? That’s interesting, of course, and after looking at them I believe the various glass elements in the lens have superb quality control and are as perfect as they could be. I also believe, having read a lot about it, that lens design today is far better than it’s ever been. But do you notice they don’t show you anything about how the lens barrel, or helicoid, or circuit boards are made. Or where they are made? Since, as best I’ve been able to find out, none of the lens manufacturers have metal foundries or machine lathes, I’m left to assume all those metal parts, and probably the circuit boards, are made by subcontracting manufacturers.
We know every manufacturer outsources some of their parts, and the outsource suppliers are rarely known. I would assume that also means suppliers for all those metal parts that hold the lens elements in supposedly perfect positions put in bids and supply the parts to the manufacturer for later assembly. The subcontractor may change, the machine tools may wear out, and one batch of helicoids may be slightly different than another batch, or one set of circuit boards more prone to short out. Perhaps this explains why one batch of a given lens seems to have problems while other batches seem to do fine. It may also explain the much speculated “silent upgrade” event (when people claim that lenses that used to have a high problem rate, no longer do because the manufacturer supposedly did some kind of ‘silent upgrade’). The “silent upgrade” may simply be that a subcontractor of a part was simply replaced by a different subcontractor that made a better part. Recently, for another example, Nikon’s 70-200 f/2.8 VR II lens has been noted to have small metal sparkles (shavings? flakes?) which Nikon states are “air holes remaining in the metal portion of the barrel in the process of component production”. We all know they won’t be there in the next batch through the assembly line. We’ll never know if that’s because the subcontractor is making better barrels or maybe Nikon got a better subcontractor.
But, even if everything is as good as it can be, I believe we now have camera sensors and optical designs that are simply too good for the current standards of mass-production. Let’s assume the source that stated a 0.0008 inch difference in alignment between the lens mount and sensor can make a visible difference in side-to-side sharpness of a 21mm lens, assuming the lens and sensor are capable of very high resolution. (by the way – I know the source, I trust the source, but I’m not at liberty to name the source.) Look at your camera mount and the lens mount, at the little screws that hold it in place. Think about dangling your 70-200 f/2.8 lens on that mount for a few days. Do you really think the mount is placed that accurately on the camera? Or even that it’s machined that accurately? I don’t believe it’s possible.
And inside each lens are up to 16 elements, some of which slide up and down a spiral grove to focus, others to zoom. How far tilted from one side to the other would one element have to be to make a noticeable difference? Or how slightly off true must the internal barrel be to cause some similar effect? I have no clue, but I’d bet it’s not much. So let’s think back on why would one batch of 50 f/1.4 lenses, bought at nearly the same time, all be softer on the right side? I’d guess because a shipment of lens barrels or helicoids or some other internal part was out of spec and all the lenses assembled using that part have an element that’s slightly forward on the right side compared to the left. Why would one group of lenses be likely to suffer electronic failure, when other copies of the same lens rarely do? Probably a batch of bad circuit boards, perhaps a batch of bad solder affecting one run of the assembly line.
In the days of film the plane of film in the back of the camera wasn’t nearly as flat, and was a much more variable distance from the back of the lens than is a digital sensor—so such minor differences probably didn’t matter. Even with low resolutions sensors, the camera couldn’t detect these small variations in lens and lens mount assembly. With full-frame, high resolution sensors and cutting edge optical design, particularly for wide-angle lenses, they have now become apparent.
To summarize, these points seem pretty obvious to me. I’m not certain they’re all completely correct, but I bet they’re all pretty close to reality.
- The mechanical parts that are assembled to form a lens, lens mount, and sensor are going to vary a bit with every lens and every camera.
- This variation will cause every copy of a lens, and every copy of a camera body, to have slightly different characteristics.
- A lens may be fine on one camera and not another. A camera may do fine with one lens and not another.
- Some lenses (and cameras) will be far enough out of spec to just suck, no matter what they are mounted to.
- It seems logical that ‘bad batches’ can occur because a shipment of one or more parts is defective and not caught during routine testing (or the manufacturer decides it’s cheaper to ‘ship and repair’ than to hold a shipment).
- When the manufacturer knows about a “bad batch”, they probably identify the problem and correct it for future lenses, but they aren’t going to announce it unless they absolutely have to – when something is so bad it’s affecting overall sales of that item. Roger’s Rule of Problem Announcements: Once its announced that 5% of lens X has a certain problem, 50% of the members of any online forum will announce their lens has the problem. Whether they own lens X or not.
- Of course future batches aren’t necessarily better, just different. Problem A may have been fixed, but the new supplier of part 32543 may make a bad batch, or the machine tools used to lathe the last set of part 2433 may have become more worn and less accurate.
First, the practical conclusion. Test every new lens you buy, right out of the box, and make sure it’s reasonable. Speaking as someone who has lots of test charts and moderately fancy equipment to do that quickly and efficiently, I’ll just say you don’t need it. Just find a couple of brick walls, fences, etc. that are in a flat plane and take some pictures. A bad problem will be evident very quickly. If the lens is bad, send it back to the store for exchange, not in to the manufacturer for repair (Because certain manufacturers are very likely to say “impact damage” and deny warranty even if you just unpacked it. No, I won’t name names, I don’t have strong enough evidence to say one does this more than another.). Which brings up another point: don’t buy anything from a place that won’t take it back. But don’t confuse the need for autofocus micro-adjustment with a bad lens, it’s not.
It is a bad time to be a pixel-peeper. If you look closely enough on a high resolution full-frame camera, chances are you’ll find some minor flaw with every lens you own. If you go through 13 copies and get one that’s just perfect, remember to never, ever upgrade your camera body, because it probably won’t be perfect on the next one. That’s just how it is: manufacturing processes are not up to the lens design and sensor resolution we have at the moment. Put on your Batman undies and cope with it. Or shoot with a Holga for a while and get over it. And stop looking at 100% images on your monitor: not only will it make you go blind, 50% screen resolution is more resolution than your printer can reproduce anyway. Unless you print billboards designed to be viewed from 5 feet away.
I suspect that the manufacturers are much more aware of this stuff than I am. Of course, their first response is going to be “let’s hope nobody notices”. But I’m sure they’re also looking at cost-effective ways to improve manufacturing tolerances and reduce the issue. By the way – for those of you who are screaming “for $2,000 my zoom lens should be perfect” let me introduce a bit of perspective. A nice, cinema quality (read the best it can possibly be) prime lens runs anywhere from $8,000 to $25,000, while a top quality cinema zoom runs from $25,000 to $70,000. And even at that price a technician will need to spend an hour or so adjusting the lens to the camera that’s using it to get the absolute best results. The best SLR lenses don’t even approach the quality (or quality control) of an entry level cinema lens. Not to mention you probably don’t have a full-time camera tech.
As an aside, I think this is probably going to put the last nail in the Megapixel war’s coffin. The 4/3 companies have already said 12 Mpix is as far as they intend to go. I suspect the full-frame manufacturers are going to call a halt at 30 Mpix or so, just because there’s no sense in it: they’re already out-resolving the quality control of their best lenses.
P.S. – someone wrote and said: “Roger, you’re always trying to predict the future, so what’s your future prediction about this?” okay, here it is: I don’t believe better manufacturing tolerances are likely at reasonable cost. What I think will be done is we’ll take advantage of the computer software programs that are already being used to correct for known lens aberrations. Panasonic does this with the 20mm f/1.7 (4) and Hasselblad does something similar with their new H4D. I could picture the last step on the assembly line being shooting a laser target and programming correction into the lens or bodies EPROM that will compensate for some of the hardware abnormalities that are inevitable. I haven’t been able to verify this, but I’ve been told Leica is doing something like that (although simpler) with S2 system lenses now. Or perhaps lenses will be designed with the ability to adjust mount variation or side-to-side element variation just like we now do front focus and back focus with autofocus microadjustment.