Plenty of criticisms on semantics but why no alternative suggestions on methodology?
Lesson learned.
M
Ps- Jrista- I look forward to seeing your tests.
Heh, well, there are definitions and math, and there are ad-hoc tests. There is nothing wrong with an ad-hoc test, so long as it is represented accurately in the correct frame of reference. The definition of EV and the math adds up, and I believe your test is effectively falsified given the facts. We can derive the EV you were actually shooting at based on your exposure settings.
You stated that in your test "0" EV was 1/60s f/2.8 ISO 400. Well, first off, EV 0 cannot really be arbitrarily defined. Officially, EV 0 is "baselined" at 1s f/1. From an available-light standpoint, ISO does not actually matter. So, if EV 0 is officially 1s f/1, then a 1/60s f/2.8 exposure is actually EV 9 (6 stops difference in shutter speed, 3 stops difference in aperture). That is a difference of 12 stops total from an actual EF -3, and a total of 16 stops off from the EV -7 you claim in your test.
This makes the assumption that at 1/60s f/2.8 ISO 400, your images were properly exposed (which, based on your blog, they were not...we'll get to that).Since it is the AF sensor that really matters in the context of DSLR phase-detection AF, the shutter speed doesn't really actually matter either. In this context, the EV designation ultimately refers to a level of illumination on your AF target. If you really want to test AF at -3 EV, you should try it out under the light of a full moon on a clear night with an f/1.4 lens, such as the EF 50mm f/1.4. Assuming you were exposing correctly at 1/60s f/2.8, that would be like photographing a few minutes after sunset during "golden hour"...which is actually a LOT more light than photographing just by moonlight, even though it may appear to be rather dim to your eyes. According to your blog, you were testing indoors with standard house lighting. Indoor illumination is usually rated at around EV 7, however your target was directly in a spot of light illuminated by a very close light fixture that appeared to have multiple bulbs in it, so I'd say EV 9 sounds about right.
You were also underexposing by a significant degree. Again, it should be noted that as far as the AF sensor is concerned, the actual camera exposure settings do not matter, so trying to "simulate" EV -3 by underexposing a photo is not the same as autofucusing in EV -3 or even EV 0. You were actually autofocusing with quite a lot of light. The illumination from the area of your wall that you focused on was probably about EV 9, however you were using an f/2.8 lens rather than an f/1 lens (which, BTW, I am again assuming...Canon cameras always AF at maximum aperture...if your maximum aperture was wider than f/2.8, and you simply chose f/2.8 to force a certain underexposure, then my calculation here will be wrong.) At f/2.8, the AF sensor is effectively working at EV 6. That means that in your test case, instead running the 6D through it's paces at EV 0 (or for that matter EV -3 or -7), you were actually running it through its paces at EV 6.
I don't mean any offense, however your blog writeup about the 6D's AF performance will be eaten alive by anyone who understands how the AF system works, and understands the rough level of illumination of your AF target. The AF Unit, which resides underneath the mirror in the mirror box, contains a special light-splitting lens that directs light from both sides of the lens mounted to your camera onto various phase-detection strips in the AF sensor. The actual amount of light each strip has to work with is minuscule. If we were to describe it in terms of EV's, at EV -3 in your scene each strip is probably working at EV -5 or EV -6, it not less. The mirror will split the light from the camera lens in half, as 50% is reflected up through the viewfinder and the other 50% is redirected through the primary mirror to the secondary AF unit mirror underneath the primary. The AF Unit lens will further split that light for each AF strip. The central cross-type point in the 6D is composed of two diagonal strips, each of which would get around half of the 50% of light directed down to the AF Unit. That is at least two stops, assuming all of this mechanical redirection of light operates at 100% efficiency. Since nothing is perfectly efficient, the phase-detect strips on the AF sensor itself are really probably working at EV -6 to EV -7, when the illumination on your AF target is EV -3. None of this has anything to do with exposure settings...all that matters is the amount of light passing down the lens at maximum aperture. Since the EV scale is baselined at 1s f/1, if you use an f/1.4 lens, the AF sensor is working with even less light. If you use an f/2.8 lens (the minimum required for the central high-sensitivity AF point in the 6D to work), the AF sensor phase-detect strips are working at around EV -10, a vanishingly small volume of light!!! Hopefully this puts the incredible feat of phase-detection AF at EV -3 into, erm,
proper light! (...yes, pun intended. ;P)
Now, I can't really say how much illumination the target on the floor under your desk may have had. It is quite possible you were getting closer to EV 0 at that point. However making a guess, even mildly educated as it may have been, is no substitute for using a proper light meter to actually measure the illumination under your desk in EV. It could very well have been that, as dark as it may have looked, the target under your desk was EV 2 or 3, if not higher. There is another caveat here as well. The kind of AF performance testing done by Canon is performed extremely carefully under very tightly controlled circumstances. The ambient temperature must be exact, the kind of lighting used to illuminate the test target must be full spectrum to introduce a proper and predictable amount of illumination, high frequency so as not to introduce a frequency oscillation that could interfere with the test (something that can occur with improperly designed ballasts for CFL bulbs that operate on a 60Hz power frequency), and test targets must be designed with the right level of contrast such that at the incredibly low levels of illumination below EV 0, the AF sensor can actually pick up differences in contrast at all.
I would not be surprised if the cameras failed or were slow to AF on the target under your desk due to a lack of contrast. Your carpet looks to be a dark gray or slightly off gray, with a black + as the target. In EV -3, I highly doubt anything would pick up enough contrast from that to really do anything. If you had stuck the target + on a piece of white paper, that would have created a slightly better test target for testing AF in low illumination, and even then, your at the boundary of the AF units rating, so AF will be spotty at best even in the most ideal of circumstances.
I hate to say it, but your test approach leaves a lot to be desired, and I believe many of the claims (such as EV -7) were fundamentally incorrect. It is highly ad-hoc, and makes use of a lot of guesswork. A lot of the critical factors, such as your definition of EV 0, or even the notion that camera settings have any impact on the AF system at all, are flat out incorrect. I think the only truly valid part of your test was the comparison of the central AF point with the outer AF points, and the conclusion that the central AF point being more sensitive was spot on, if already common knowledge.
As Neuro stated, you have the perfect set of cameras to perform such a test. Maybe, armed with more knowledge from my post here, and further armed with an actual incident light meter to get an accurate EV reading at your AF test targets, you can perform a more accurate and realistic test in a proper frame of reference.
