dilbert said:
I'm willing to bet that the lighting level they choose to use is for a specific purpose and not just some randomly picked number.
For example, maybe it is easier for them to detect the difference between f-stop and t-stop at that level. Maybe there is less internal reflections and flaring at that level. And so on. Whilst we don't know why they chose that level of illumination, I'd wager that it isn't an arbitrary choice.
Of course it's not a randomly picked number, and they state why they picked it: "
...we choose the scene illumination to be 150 lux and the exposure time 1/60s. Such conditions were chosen as we believe low-light performance is particularly important for today’s photography and it is also important for photographers to know how well lenses perform at widest aperture." The point is that choosing a low level of illumination like that introduces a
bias into their scores, and it's a non-obvious bias given the way the scores are presented (yes, you can find out about the 150 lux testing, but you have to look for it).
Even the explanation for their bias is applied in a biased (and undisclosed) manner. They state that, "...it is also important for photographers to know how well lenses perform at widest aperture," and the implication is that since the 50/1.8 scores higher than the 600/4, for example, the 50/1.8 performs better at f/1.8 than the 600/4 at f/4.
dilbert said:
It is entirely possible that a 50/1.8 can reproduce an image better than a 600mm lens. Look at all of the noise people have made about Zeis's 55mm recent announcement. Just because a lens is bigger or heavier or has a longer focal length doesn't mean it is better at reproducing an image.
According to DxO's own data, the 600 II is sharper, has less distortion, less vignetting, and less chromatic aberration. The 600/4 also loses less light relative to it's f/number than the 50/1.8, i.e. the 600's max T-stop is closer to it's max f/stop than is the case for the 50/1.8. So, on every measure of optical performance
except absolute light transmission, the 600 II is a better lens. But the 50/1.8 gets a better Score. So…a higher DxO Score is saying that a faster lens is "better" simply because it's faster, as if people are unable to figure that out from the f/1.8 vs. the f/4 spec. Even people who think they can shoot video with a lens that doesn't have a camera attached to it are probably able to understand that an f/1.8 lens will transmit more light than an f/4 lens, and hopefully don't need DxO's BS to tell them that.
As for it being, "
...entirely possible that a 50/1.8 can reproduce an image better than a 600mm lens," you can easily see how much better the 50/1.8 II at f/1.8 reproduces an image, compared to the 600mm f/4L IS II at f/4. Just
look for yourself. :
:
:
Incidentally,
according to DxOMark, the Canon 300mm f/2.8L IS II is:
- sharper than the Zeiss Otus 55mm f/1.4
- has less distortion than the Zeiss Otus 55mm f/1.4
- has less vignetting than the Zeiss Otus 55mm f/1.4
- has less chromatic aberration than the Zeiss Otus 55mm f/1.4
…but still, the Zeiss Otus 55mm f/1.4 gets a significantly higher score. Why? Because it's f/1.4…and in world of DxOMark's BS, that automatically makes it "better".
