moreorless said:
The PROBLEM, here, is that Nikon's lens was "scored" the same as Canon's lens. Given the raw data, the Canon lens should have scored better, without question. That begs the question...why? The only edge the Nikon 600mm had was 0.1 T-stops better transmission. So, DXO is really telling us that the end-all be-all of optical quality is having a tenth of a stop better transmission? Seriously?
Again I can see the worth of measuring the whole package although it obviously means comparing lenses will be harder.
The "whole package" has nothing to do with it. The sole area where the 600/4+D800 performed better than the 600/4+5DIII was in the T-stop category. It performed better by a "whopping" whole 0.1 stops (a TENTH of a stop). Somehow, that one tenth of a stop in better transmission was enough, by DXO's standards, to overcome lower resolving power, worse vignetting, and worse CA? The issue isn't the whole package. The issue is the bias. There is no way that a tenth of a stop of additional transmission is worth that. That is immeasurable, given that camera metering can vary by that much or more for the same scene if several shots are taken in succession.
There is no whole package issue here...there is a blatant bias issue. The two lenses should not score the same, even WITH the D800's sensor being factored into the mix. The sensor is plain and simply not even close to enough to make the Nikon 600mm f/4 lens compare to the new Canon 600mm f/4 L II.
moreorless said:
There is clearly something wrong with DXO's scoring. Their lens tests are unabashedly biased, which makes you question every single one of their scores, and to some degree their approach. That's all I am saying. I'm not making any comparison of the general accuracy of DXO vs. DPR or anything like that...just saying that DXO has a severe and blatant bias, and people who use their information need to be aware of that.
Not sure where I stand on the "bias" but to me DxO's "pro Canon" P-Mpix score seems much much dodgier than there "anti Canon" DR scores. The methodology of it is only vaguely hinted at and all we get is a headline score that doesn't even state how its achieved(one focal length? an average?). The results disagree with my own viewing of sample images as well, the D800 has a much larger advantage in resolution than these numbers hint at. I see no such discrepancy with the DR figures where the Nikon cameras clearly do outperform the Canon's in recovering shadows at low ISO.
There isn't anything dodgy about the P-mpix score. It tells you how much of your base sensor resolution is perceptually "lost" due to sensor and lens manufacturing defects. In the case of the 5D III + 600/4, you lose 2.3 megapixels...indicating both the sensor and the camera are excellent in combination. In the case of the D800 + 600/4, you lose 18.3 megapixels. In terms of relative loss, you lose 10% resolution with the 5D III+600/4, and 50% resolution with the D800+600/4. The P-mpix score uses the lenses ideal aperture, where vignetting and CA are at their lowest without losing too much to diffraction. In the case of the Nikon 600mm f/4, based on the 18 P-mpix score, you clearly have to stop down a fair bit in order to reduce CA to the 9µm that DXO measured...CA would otherwise be higher at wider apertures, which would have indeed detracted from the final "score".
Remember that output resolution is a convolution of the resolutions of the components involved. The Canon lens has extremely high resolving power, which when combined with the 22.3mp sensor, results in a high quality result. The simple formula sqrt(lensBlur^2 + sensorBlur^2) closely approximates output blur circle for a DSLR, assuming you know the other two factors. Blur circle size can easily be converted into spatial resolution. We know the 5D III has a 6.25µm pixel pitch. If we also knew the outputBlur size, we could solve for the lens as so:
outputBlur = sqrt(lensBlur^2 + sensorBlur^2)
outputBlur^2 = lensBlur^2 + sensorBlur^2
outputBlur^2 - sensorBlur^2 = lensBlur^2
lensBlur = sqrt(outputBlur^2 - sensorBlur^2)
Now I don't know of any way to convert DXO's P-mpix measure into simple lp/mm or blur circle size, as they take into account a number of perceptual factors like acutance, and use the lens' best performing aperture. Regardless, P-mpix is taking into account the convolution of the final image. You will never see any lens produce as many P-mpix as the sensors the lens was tested with...the P-mix value will always be lower, because you can never achieve the maximum potential of either lens or sensor.
