...how dxomark has achieved such a respected and credible status when it comes to sensor reviews when it seems they "get it wrong" so consistently with lenses.
I'm not sure 'wrong' is right. Rather, I'd say their scores only apply in a very specific circumstance, which most likely is not often relevant to actually using the lens. My real beef with their scores, both for lenses and for sensors, is that the 'overall' scores are derived from the individual sub-measurements in an undisclosed manner, and also for the sensors that their normalization step is flawed (it generates 'impossible' data points, like >14-bit DR for a sensor with a 14-bit ADC). Frankly, I give them no credibility for their Score metrics, but lots for their raw measurements.
In the case of lenses, they are reporting resolution as a peak measurement - the highest resolution measured at any location in the lens' FoV, at any aperture setting, and for zooms at any point in the focal range. Maybe the lens is crap wide open and crap through most of the zoom range - DxOMark's resolution score doesn't care. So...as long as you are using it at that aperture, and your subject is in the right place in the frame, great. See what I mean about specific sircumstances? DxoMark measures lateral CA, but not longitudinal CA - and LoCA is a weakness of some lenses, particularly the 85/1.8.
They're on crack to believe that the 85 1.8 is better than the 135L. Not even in the same league.
And right there, my friend, you've fallen straight down into the pit trap that leads to heated arguments and internet flame wars.
They are
not saying the 85/1.8 is the best lens. They are saying that it achieves a higher lp/mm resolution, with
their copy of the lens on
their 1DsIII, at a
specific aperture and a
specific location in the image field. Nothing more. The rank-ordered list you linked does not even take into account the other factors they do measure (distortion, vignetting, LCA) much less the things they
don't measure (color transmission, LoCA, bokeh, AF speed, etc.), and
all of those are important to overall lens performance.
Honestly, you've illustrated the real problem with the DxOMark scoring - it's not the flawed normalization, not the 'black box' determination of overall score. It's the fact that by taking a complex optical system - lens or camera - and reducing it's multifaceted aspects of performance to a single number, they make it far too easy for human nature to pounce on that number and say, "This one is the best."
To sum up, IMO, DxO's Measurements are valid and useful, their Scores are meaningless, and the inappropriate interpretation that many forum posters apply to their conflated scores is reprehensible.
Just my 2ยข.
