I cannot help but question the measurement methodology of a group who (either fraudulently or ignorantly) uses biased and misleading summary statistics to put forth claims about camera/lens performance. That is to say, if you don't analyze your data properly but staunchly claim to be fair and objective, then it is my obligation to question your data collection methods as well, because your entire process is now suspect. That is what any good scientist does.
DxO has the phrase "Image Science" as part of their logo, but their practices aren't consistent with that phrase. I'm most concerned by the 'black box' calculation for their summary Sensor Score and Lens Score (methods should be published), and by the fact that they released data which was incorrect, defended it, then subsequently changed it with no acknowledgement of their error. Also, I'm noticing that the more I delve into their Measurements, the more I find errors (for example, I just looked at the Canon 28-300L measurements and their actuance data shown visually as field maps are ~10% lower relative to the same data plotted on a graph as a profile).
This right here sums up my problems with DXO in a nutshell. They HAVE made mistakes in the past, ignored them, then quietly tried to fix them without a word. That's irksome. I'm not saying that their DR tests (or rather, Screen DR measurements) are wrong...they are just as valid as any other form of DR test, so long as the same method is applied with consistency.
My problem is that DXO does not clearly explain all of their methods, and sometimes their methods seem sketchy. For example, Print DR is not actually measured, it is derived. Print DR is the "measure" (as they call it) used in scoring their camera sensor tests. But it is not a measure, it is a derivation from the actual Screen DR measure. Problem with Screen DR is, they don't actually publish the actual method by which they measure it, so not only do you have Print DR with is derived from Screen DR...you don't really know how Screen DR is computed. That's quite frustrating. (It gets even worse when you download DXO Optics Pro to see how it works, and find that DXO's algorithms result in FAR more noise with Canon CR2 files than ACR/LR, RawThearapy, or any one of various astrophotography tools...makes me even that much more suspicious that DXO doesn't really know how to properly process Canon CR2 files for optimal performance.)