« on: February 08, 2014, 07:48:57 AM »
This thread is a festival of misinformation.
First of all, every test is based on 3 stages: design, data collection, data interpretation.
DXO is quite transparent on the way they design the tests and collect the data. They are not transparent at all in the way they interpret the results and produce a verdict (score). Are they biased? Sure. Every test is. No test can possibly encompass every scenario and variable; the testers decide what to focus on - in this case ISO 100, 8 MP, blah bla. You may agree or disagree with their choices, but your disagreement doens't make the test "stupid", because your choice is not worth any more than theirs.
It must be noticed, for example, that they don't test lenses per se, but lens-camera combinations; this is why lens scores change camera-to-camera. The sharpness of the final output depends on camera AND lens and so this is why they do it. Again, you may disagree. However, every lens will perform differently on a D700 and a D800; testing a lens on a D700 and saying that it's great it's no indication of how it will fare on a 3x resolution body. In this sense their measurements are far more accurate.
It should also be noted that their tests are based on a resized 8 MP file. This is why Canon's sharpness scores are often higher, and Nikon's scores are higher in most other fields. This is not the bread & butter of pixel peepers, but it can better reflect a real world scenario of printed pictures. Again, you may disagree, but it stands true that you need around 8 MP for a 300dpi A4 print, so how the final output will look like can be a more important information than a 100% crop.
Ranting about something while swearing that it's useless and meaningless, is paradoxical and childish. For as much as I agree with most people saying that the way DXO interprets results is disputable and pretty much useless, the data they collect is quite good. You can have DR graphs at different ISOs, compare screen and print output from different cameras and lenses, etc. All of these data are freely accessible and everyone can then draw his own conclusions - probably a more interesting endeavour than just ranting about DXO's.
Every test is useful in its limited purpose. Ranting derives from 2 major causes:
- Incapability of understanding the test methodology, and thus both its usefulness and limitations: this leads to labeling as stupid.
- Results are not what one wants to hear: this leads to discrediting the tests, claiming bias and second interests, etc etc.
I agree with those who thinks that, if it was the other way around, this forum would be full of DXO ambassadors. Same as the recent "conversion" of Scott Kelby to Canon has not raised any suspicions about the actual circumstances as they were presented; it has been a genuine event of a prodigal son finally seeing the true light.