So, I was having a discussion with a friend of mine who is also a photographer about the accuracy of DxO Mark.
In general, he was in favour of the immediate report and scoring that DxOMark.com refers to when referring to the 'Camera Sensor Ratings' and the scores they show in their ranking list. I disagreed, claiming that the results shown on the immediate report are, in my opinion, inaccurate.
For reasons not mentioned, we were comparing the Nikon D800 and the Canon 1D X. Specifically, the Dynamic Range.
I claimed that the results that the D800 had a Dynamic Range of 14.4 Evs compared to the 1D X's 11.8 is in part true, but an inaccurate representation of the tests overall results. Here is why:
The reported Dynamic Range of the D800 is 14.4 EV. While this is true, it is a representation of the MAXIMUM possible Dynamic Range at only one ISO, not the actual performance throughout the ISO range.
The Dynamic Range of the D800 drops off at an average rate of .91 Evs every increase in ISO.
Compared to the Canon 1D X, which averages an average .46 Evs loss every increase in ISO.
To put it relatively, .46 Evs is less than the smallest loss the D800 sees in an increase in ISO.
After three increases in ISO, the D800 has lost 2.31 Evs.
Compared to the 1D X that has lost 0.16 Evs.
After five increases in ISO, the 1D X has better Dynamic Range throughout the rest of the ISO.
After nine increases in ISO [limit of D800] the D800 has dropped to half its original Dynamic Range.
Compared to the 1D X that has lost as much as the D800 did after five increases in ISO.
The average Dynamic Range throughout the ISO differs by .33 Evs
1D X - 10.60
D800 - 10.93
The deviation between highest and lowest possible levels of Dynamic Range throughout ISO differs by 90%.
1D X - 3.73
D800 - 7.33
In conclusion: [referring to Dynamic Range throughout ISO]
The results displayed on DxOMark's 'Camera Sensor Ratings' scorecards are in my opinion mis-informative of the actual results shown throughout the camera's performance. While in 'perfect' conditions the Dynamic Range of the D800 WILL outperform the 1D X, the average results, standard deviation and high-to-low end loss of performance would suggest that the 1D X is a better overall performer in regards to Dynamic Range throughout the ISO spectrum.
[These results are based only upon those given by DxOMark. I do not believe DxO is an inaccurate or dishonest company. I posted this only to represent a more in depth look at the results given on the immediate Camera Sensor Ratings scorecard. In my honest opinion, this is why I PERSONALLY believe the results shown on the Scorecard are an inaccurate representation of the camera's overall performance]
Now I can wait for the D800 lovers to tell me how much of a moron I am.
Here is a 'graph' of the results:
1DX
11.77
11.81 +.04
11.74 -.07
11.61 -.13
11.18 -.43
10.37 -.81
9.88 -.49
8.99 -.89
8.08 -.91
Deviation 3.73
Average 10.60
-
D800
14.33
13.65 -.68
12.83 -.82
12.02 -.81
11.12 -.90
10.09 -1.03
9.17 -.92
8.16 -1.01
7.00 -1.16
Deviation 7.33
Average 10.93
In general, he was in favour of the immediate report and scoring that DxOMark.com refers to when referring to the 'Camera Sensor Ratings' and the scores they show in their ranking list. I disagreed, claiming that the results shown on the immediate report are, in my opinion, inaccurate.
For reasons not mentioned, we were comparing the Nikon D800 and the Canon 1D X. Specifically, the Dynamic Range.
I claimed that the results that the D800 had a Dynamic Range of 14.4 Evs compared to the 1D X's 11.8 is in part true, but an inaccurate representation of the tests overall results. Here is why:
The reported Dynamic Range of the D800 is 14.4 EV. While this is true, it is a representation of the MAXIMUM possible Dynamic Range at only one ISO, not the actual performance throughout the ISO range.
The Dynamic Range of the D800 drops off at an average rate of .91 Evs every increase in ISO.
Compared to the Canon 1D X, which averages an average .46 Evs loss every increase in ISO.
To put it relatively, .46 Evs is less than the smallest loss the D800 sees in an increase in ISO.
After three increases in ISO, the D800 has lost 2.31 Evs.
Compared to the 1D X that has lost 0.16 Evs.
After five increases in ISO, the 1D X has better Dynamic Range throughout the rest of the ISO.
After nine increases in ISO [limit of D800] the D800 has dropped to half its original Dynamic Range.
Compared to the 1D X that has lost as much as the D800 did after five increases in ISO.
The average Dynamic Range throughout the ISO differs by .33 Evs
1D X - 10.60
D800 - 10.93
The deviation between highest and lowest possible levels of Dynamic Range throughout ISO differs by 90%.
1D X - 3.73
D800 - 7.33
In conclusion: [referring to Dynamic Range throughout ISO]
The results displayed on DxOMark's 'Camera Sensor Ratings' scorecards are in my opinion mis-informative of the actual results shown throughout the camera's performance. While in 'perfect' conditions the Dynamic Range of the D800 WILL outperform the 1D X, the average results, standard deviation and high-to-low end loss of performance would suggest that the 1D X is a better overall performer in regards to Dynamic Range throughout the ISO spectrum.
[These results are based only upon those given by DxOMark. I do not believe DxO is an inaccurate or dishonest company. I posted this only to represent a more in depth look at the results given on the immediate Camera Sensor Ratings scorecard. In my honest opinion, this is why I PERSONALLY believe the results shown on the Scorecard are an inaccurate representation of the camera's overall performance]
Now I can wait for the D800 lovers to tell me how much of a moron I am.
Here is a 'graph' of the results:
1DX
11.77
11.81 +.04
11.74 -.07
11.61 -.13
11.18 -.43
10.37 -.81
9.88 -.49
8.99 -.89
8.08 -.91
Deviation 3.73
Average 10.60
-
D800
14.33
13.65 -.68
12.83 -.82
12.02 -.81
11.12 -.90
10.09 -1.03
9.17 -.92
8.16 -1.01
7.00 -1.16
Deviation 7.33
Average 10.93
