DxO Sensor data: 5D Mark III sensor is better than 5D Mark II

[dilbert]

This guy uses the raw sensor data from DxO (remember, they're worthless!) to create some different measurements:

http://www.sensorgen.info/

Model	QE	Min Read Noise (e-)	Max Saturation capacity (e-)
5D2	33	3.2	64600
5D3	49	2.4	67531
D800	56	2.6	44972

This table also makes the 60D better than the 50D and 7D.

[Mt Spokane Photography]

His chart claims that a D3100 has 0.0 read noise! 
That should give a clue as to accuracy / credability.

[dilbert]

His chart claims that a D3100 has 0.0 read noise! 
That should give a clue as to accuracy / credability.

Why, you have your own data from doing the same measurement with a D3100 sensor that contradicts it?
Or you have another source that you can cite to show that it is wrong?

No, I didn't think so.

There's a whole bunch of very information available in those tables if you take the time to read them. For example, they generally agree with people saying that the 5D3 is slightly less noisy than the 5D2 but not significantly so.

[heptagon]

His chart claims that a D3100 has 0.0 read noise! 
That should give a clue as to accuracy / credability.

Well, that comes from one data point at ISO 3200 which obviously is wrong. Did you already send him an email with the correct value?

[neuroanatomist]

His chart claims that a D3100 has 0.0 read noise! 
That should give a clue as to accuracy / credability.

Why, you have your own data from doing the same measurement with a D3100 sensor that contradicts it?
Or you have another source that you can cite to show that it is wrong?

No, I didn't think so.

Is one necessary? If I told you that I dropped a penny and a bowling ball off the Leaning Tower of Pisa, and both of them flew straight up into the stratosphere, would you need a source to prove I was wrong?  0.0 read noise is an electronic impossibility (if it had stated below the LLOQ, or something similar, fine, but not zero).

[dilbert]

His chart claims that a D3100 has 0.0 read noise! 
That should give a clue as to accuracy / credability.

Why, you have your own data from doing the same measurement with a D3100 sensor that contradicts it?
Or you have another source that you can cite to show that it is wrong?

No, I didn't think so.

Is one necessary? If I told you that I dropped a penny and a bowling ball off the Leaning Tower of Pisa, and both of them flew straight up into the stratosphere, would you need a source to prove I was wrong?  0.0 read noise is an electronic impossibility (if it had stated below the LLOQ, or something similar, fine, but not zero).

First, "0.0" is not "0". Second, it could mean LLOQ, but I don't know enough about the method, etc.

Generally speaking, if you want to criticise someone else's results of a scientific method then you do so by doing your own experiments that prove your point. And yes, I'm willing to bet that the method DxO use to generate the raw numbers that goes into those calculations and the method of arriving at the calculations in that table is a lot more scientific than the process of saying "That 0.0 is wrong and because that 0.0 is wrong all other results lack credibility."

Further, all of the results (except for that line) seem completely reasonable which tends to suggest that there is more than fiction behind them. If it was Ken Rockwell posting this information then I'd be extraordinarily suspicious. But it isn't and more to the point, the page is free of ads so quite clearly there's no need by the author to generate lots of page views by creating sensationalist content.

Finally, as the "About" page says (http://www.sensorgen.info/Calculations.html), all of the calculations are based on DxO data. If the "0.0" seems outrageous then you're free to download the data for yourself and see if the published methodology confirms that "0.0" is correct (or not) and also to contact DxO and challenge them on the accuracy of the data for the Nikon D3100 or even start threads all over the Internet about how the true extent of Nikon fanboy-ism at DxO is revealed by results that are obviously bogus for the D3100.

[neuroanatomist]

Actually, it seems like a case of automated/formulaic calculations without the benefit of a human eye looking at the data.  The D3100 read noise plot has no data point at ISO 3200, which was treated as 0.0 instead of being properly flagged as n.d.  There's also a missing data point for ISO 3200 DR, which was assigned a DR of 25 stops in the table, another clear impossibility.

I have respect for most of DxO's methods (generally, and with the caveat that you have to understand what they measure, what they don't measure, and where the black boxes are so you can ignore meaningless values like the 'sensor score').  I also have respect for people who re-analyze others' published data.  But I lack respect for those who report data that have obviously not been QC'd, which is immediately apparent from the read noise = 0.0 and the DR = 25 stops. 

Since DxOMark does have data for the D3100 at ISO 3200 for both ISO Sensitivity and Full SNR curves, which Sensorgen draws upon for their source data, it seems that the problem is with Sensorgen's reanalysis, and not the original DxO data.

It is not necessary to attempt to reproduce someone else's results to invalidate those results in cases where the results are clearly not credible - and 0.0 noise with 25 stops of DR is simply not credible, any more than pennies and bowling balls defying gravity.

[Mt Spokane Photography]

Generally speaking, if you want to criticise someone else's results of a scientific method then you do so by doing your own experiments that prove your point. And yes, I'm willing to bet that the method DxO use to generate the raw numbers that goes into those calculations and the method of arriving at the calculations in that table is a lot more scientific than the process of saying "That 0.0 is wrong and because that 0.0 is wrong all other results lack credibility."

A wrong answer on one calls every answer into question.  That is how a engineer views it.  And, I have done plenty of scientific reports for NASA.
The difference is that my reports were checked by a person, not just something kicked out by a formula in a spread sheet.  If the spread sheet calculated one wrong, its not up to me to verify all of them.

[Dasengle]

Wow.  Way to throw out the baby with the bath water guys.  Yeah there are a few screwy data points, and the algorithm needs to be vetted.  But this is a promising first step.  Way better than sensor score.

[heptagon]

Generally speaking, if you want to criticise someone else's results of a scientific method then you do so by doing your own experiments that prove your point. And yes, I'm willing to bet that the method DxO use to generate the raw numbers that goes into those calculations and the method of arriving at the calculations in that table is a lot more scientific than the process of saying "That 0.0 is wrong and because that 0.0 is wrong all other results lack credibility."

A wrong answer on one calls every answer into question.  That is how a engineer views it.  And, I have done plenty of scientific reports for NASA.
The difference is that my reports were checked by a person, not just something kicked out by a formula in a spread sheet.  If the spread sheet calculated one wrong, its not up to me to verify all of them.

Yet the other datapoints LOOK good. Can we verify at least two cameras 5D3 and D800? Getting second opinions on them would be good.