DxOMark vs. Reality

[marekjoz]

Can someone please calculate and compare the surface areas between the orange and red lines because I'm still not convinced when and how to interprete those graphs and which sensor is better and when?
DxO measurements is a bullshit if they can't provide the simple math formulas to calculate the integrals. I'm so dissapointed...

Euh, I don't see any reason to calculate the surface areas between the lines in order to know which sensor is better and when.
Just have a look at theses graphs again and forget everything about integral.

<sarcasm off>

Anyway - the bigger the area in either low or high iso part of the graph, the bigger the difference between them there and the simplest way to calculate this area are integrals when you know the math formulas. But anyway it was just to cheer you all up in this serious discussion...

[mystic_theory]

The 1DX got good grades from DxO: does that mean it's a crappy camera since those grades have nothing to do with reality? 

[PackLight]

The 1DX got good grades from DxO: does that mean it's a crappy camera since those grades have nothing to do with reality? 

No, it means that the 1D X tested well with the three tests that DxO uses to generate their make believe scores.

[Alrik89]

Guys, you've got a real problem, when you assess camera sensors basing on some stupid numbers like DxO.

If you're a professional photographer (a skilled one) then none of your customers will ever see the difference in the IQ/dynamic range/whatever between a Nikon and a Canon DSLR.

So, this discussion, which one has mounted the biggest d1ck... sorry, sensor, in a camera case, is total bu11shit. I think, none of you has ever come into a situation, when he thought stuff like "crap, the dynamic range in this picture is too low. The customer will be very unpleased. I wish, i had a Nikon."

Get a life.

A skilled photographer is not the same as  he or she  is a good  copyist and know how a digital work flou works.  Have you  anytime worked in a darkroom with a negative film  then you will understand what I mean and likewise it is with a raw files.

A skilled photographer with also skills in the digital "dark room" kan do so much more than a photographer with no  photoshops skills.  And even more with a camera who has 14 stops DR.
best regards , pro since 30 years back.

Well, when you need 14 stops DR to "rescue" your pictures via Photoshop, then you should ask yourself, if you`re a real pro.
When it comes to the the point where you really need 14 stops of DR, you've taken your picture in a wrong way.

And: none will see the difference. Neither a customer nor other pro photographer.

[marinien]

Can someone please calculate and compare the surface areas between the orange and red lines because I'm still not convinced when and how to interprete those graphs and which sensor is better and when?
DxO measurements is a bullshit if they can't provide the simple math formulas to calculate the integrals. I'm so dissapointed...

Euh, I don't see any reason to calculate the surface areas between the lines in order to know which sensor is better and when.
Just have a look at theses graphs again and forget everything about integral.

<sarcasm off>

Anyway - the bigger the area in either low or high iso part of the graph, the bigger the difference between them there and the simplest way to calculate this area are integrals when you know the math formulas. But anyway it was just to cheer you all up in this serious discussion...

Haha, thanks 

[ankorwatt]

Guys, you've got a real problem, when you assess camera sensors basing on some stupid numbers like DxO.

If you're a professional photographer (a skilled one) then none of your customers will ever see the difference in the IQ/dynamic range/whatever between a Nikon and a Canon DSLR.

So, this discussion, which one has mounted the biggest d1ck... sorry, sensor, in a camera case, is total bu11shit. I think, none of you has ever come into a situation, when he thought stuff like "crap, the dynamic range in this picture is too low. The customer will be very unpleased. I wish, i had a Nikon."

Get a life.

A skilled photographer is not the same as  he or she  is a good  copyist and know how a digital work flou works.  Have you  anytime worked in a darkroom with a negative film  then you will understand what I mean and likewise it is with a raw files.

A skilled photographer with also skills in the digital "dark room" kan do so much more than a photographer with no  photoshops skills.  And even more with a camera who has 14 stops DR.
best regards , pro since 30 years back.

Well, when you need 14 stops DR to "rescue" your pictures via Photoshop, then you should ask yourself, if you`re a real pro.
When it comes to the the point where you really need 14 stops of DR, you've taken your picture in a wrong way.

And: none will see the difference. Neither a customer nor other pro photographer.

you do not understand -do you, please point out what you mean by wrong way

[ankorwatt]

The 1DX got good grades from DxO: does that mean it's a crappy camera since those grades have nothing to do with reality? 

This will be very intresting to know, Canon have increased the DR   and shows very good high iso scores
if the figures are OK  ( are  DXO still crap?)

[PackLight]

The 1DX got good grades from DxO: does that mean it's a crappy camera since those grades have nothing to do with reality? 

This will be very intresting to know, Canon have increased the DR   and shows very good high iso scores
if the figures are OK  ( are  DXO still crap?)

Yes, the assumptions and overall score they draw from the tests are still crap. There scores are drawn from to narrow of a parameter to be of real use.

[shinyknights]

I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Canon/EOS-5D-Mark-III
http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Nikon/D800

And I'm reading more and I see examples on cameralabs.com.
And here, they show that Canon's 5D Mark III destroys Nikon's D800, particularly at 12800 ISO.

http://www.cameralabs.com/reviews/Canon_EOS_5D_Mark_III/Canon_5D3_vs_Nikon_D800_noise.shtml

Any thoughts on this guys?

[PackLight]

I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Canon/EOS-5D-Mark-III
http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Nikon/D800

And I'm reading more and I see examples on cameralabs.com.
And here, they show that Canon's 5D Mark III destroys Nikon's D800, particularly at 12800 ISO.

http://www.cameralabs.com/reviews/Canon_EOS_5D_Mark_III/Canon_5D3_vs_Nikon_D800_noise.shtml

Any thoughts on this guys?

I think if you use DxO information it is important to know in what context they test, the test reults and ignore the values they place as scores.

As for the test at ISO 12800 you mention, as far as I am concerned the samples would be irrelevant as well because I wouldn't go that far with the 5D. I would with the 1D X.  I think a comparison at ISO 6400, an ISO I might go to on occasion tells more as does the ISO 3200 comparison.

[K-amps]

I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Canon/EOS-5D-Mark-III
http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Database/Nikon/D800

And I'm reading more and I see examples on cameralabs.com.
And here, they show that Canon's 5D Mark III destroys Nikon's D800, particularly at 12800 ISO.

http://www.cameralabs.com/reviews/Canon_EOS_5D_Mark_III/Canon_5D3_vs_Nikon_D800_noise.shtml

Any thoughts on this guys?

They have certain criteria like 30db signal to noise ratio ... any shot that gets below this is tossed out. If they change their criteria to (taking an arbitrary number) 25db, then perhaps 5d3 will have 11000 ISO (another arbitrary number)  and D800 will have 4000 ISO (Yet another arbitrary number)... shifting the criteria around will swing the score back to the Canon Camp. Thats what pissed people about them is the criteria they use to weigh qualities of a sensor for it's "Overall Score".

The criteria they use effectively nullifies Canon's real world high ISO advantage.

The other criteria they have in place is that at the 30db SNR level, at least 18 bit color depth be mantained for the ISO score, now who plucked this number from someones...  expletive is another debate altogether, but it again weights the balance in favor of the Sonikon sensors... (The color bit depth rears it's ugly head again for overall scores.)

I think even with current technology levels fo Canon sensors, with their high ISO advantage, DxO could tweak their criteria and Canon scores could be above Nikons... but there is one area where playing fuzzy math will not help canon and that is the DR number... this is what many Canon Loyaists want Canon to improve.

[elflord]

I'm curious if DxOMark is a credible source of info.
Here, they say that the Canon 5D Mark III is good until 2293 ISO
With Nikon's D800, it says that it is good until 2853 ISO.
They dont even provide picture samples!

Their scores are based on measurements, not picture samples.

If you want the full story behind the scores, click on the measurements tab. The scores compress a lot of information using subjective (and therefore somewhat arbitrary) criteria.

[straub]

Yes - but possibly not in the way I think you mean. The original RAW image is spatially oversampled (20MP or more) with respect to the final output resolution (8MP). The process of converting the original high-resolution image to the smaller image is downsampling.

The Wikipedia pages are helpful here, and explain this probably better than I can:

  http://en.wikipedia.org/wiki/Oversampling
  http://en.wikipedia.org/wiki/Downsampling

These also give an example of how this can be exploited to give more resolution (DR) than the actual converter (sensor pixel).

And that must be why an 8KHz downsampling of a CD has more than double the dynamic range of the original, right? You can google theorems, fine, but apparently have no idea what they mean and where they are applicable. So you must work at DXO, then?

Try a test--take a D800 photo of Stouffer wedges, push exposure in post how much you like, and make a note of the darkest one. Now downsample to 1/64, i.e. ~0.5mpix and repeat. How many extra wedges appeared? Exactly as many as how much the *oversampling theorem* is applicable here--i.e. zero.

[MarkII]

And that must be why an 8KHz downsampling of a CD has more than double the dynamic range of the original, right? You can google theorems, fine, but apparently have no idea what they mean and where they are applicable. So you must work at DXO, then?

The downsampling will reduce the bandwidth but increase the resolution of the samples. You trade off precision in the amplitude (quantisation noise) for bandwidth (loosing high frequencies). This is exactly the same as happens when you down sample a digital image - and just as you might not want to listen to all your CDs at 8KHz you probably don't want to print out all your images at 600x400 no matter how much this improves the DR.

I posted the Google links as they are usefully informative on the subject. You, rather than posting an informed and referenced response, just posted an insult (FYI, I am a Physicist  who has worked with DSP and signal processing for more than 20 years, and I have no connection to DXO).

Try a test--take a D800 photo of Stouffer wedges, push exposure in post how much you like, and make a note of the darkest one. Now downsample to 1/64, i.e. ~0.5mpix and repeat. How many extra wedges appeared? Exactly as many as how much the *oversampling theorem* is applicable here--i.e. zero.

Well, I do not have a D800 to test (and have not commented on this specifically).

I find that the DXO measurement data matches pretty well my experience when shooting with those cameras that I have (GF1, 40D, 5DII, 5DII). You can take issue with the fairly arbitrary criteria that they use to produce their overall scores and I am sure that DXO sometimes make mistakes, but the individual measurements - if you understand what they are - have always looked pretty good to me and correlate well with my experience of the 5DIII (particularly when compared to the 5DII).

Simply wanting something to be different does not make it so.

[tron]

I can understand the improvement in noise when downsampling but can someone prove that downsampling an image (to 8Mpix for example) improves DR?

I ask this because I believe in practice what is burned in the highlights has been lost forever and when  there are shadow areas I cannot image how a "dark" pixel will benefit from its neighbour equally dark ones.

Thanks.