Why the DxO bashing?

neuroanatomist said:
ajfotofilmagem said:
philmoz said:
There's obviously a lot of very knowledgeable people here so I'll pose a question that's been bugging me for a long time.
How is it possible for DxO to claim > 14 stops of dynamic range for cameras with a 14 bit ADC ???
Phil.
Noise determines what is considered "absolute black", and from this absolute black is counted how many points of DR to reach full white (highlights without texture). When DXO makes downsize to 8 megapixel, the noise is reduced, and this aspect of sensor 36 megapixel lead comparative advantage. If you do not apply to downsize 8 megapixel count DR will not reach 14 stops.
The 'screen DR' of the D800 is 13.2 stops. When an image with >13.2 stops of DR is captured at 36 MP, detail from the highlights, shadows, or both is irrevocably lost. If the 36 MP image is downsampled to 8 MP to yield 14.4 stops of mathematically calculated DR, how is the detail that should exist in the extra 1.2 stops of DR created? Do blacker blacks matter it they are completely devoid of detail?
That's right, there is not will be more texture in deep black after downsized. It's just a way of "artificially enhance" the numbers to appear more attractive. As they say: "Statistics is the whore of mathematics."
 
Upvote 0
ajfotofilmagem said:
That's right, there will be more texture in deep black after downsized. It's just a way to "smooth out" the numbers to appear more attractive. As they say: "Statistics is the whore of mathematics."

"More texture?" Texture has detail. From where did that detail come? Was it part of the original scene? Part of the original image? If neither, it's not only meaningless as additional DR, it's an artifact and as such, detrimental to the image.
 
Upvote 0
neuroanatomist said:
ajfotofilmagem said:
That's right, there will be more texture in deep black after downsized. It's just a way to "smooth out" the numbers to appear more attractive. As they say: "Statistics is the whore of mathematics."
"More texture?" Texture has detail. From where did that detail come? Was it part of the original scene? Part of the original image? If neither, it's not only meaningless as additional DR, it's an artifact and as such, detrimental to the image.
Correcting: "There is not will be more texture..."
 
Upvote 0
philmoz said:
There's obviously a lot of very knowledgeable people here so I'll pose a question that's been bugging me for a long time.

How is it possible for DxO to claim > 14 stops of dynamic range for cameras with a 14 bit ADC ???

Phil.

Dynamic range has two key interpretations, and they are often conflated resulting in the kind of confusion you see here.

Mathematically, dynamic range is simply defined as the ratio between the smallest and largest (or darkest and brightest) values of some measurable quantity. In the case of photography, it is more specifically defined as the ratio between the RMS of read noise (read noise specifically, that's important) and the maximum saturation point of a pixel. The reason the root mean square (or some similar average) of read noise is used as the "darkest" value is because read noise intrudes into and eats away at useful values below a certain level. When read noise is higher, the ratio decreases, thus reducing dynamic range.

Another key interpretation is the bit error ratio (BER). This has to do with noise in the signal at every measurable point between the extremes as defined above, not just in the shadows. When you hear someone say "a full-frame sensor has more dynamic range than an APS-C sensor", they are really referring to a lower BER. A full frame sensor like the 5D III has a full well capacity of ~68ke-, where as an APS-C sensor may have a full well capacity of 22ke-. The 5D III has three times the maximum signal power, meaning it has less noise at all levels, not just read noise but intrinsic signal noise as well. When DXO says an image gains dynamic range by downsampling, it is because the BER was reduced thanks to the averaging involved in downsampling. Not just in the deep shadows, but at all levels of the signal.

These two interpretations are basically two sides of the same coin, but there is a key difference between them that really matters: One has to do with the quality of the signal between the extremes, the other has to do with what the extremes are. Screen DR deals with what the extremes are. Print DR deals with the quality of the signal between the extremes. Read noise has a direct impact on EDITING LATITUDE, and as far as photographers are concerned, that is synonymous with shadow and highlight recovery (really, shadow lifting.) Less read noise, regardless of the maximum signal strength or the amount of intrinsic noise in the signal, the more you can lift shadows without having them look ugly because of read noise.

Screen DR tells you how many stops of dynamic range you have to work with when editing RAW. Print DR tells you how nicely the signal cleans up when you downsample to a specific size, but it really has nothing to do with editing latitude...it won't increase it...not in any way that will allow you to recover clipped highlights or shadows. You may have 14.4 stops or DR because the signal (the range of tones between the deepest black and the brightest white) is cleaner...lower error rate per pixel, less noisy.

If you have a 14.4 stop scene that you try to capture with the D800 or D600 in a single frame, you will be clipping 1.2 stops of highlight and/or shadow detail. That 1.2 stops is lost, gone for good, never recoverable...because the SENSOR (i.e. the RAW image) only has 13.2 stops of DR. If you clip 1.2 stops of highlights, pulling down highlights in post will result in clearly blown highlights...you can make them gray rather than white, but those regions will always lack detail. If you clip 1.2 stops of shadows, lifting in post will eventually result in nothing but the stretching of read noise by 1.2 stops, you won't be recovering those lost 1.2 stops of shadows. The D800 is only capable of capturing 13.2 stops of dynamic range in a single shot, as far as editing latitude (the ability to lift shadows without also lifting noise so much that it affects the image) is concerned.

Finally, DXO is making a claim that isn't even backed up by measurements. Print DR is derived from ScreenDR...it is not measured:

Code:
DRprint = DRscreen * log2 sqrt(originalMpix/referenceMpix)

DXO uses a fixed formula for extrapolating what the noise level should theoretically be in an image downsampled from the original megapixel count of the sensor to the reference megapixel count (3200x2400, or 7.68mp, which is an 8x12" 300ppi 3:2 ratio print). Therefor, a Print DR or 14.4 is not based on actually measuring the dynamic range of a D800 image that was exported at full size from a RAW image, then actually downsampled to 3200x2400 pixels. It's a mathematic extrapolation, not a measure....even though DXO calls it a measure. (I'd be willing to bet actually measuring the D800's 3200x2400 image would result in a lower dynamic range.)

Personally, this is one of my only real pet peeves with DXO's sensor tests. They extrapolate, rather than measure. (The other pet peeve is they weight...they give out bonuses when a camera "beats" a certain threshold...which skews the differences even more. Two cameras might actually be within the most minimal difference of each other from actual RAW measurements, but because of the fact that bonus weighting comes into play, a camera with a sensor that beats say a DR dB threshold by 0.1dB would appear to be quite a bit better than the competitor, despite the fact that the two are effectively identical as far as the human eye's capacity to observe differences is concerned.)

If you care about editing latitude (which I believe is pretty much THE case here, that everyone is really concerned about editing latitude, given how much the shadow lifting ability of the D800 is brought into the discussion), then you should only concern yourself with DXO's "Screen DR" measure. Screen DR is an ACTUAL MEASURE...it is directly measured from the RAW image data itself, without any extrapolating or resizing or anything else inbetween. It is a REAL value, it is a REAL WORLD value, and it actually tells you something explicit that directly affects an actual post-processing ability you are probably actually concerned about.

Ignore Print DR. It's mathematic magic that doesn't really tell you anything, and is only part of a larger <sarcasm> game of mathematic magic to reduce the complexity of digital cameras to a single number that supposedly tells you everything you need to know about comparing one camera to another on an IQ basis. </sarcasm>
 
Upvote 0
Here is a pretty crappy image shot with a completely outdated tech Canon sensor, that many posters here will adamantly tell you is impossible to do. Now I know it is no award winner, but I also know it would get me out of the sh-t if it was an important moment. I just thought it might be interesting to post an actual photographic example of the appallingly bad Canon tech.

Anybody care to guess how many stops under exposed the image was? It was at a wedding reception and the on camera flash that I was bouncing hadn't recharged in time, so did not fire.
 

Attachments

  • 1.jpg
    1.jpg
    147.7 KB · Views: 548
Upvote 0
dilbert said:
jrista said:
...
If you have a 14.4 stop scene that you try to capture with the D800 or D600 in a single frame, you will be clipping 1.2 stops of highlight and/or shadow detail. That 1.2 stops is lost, gone for good, never recoverable...because the SENSOR (i.e. the RAW image) only has 13.2 stops of DR. If you clip 1.2 stops of highlights, pulling down highlights in post will result in clearly blown highlights...you can make them gray rather than white, but those regions will always lack detail. If you clip 1.2 stops of shadows, lifting in post will eventually result in nothing but the stretching of read noise by 1.2 stops, you won't be recovering those lost 1.2 stops of shadows. The D800 is only capable of capturing 13.2 stops of dynamic range in a single shot, as far as editing latitude (the ability to lift shadows without also lifting noise so much that it affects the image) is concerned.
...

And with Canon equipment, you're at about 10 stops of usable dynamic range.

With Canon equipment you have about 11 stops. It is usually 10.something, and the something is usually closer to .9 than not. For example, the 5D III has 10.97 stops of dynamic range...11 stops. A full stop is very meaningful when discussing DR...it is a DOUBLING of the ratio, another 3dB. You can't just round off the entire decimal fraction and call it an even 10. Your just as bad as AvTvM when it comes to facts man...as in, you pervert and twist them at will to win the argument.

And, just to make sure everyone knows where MY FACTS are coming from:
 

Attachments

  • 5DII-DXO-DR.jpg
    5DII-DXO-DR.jpg
    46.3 KB · Views: 564
Upvote 0
dilbert said:
jrista said:
dilbert said:
jrista said:
...
If you have a 14.4 stop scene that you try to capture with the D800 or D600 in a single frame, you will be clipping 1.2 stops of highlight and/or shadow detail. That 1.2 stops is lost, gone for good, never recoverable...because the SENSOR (i.e. the RAW image) only has 13.2 stops of DR. If you clip 1.2 stops of highlights, pulling down highlights in post will result in clearly blown highlights...you can make them gray rather than white, but those regions will always lack detail. If you clip 1.2 stops of shadows, lifting in post will eventually result in nothing but the stretching of read noise by 1.2 stops, you won't be recovering those lost 1.2 stops of shadows. The D800 is only capable of capturing 13.2 stops of dynamic range in a single shot, as far as editing latitude (the ability to lift shadows without also lifting noise so much that it affects the image) is concerned.
...

And with Canon equipment, you're at about 10 stops of usable dynamic range.

With Canon equipment you have about 11 stops. It is usually 10.something, and the something is usually closer to .9 than not. For example, the 5D III has 10.97 stops of dynamic range...11 stops. A full stop is very meaningful when discussing DR...it is a DOUBLING of the ratio, another 3dB. You can't just round off the entire decimal fraction and call it an even 10. Your just as bad as AvTvM when it comes to facts man...as in, you pervert and twist them at will to win the argument.

Ok, let me join the DxO bashers and say this:

DxO's representation of DR overestimates the amount of usable DR on Canon cameras and is therefore DxO is trash in that whilst DxO might say the 5D III has 10.97 stops of DR, you can't use all of that due to noise (which DxO don't take into account), leaving you with less than 10.

There, do you feel better now?

OMG. You really just don't get it. The 10.97 stops is SCREEN DR!!! That's REAL DR!! The DXO Print DR for the 5D III is 11.74 stops. The 10.97 is NOT an overestimation...it is THE REAL DEAL. I mean, I even included a screenshot of DXO's actual DR screen for you so you would know EXACTLY what I was talking about!

You are so off base, however, that you just aren't satisfied with that...you have to make up some further imagined discrepancy in order to reduce it by another full stop. That's just ridiculous! I ALREADY factored in DXO's MISREPRESENTATION OF PRINT DR when I said they had 10.97...I took that value from the only real measurement DXO has for DR! You don't need to factor in the misrepresentation again!

Do you get it now, why people rage against you so hard? You JUST AREN'T FACTUAL. Your misrepresenting the facts even worse than DXO misrepresents them, and that's really saying something...
 
Upvote 0
jrista said:
neuroanatomist said:
dilbert said:
There, do you feel better now?

Does your distortion of facts, either through intentional obfuscation or simple lack of comprehension, make you feel better? I just see it as rather sad and pathetic...

^-- Seriously --^

I....WOW....just WOW.... ???

You find it surprising? I lost what shred of remaining respect I had for dilbert when he called a Canon 'box lens' (DIGISUPER field lens for broadcast TV) a camera, and refused to admit he was wrong.
 
Upvote 0
That's a good explanation on the previous page JR, thanks for posting that for those who need it.

The only thing I'd add to that even if we know the RMS (or averaged) read noise numbers used for their base DR calculation, we would still have no clue as to HOW UGLY lifted shadows will look. That's what pushed-4-stops-lens-cap-on shots are for! :)

If DxOmark would publish the peak-to-peak read noise, and they'd still likely have to do a compromise of that, or add a read-noise histogram to be more thorough, then we would have a more clear indicator of whether that noise is random in nature, and therefore subjectively less bothersome, or patterned, and therefore more difficult to mask.

E.g. In the case of the 7Ds I looked at, alternating vertical bands of (8 per, i think) pixels had different levels of read noise and this produces obvious stripe artefacts when lifting shadow areas. Since the ratio of read noise to signal there is already very low the read noise difference is made obvious.
In some other cameras the read noise is more uniformly distributed and is less obvious when lifting shadows.

Imaging Resource provides a different way of presenting the DR measurement, by providing DR figures at different signal to noise ratios, still likely based on RMS read noise, but it provides more of a spread of figures which can be helpful and they also publish more noise data as well, including individual RGB curves.

Here they are for the 7D, and the tiny-sensored Pentax Q.

www.imaging-resource.com/PRODS/E7D/Z00100_ALO_OFF_acrauto_Step_2.gif
www.imaging-resource.com/PRODS/PENTAXQ/Z0125_ACRauto_Step_2.png

Even I'll say I'd take the 7D's file over the Q's file for lower overall (BER) noise on a real image - at least until I wanted to lift shadows, where the tiny Q's lack of FPN can then show the 7D how that's done.

http://a2bart.com/tech/allcamdknz.htm

Then throw in unit-to-unit variability and improvements that may be made during the mfg run of a given body and there are more things to argue about. For instance, my late model 40D, a model initially reputed to have some noise issues, provides noticeably less FPN in pushed files than my early production 7D or early 5D2 did, yet some claim noisy 40Ds and clean 7Ds or 5D2s.

So, back to DxOmark - or any site that does not define how they make those measurements or what baselines they use, even the uninterpretted the data provided is less than ideal or conclusive. (But it's still more informative than "real world pictures" for such matters)
The best rough data from DxOmark on this is the FULL SNR curves because you can see how each sensor performs at each ISO and at a lot of different levels from dark to white levels.
/editorial
 
Upvote 0
dilbert said:
neuroanatomist said:
dilbert said:
What fact am I distorting?

"And with Canon equipment, you're at about 10 stops of usable dynamic range."

That particular quote comes from Gale Tattersall - the guy that used the 5D Mark II for Home where they used the 5D Mark II to shoot TV episodes. So what you're now saying is that a well respected professional is wrong? (His comment of there being about 10 stops of usable response from the Canon was to contrast with the measured response of over 11.)

House!

What I find really interesting is when the DR/FPN Evangelists are actually challenged with an image that squarely disputes their assertions religion, they all ignore it.
 
Upvote 0
Upvote 0
Aglet said:
That's a good explanation on the previous page JR, thanks for posting that for those who need it.

The only thing I'd add to that even if we know the RMS (or averaged) read noise numbers used for their base DR calculation, we would still have no clue as to HOW UGLY lifted shadows will look. That's what pushed-4-stops-lens-cap-on shots are for! :)

If DxOmark would publish the peak-to-peak read noise, and they'd still likely have to do a compromise of that, or add a read-noise histogram to be more thorough, then we would have a more clear indicator of whether that noise is random in nature, and therefore subjectively less bothersome, or patterned, and therefore more difficult to mask.

Aye! I totally agree! Some actual visual examples of the noise patterns, and a totally public methodology that explains how they analyze read noise, how they compute the average (is it actually an RMS? Is it a meadian? A mean?), would be much more helpful.

There is no question that the 5D III, 6D and 1D X greatly improved the quality of their read noise. The pronounced horizontal and vertical cross hatching of the previous generation has been considerably reduced, and what remains one is fairly hard pressed to find in an average lift (even at ISO 100/200). Thankfully, Roger Clark does produce visual examples of the noise patterns for all ISO settings, which you can find here (Note: These have been tonal compressed in order to maximize the noise patterns...keep in mind, this noise exists in the bottom couple stops of DR, and requires a SIGNIFICANT lift to become this visible in an actual photo):

1D X: http://clarkvision.com/articles/evaluation-canon-1dx/index.html
1D IV: http://clarkvision.com/articles/evaluation-canon-1div/index.html
5D III: http://clarkvision.com/articles/evaluation-canon-5diii/index.html
5D II: http://clarkvision.com/articles/evaluation-canon-5dii/index.html
6D: http://clarkvision.com/articles/evaluation-canon-6d/index.html
7D: http://clarkvision.com/articles/evaluation-canon-7d/index.html

(You can clearly see the 7D's vertical banding problem. The 5D II has much more pronounced banding at ISOs 100, 200, and 400. It also clearly has more noise at all ISO settings than either the 5D III or 6D, and the more random noise of the 5D III vs. the 5D II is quite clear. NOTE: The 5DII page includes ISO 50 at the top, so you need to scroll that page down a little bit more to compare ISO for ISO with any of the other cameras.)

Aglet said:
E.g. In the case of the 7Ds I looked at, alternating vertical bands of (8 per, i think) pixels had different levels of read noise and this produces obvious stripe artefacts when lifting shadow areas. Since the ratio of read noise to signal there is already very low the read noise difference is made obvious.
In some other cameras the read noise is more uniformly distributed and is less obvious when lifting shadows.

Aye, the 7D's vertical banding is pretty bad, but it is most pronounced at the lower ISO settings. Above ISO 800 (which is where most bird photographers live most of the time) it's invisible. (You can see this in Roger Clark's samples above.) ISO 400, ironically, tends to be the worst ISO for banding...I guess because it is still about as pronounced as at ISO 100, but you have two stops less DR. At ISO 100, because you have so much more DR, it is actually less of a problem.

Aglet said:
Imaging Resource provides a different way of presenting the DR measurement, by providing DR figures at different signal to noise ratios, still likely based on RMS read noise, but it provides more of a spread of figures which can be helpful and they also publish more noise data as well, including individual RGB curves.

Here they are for the 7D, and the tiny-sensored Pentax Q.

www.imaging-resource.com/PRODS/E7D/Z00100_ALO_OFF_acrauto_Step_2.gif
www.imaging-resource.com/PRODS/PENTAXQ/Z0125_ACRauto_Step_2.png

Even I'll say I'd take the 7D's file over the Q's file for lower overall (BER) noise on a real image - at least until I wanted to lift shadows, where the tiny Q's lack of FPN can then show the 7D how that's done.

http://a2bart.com/tech/allcamdknz.htm

Hmm, I'd never seen the a2bart.com page before. That's a handy page! It is quite interesting in that you can actually see the Nikon range of cameras are clearly clipping the negative signal, vs. Canon which uses a bias offset and therefor preserves negative signal. I've been getting much more into astrophotography lately. Nikon cameras are called "star eaters" because they all clip the deepest shadows (they are gone for good, absolutely ZERO hope of recovering them, even with dark frames). This actually results in dimmer stars which could be recovered by the application of dark and bias frames, to completely disappear from the signal, rendering them "eaten" by Nikon.

Canon cameras, on the other hand, keep the entirety of the image signal, including all of the values below the bias offset, and don't throw away anything. This results in more noise in Canon cameras at low ISO, however you can use dark and bias frames to nearly eliminate that noise, and almost fully recover the real image signal detail below the bias offset. For this reason, Canon cameras are used almost exclusively in the amateur deep sky astrophotography world (by those who aren't willing to spend $4000+ on a dedicated, supercooled CCD monochrome astrocam, that is), where SNR is often a mere fraction of what it is in normal photography, and every pixel counts.

Aglet said:
Then throw in unit-to-unit variability and improvements that may be made during the mfg run of a given body and there are more things to argue about. For instance, my late model 40D, a model initially reputed to have some noise issues, provides noticeably less FPN in pushed files than my early production 7D or early 5D2 did, yet some claim noisy 40Ds and clean 7Ds or 5D2s.

I actually think it is a little ironic. The 5D II was a MASSIVELY POPULAR camera...yet it really had some BAAAD noise performance. The 5D III is quite a bit better (than both the 5D II and D800), all the way up through ISO 12800...and yet, it still doesn't get any justice.

Aglet said:
So, back to DxOmark - or any site that does not define how they make those measurements or what baselines they use, even the uninterpretted the data provided is less than ideal or conclusive. (But it's still more informative than "real world pictures" for such matters)
The best rough data from DxOmark on this is the FULL SNR curves because you can see how each sensor performs at each ISO and at a lot of different levels from dark to white levels.
/editorial

Aye, DXO's raw measurements (the actual measurements) are useful data. I don't really have a problem with DXO's data...I really just have a problem with how they interpret it, how they weight it (weighting has absolutely ZERO place in any system that aims to objectively compare anything), and how they try to reduce it all to a single scalar number.
 
Upvote 0
dilbert said:
privatebydesign said:
What I find really interesting is when the DR/FPN Evangelists are actually challenged with an image that squarely disputes their assertions religion, they all ignore it.

hmmm?? Don't get "FPN"?

Fixed Pattern Noise, it is the banding that can be seen if you process files badly, particularly if they are badly exposed files processed badly.
 
Upvote 0
privatebydesign said:
dilbert said:
privatebydesign said:
What I find really interesting is when the DR/FPN Evangelists are actually challenged with an image that squarely disputes their assertions religion, they all ignore it.

hmmm?? Don't get "FPN"?

Fixed Pattern Noise, it is the banding that can be seen if you process files badly, particularly if they are badly exposed files processed badly.

FPN = Five-stops Pushed Noise
 
Upvote 0