DxOMark Sensor Performance: Nikon vs. Canon

[jrista]

(And if you think my analogy with print is invalid, you need to *explain* why.  I may be wrong, but simply repeating your assertion that this is so does not show it to be the case).

I think this is a critical point, and possibly the root of the contention of the "DXO DR naysayers." As someone who prints a lot myself, perhaps I can offer some insight.

Assuming you print at native resolution, printing does not average the original amount of information into something less. A print at native resolution represents the same, original, fully detailed image information at a higher density. That higher density may be anywhere from three to six times greater than the 100ppi of my screen (or even more dense, for those who use a 72ppi screen). That is very different than downsampling, which is destructive to information. Generally speaking, I downscale my images for display online. I print at native size at home, or perhaps slightly enlarged, and I might upscale (at a lower PPI) for large canvas prints from a lab. Depending on the amount of cropping, I might print as low as 8x10 with a small amount of downscaling (no where near a two-fold reduction for proper averaging, though), although most of the time it is 11x17", 12x18", 13x19" (the latter being my most common print size, usually at 300ppi).

There are other problems with DXO calling their rated DR "Print DR", though. Assuming you are using a godly form of paper, such as Innova FibaPrint Gloss, which has a dMax of over 2.7, you might be able to get 7 stops or so from a print. Your average fine art print paper has a dMax randing from around 1.3 to 1.5 on average to 1.75 or so for some of the more recent higher-end fine art papers. That gets you maybe 5-6 stops of DR. Most papers also don't have particularly high white points. Again, Innova FibaPrint bleached or bright white glossy papers have a pretty high brightness, and the Moab Lasal papers might be some of the brightest papers available with the highest L* I've ever seen (although they achieve it via OBAs, which require proper lighting properly produce the papers full DR.) Despite that, none of those papers, even when printed with top of the line pigment inks like Lucia or UltraChrome, will offer much more than around 7 stops of DR.

So that takes us back to the definition of DR. I'm happy to accept that DXO has a purely mathematical interpretation of DR, the ratio between white point (maximum saturation) and black point (noise floor). Again, though, I am not sure it is a useful or realistic definition of what dynamic range is. When one thinks about the value of dynamic range in digital photography, the first thing that usually comes to mind is the ability to recover useful detail from deep shadows. I say from the shadows, as I think any photographer who uses digital knows that it is critical to preserve the highlights, as once they are clipped, detail is well and truly gone. Since detail generally "fades" into noise on the shadow end, an improvement in photographic dynamic range has the benefit of improving one's ability to "recover", or pull up, detail out of the deep shadows.

Now, in this respect, I don't think anyone who has been involved in this debate will honestly dispute the fact that Sony Exmor sensors offer more dynamic range. That has never been in dispute...its a simple fact, clear as day for anyone who has seen or directly used a Nikon RAW image from any camera with an Exmor sensor. The dispute on record here, if I may define it according to my own views as well as that which I've read from other DXO DR naysayers, is this:

What value does DXO PrintDR (the mathematically derived ratio between white point (maximum saturation, FWC) and black point (electronic noise floor)) have in a real-world context?

From the standpoint of simply moving the black point in a downsampled image, the only thing that occurs is shadows become darker. One LOSES information during the process of downsampling, so the primary benefit of having additional DR in the hardware no longer applies. In the context of viewing images on a computer screen, primarily done via the web, having a deeper black point might be valuable. Computer screens generally support a much deeper black point than actual prints on paper (particularly prints on high quality fine art paper), although none actually support 14 stops of DR regardless, and the average consumer screen is only 6-bit, so roughly the same DR as a print.

When it comes to real print, assuming one is printing at native size, or an upsampled image, original detail is preserved or slightly softened, but none of it is lost due to downsampling. Regardless, assuming one even does significantly downsample a D800 image so they can print at 8x10", even printed on the highest dMax papers on the market with the brightest L* rating, your going to get HALF the DR you should supposedly be getting from DXO's 14.4 stop Print DR rating. If we assume you tweak the white and black points, curves, and levels in Photoshop to manually and ideally compress all that extensive 14.4 stops of DR into the 5-7 stops of DR your paper is capable of, then the additional mathematical DXO DR (darker black point) offered by downsampling is still of no benefit. It might actually make it more difficult to compress a greater range of shadow tones into the limited dynamic range of your paper, resulting in some funky tonality.

These are the issues I have with DXO's "Print DR" statistic. It is an unrealistic number, purely mathematical in derivation, that does not seem to translate into any real-world improvement in "print". It MIGHT offer slightly better blacks when downsampled images are viewed on the web. Blacks might be just a bit richer, assuming someone actually has a properly calibrated screen with high enough bit depth to actually support it. The average home user still uses a 6-bit screen. Most serious photographers have an 8-bit screen, and some serious professionals might have thousand-dollar 10-bit screens. At best, an observer viewing a photo on the web will be able to observe about 8-10 stops of DR, although on average we are still back to 6 stops on average.

At best, DXO's downsampled DR rating should probably be called Web DR. It is not detail-preserving Photographic DR, as upon downsampling you lose detail.  It is definitely not Print DR, since a print is inherently more about color richness and gamut than white-to-black point dynamic range. The depth of blacks sometimes matters in a print, however the deeper your black point in print, the harder it tends to be to actually discern fine shadow detail. Things look richer and more contrasty, but not necessarily more detailed. Papers with a higher white point AND slightly less dense blacks tend to look better, despite having far lower dynamic range than the original photo.

So, what is the value of DXO Print DR? Realistically, practically, physcally...what do I actually gain by downsampling my full-detail RAW into a smaller-sized TIFF? For that matter, what value does DXO Print DR have if I save as a compressed JPEG for viewing on the web? Are we really just talking about a DXO weighted score, and nothing more? If so, should it really be called Dynamic Range, or is there a better term DXO could use that wouldn't come off as some kind of sketchy maneuvering (real or simply perceived) of their results in favor of a major monetary contributor?

[tnargs]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. Hello DxO!

Can you imagine a food nutrition analysis website published by scientists, where they publish highest scores for palm oil and birthday cake, and lowest scores for raw fish and broccoli. After delving several levels further into their website you find other surprising and almost bizarre scores and numerics. Then after delving another 40 pages into explanatory papers and definitions, if you are sufficiently scientific in mindset, you will discover that there are no flaws in their logic or measurement methods and their system is completely consistent within itself. How useful is this website? How misleading is it?

[jrista]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Well, I wouldn't go so far as to say DXO in its entirety should be completely ignored. My only beef with DXO is quite specifically their Print DR "score", for all the reasons I stated above. DXO does provide valuable statistics and measurements. I just have a hard time with their scoring structure being so heavily weighted in favor of a statistic that...seems entirely meaningless, and utterly lacking in real-world value.

[elflord]

(And if you think my analogy with print is invalid, you need to *explain* why.  I may be wrong, but simply repeating your assertion that this is so does not show it to be the case).

I think this is a critical point, and possibly the root of the contention of the "DXO DR naysayers." As someone who prints a lot myself, perhaps I can offer some insight.

Assuming you print at native resolution, printing does not average the original amount of information into something less. A print at native resolution represents the same, original, fully detailed image information at a higher density. That higher density may be anywhere from three to six times greater than the 100ppi of my screen (or even more dense, for those who use a 72ppi screen). That is very different than downsampling, which is destructive to information. Generally speaking, I downscale my images for display online. I print at native size at home, or perhaps slightly enlarged, and I might upscale (at a lower PPI) for large canvas prints from a lab. Depending on the amount of cropping, I might print as low as 8x10 with a small amount of downscaling (no where near a two-fold reduction for proper averaging, though), although most of the time it is 11x17", 12x18", 13x19" (the latter being my most common print size, usually at 300ppi).

If you're comparing two different cameras, it doesn't make sense to compare dynamic range at two different resolutions (and therefore different noise/resolution tradeoffs)

As long as your sampling to some known target resolution (whether it's 2400x3000 or 3300x5100  or 3900x5700), you need take into account that the higher resolution camera will benefit from either more downsampling or less upsampling.

For any given print size, as long as dynamic range per pixel is equal, you always get either better dynamic range (lower black point) or more detail (resolution) from a higher megapixel camera.

From the standpoint of simply moving the black point in a downsampled image, the only thing that occurs is shadows become darker.

No. The point at which the SNR is 0db moves down. So does the point where the SNR is  5db, 10db, etc. The shadows don't become darker, they become less noisy

One LOSES information during the process of downsampling,

I'm not sure which information you're talking about "losing". A 40 megapixel camera sampled to 10 megapixels doesn't have less "information" than the image from a 10 megapixel camera.

The point is to put everyone on the same playing field.

If you're looking at a measure in isolation, perhaps the screen score is easier to understand. But if you're comparing sensors with different resolutions, you need to make some adjustment to account for the fact that the higher resolution sensor has more pixels and therefore more photographic detail.

I think the point about only 6 stops of dynamic range being available for paper is a bit of a digression -- if you don't need more than 6 or maybe 8 stops for some safety margin, why would you care about dynamic range ? Any modern DSLR has more than that.

[jrista]

The only thing I can gather from the Pro-DXODR camp's arguments so far, is that we are not really comparing images on a dynamic range basis with DXO's Print DR statistic. We are comparing images on a noisiness basis. Black points and white points and noise may make for a neat and simple definition of DR, but as far as I can gather, DR is not what what is of interest here. The amount of noise, and how that noise presents...explicitly in relation to other images from other sensors...is what is of interest here. If the sole purpose of downscaling a large megapixel image to the size of a smaller megapixel image (or, for that matter, the inverse) is to see which camera is NOISIER, then I have no issue. I don't dispute the necessity of that, although I've never called the improvement in IQ when downsampling a large image to a small image an improvement in dynamic range.

I do dispute that you actually gain anything beneficial by downscaling a high megapixel image to a smaller size, in a real-world context (as differentiated from a "clean-room" test and compare context, within which normalizing noise for the purposes of subjective IQ comparison DOES make sense). I dispute the simple notion of scaling at all to determine a sensor's dynamic range, given that the point of having MORE dynamic range is to preserve detail under a wider range of tones, and in the case of digital photography allow that detail to be "recovered" via post-process exposure adjustments. Adjustments that will only really work properly when editing a RAW...which intrinsically cannot be scaled.

I'm not sure which information you're talking about "losing". A 40 megapixel camera sampled to 10 megapixels doesn't have less "information" than the image from a 10 megapixel camera.

Still assuming that one is perpetually comparing camera sensors. Statistical knowledge about a sensor is not necessarily solely useful for the purpose of comparing them. If I buy a 40mp camera, I have no intention of comparing it to anything else. I have the intention of using it...as a 40mp camera. As such, I want the full stinkin fourty megapixels, not something less. I also want to know what my camera is capable of natively, unscaled, untainted. I want to know what my 40mp camera can do itself, not how well it might do thing X in comparison to cameras A, B, and C! In other words, I want to know its intrinsic capabilities, not its relative capabilities.

In that context, scaling my 40mp image DOWN to any size below its native size means a LOSS of information...a true and realized reduction in real image detail that used to exist, and no longer does at 10mp. It also means I have to print at a smaller size, which could be counted as a different kind of loss...but I guess that is beside the point.

The point is to put everyone on the same playing field.

Well, that is one of the points. It is not the only point of having statistical information about a camera's sensor, as noted above.

If you're looking at a measure in isolation, perhaps the screen score is easier to understand. But if you're comparing sensors with different resolutions, you need to make some adjustment to account for the fact that the higher resolution sensor has more pixels and therefore more photographic detail.

Agreed. From the standpoint of normalizing noise to compare how noisy one sensor is vs. another (which is what I gather the DXO Print DR statistic is all about based on the arguments from you an LTRLI), then sure, you should be normalizing image size. I've argued that myself in the past, although I've never called the change in noise an improvement in dynamic range before, but thats probably simply because I define dynamic range a bit differently than DXO (although admittedly in a different context as well...that of "real-world digital RAW post-processing" where most of us photographers actually live, so it is not necessarily to say DXO's definition is wrong...just meant for a different context.) I've simply called downscaling for the sake of comparing noise the normalization of noise. Something along the lines of "Normalized IQ Score" would be a much better term for this than "Print DR", though, as it really has nothing whatsoever to do with print. Print, paper, inks, ink density, lighting, etc. make actual print so far removed from what DXO is actually trying to demonstrate with this statistic that they should really rename "Print DR" to something less controversial, and something more meaningful.

I believe Imatest, technically a competitor to some of DXO's other business I guess, named their score for roughly the same thing accordingly: SQF, or Subjective Quality Factor. I believe DXO "Print" statistics are really another form of Kodak's SQF. I believe Imatest got the name right for IQ measurements as well...they simply call it MTF, rather than something more obscure like "Screen".

I think the point about only 6 stops of dynamic range being available for paper is a bit of a digression -- if you don't need more than 6 or maybe 8 stops for some safety margin, why would you care about dynamic range ? Any modern DSLR has more than that.

Sorry, but it's been my entire point all along, as my complaint is with the fact that DXO sells "Print DR" as something it is not, by its very name, even. Or, at the very least, they explain their modeling system in a way that many people have incorrectly interpreted, as people are inevitably going to think "ink on paper" when they hear the word "print". I also believe the heavily weighted use of "Print DR", if it is weighted rather than purely a measured statistic (I can't find the page right now that describes how and where DXO adds "bonus points") in their scoring mechanism is extremely misleading.

Additionally, the same statistics, assuming they are accurate enough in the broader context (which in the case of Print DR, it is not...it has a very narrow and specific usage solely in the context of comparing sensors scored by DXO), can be used to learn about what a sensor offers in and of itself outside of any context of comparison. Such as how much exposure latitude do I REALLY, ACTUALLY have when I am pushing shadows around with my RAW images in a RAW editor. Or, say, what the sensor offers when ACTUALLY PRINTING IT AT NATIVE SIZE (so no scaling or averaging of any kind)? DXO's Print DR statistic is insufficient to answer those questions, while Screen DR, effectively being hardware measurements as they are otherwise untainted by software, IS sufficient to answer those questions. Hence my reliance on Screen DR to describe intrinsic capabilities in contexts where relative capabilities are meaningless.

Perhaps my complaint is simply with how DXO sells, names, uses, and refers to their statistics. If they had used the terms "MTF" or "Measurements" and "SQF" in place of "Screen" and "Print" respectively, I probably would have never had a problem. Labeling their overall dynamic range score as the "Landscape" score (immediately putting it in a context where actual camera USERS might literally try to photograph scenes with more than 14 stops of real-world dynamic range), then stuffing in a number that (potentially greatly) inflates the intrinsic hardware capabilities, is excessively misleading. At the very least, SQF is explicitly defined as a way to produce normalized comparisons for the purpose of accounting for "viewer experience"...a subjective factor, which sounds a lot like what DXO's "Print" scores are supposed to be. SQF has also been shown to be an ideal way to statistically measure image IQ produced by digital sensors (when, ironically, it wasn't so ideal for measuring IQ from film), one which has never risen so much ire as DXO's "Print DR" statistic.

[LetTheRightLensIn]

(And if you think my analogy with print is invalid, you need to *explain* why.  I may be wrong, but simply repeating your assertion that this is so does not show it to be the case).

I think this is a critical point, and possibly the root of the contention of the "DXO DR naysayers." As someone who prints a lot myself, perhaps I can offer some insight.

Assuming you print at native resolution, printing does not average the original amount of information into something less. A print at native resolution represents the same, original, fully detailed image information at a higher density. That higher density may be anywhere from three to six times greater than the 100ppi of my screen (or even more dense, for those who use a 72ppi screen). That is very different than downsampling, which is destructive to information. Generally speaking, I downscale my images for display online. I print at native size at home, or perhaps slightly enlarged, and I might upscale (at a lower PPI) for large canvas prints from a lab. Depending on the amount of cropping, I might print as low as 8x10 with a small amount of downscaling (no where near a two-fold reduction for proper averaging, though), although most of the time it is 11x17", 12x18", 13x19" (the latter being my most common print size, usually at 300ppi).

There are other problems with DXO calling their rated DR "Print DR", though. Assuming you are using a godly form of paper, such as Innova FibaPrint Gloss, which has a dMax of over 2.7, you might be able to get 7 stops or so from a print. Your average fine art print paper has a dMax randing from around 1.3 to 1.5 on average to 1.75 or so for some of the more recent higher-end fine art papers. That gets you maybe 5-6 stops of DR. Most papers also don't have particularly high white points. Again, Innova FibaPrint bleached or bright white glossy papers have a pretty high brightness, and the Moab Lasal papers might be some of the brightest papers available with the highest L* I've ever seen (although they achieve it via OBAs, which require proper lighting properly produce the papers full DR.) Despite that, none of those papers, even when printed with top of the line pigment inks like Lucia or UltraChrome, will offer much more than around 7 stops of DR.

So that takes us back to the definition of DR. I'm happy to accept that DXO has a purely mathematical interpretation of DR, the ratio between white point (maximum saturation) and black point (noise floor). Again, though, I am not sure it is a useful or realistic definition of what dynamic range is. When one thinks about the value of dynamic range in digital photography, the first thing that usually comes to mind is the ability to recover useful detail from deep shadows. I say from the shadows, as I think any photographer who uses digital knows that it is critical to preserve the highlights, as once they are clipped, detail is well and truly gone. Since detail generally "fades" into noise on the shadow end, an improvement in photographic dynamic range has the benefit of improving one's ability to "recover", or pull up, detail out of the deep shadows.

Now, in this respect, I don't think anyone who has been involved in this debate will honestly dispute the fact that Sony Exmor sensors offer more dynamic range. That has never been in dispute...its a simple fact, clear as day for anyone who has seen or directly used a Nikon RAW image from any camera with an Exmor sensor. The dispute on record here, if I may define it according to my own views as well as that which I've read from other DXO DR naysayers, is this:

What value does DXO PrintDR (the mathematically derived ratio between white point (maximum saturation, FWC) and black point (electronic noise floor)) have in a real-world context?

From the standpoint of simply moving the black point in a downsampled image, the only thing that occurs is shadows become darker. One LOSES information during the process of downsampling, so the primary benefit of having additional DR in the hardware no longer applies. In the context of viewing images on a computer screen, primarily done via the web, having a deeper black point might be valuable. Computer screens generally support a much deeper black point than actual prints on paper (particularly prints on high quality fine art paper), although none actually support 14 stops of DR regardless, and the average consumer screen is only 6-bit, so roughly the same DR as a print.

When it comes to real print, assuming one is printing at native size, or an upsampled image, original detail is preserved or slightly softened, but none of it is lost due to downsampling. Regardless, assuming one even does significantly downsample a D800 image so they can print at 8x10", even printed on the highest dMax papers on the market with the brightest L* rating, your going to get HALF the DR you should supposedly be getting from DXO's 14.4 stop Print DR rating. If we assume you tweak the white and black points, curves, and levels in Photoshop to manually and ideally compress all that extensive 14.4 stops of DR into the 5-7 stops of DR your paper is capable of, then the additional mathematical DXO DR (darker black point) offered by downsampling is still of no benefit. It might actually make it more difficult to compress a greater range of shadow tones into the limited dynamic range of your paper, resulting in some funky tonality.

These are the issues I have with DXO's "Print DR" statistic. It is an unrealistic number, purely mathematical in derivation, that does not seem to translate into any real-world improvement in "print". It MIGHT offer slightly better blacks when downsampled images are viewed on the web. Blacks might be just a bit richer, assuming someone actually has a properly calibrated screen with high enough bit depth to actually support it. The average home user still uses a 6-bit screen. Most serious photographers have an 8-bit screen, and some serious professionals might have thousand-dollar 10-bit screens. At best, an observer viewing a photo on the web will be able to observe about 8-10 stops of DR, although on average we are still back to 6 stops on average.

At best, DXO's downsampled DR rating should probably be called Web DR. It is not detail-preserving Photographic DR, as upon downsampling you lose detail.  It is definitely not Print DR, since a print is inherently more about color richness and gamut than white-to-black point dynamic range. The depth of blacks sometimes matters in a print, however the deeper your black point in print, the harder it tends to be to actually discern fine shadow detail. Things look richer and more contrasty, but not necessarily more detailed. Papers with a higher white point AND slightly less dense blacks tend to look better, despite having far lower dynamic range than the original photo.

So, what is the value of DXO Print DR? Realistically, practically, physcally...what do I actually gain by downsampling my full-detail RAW into a smaller-sized TIFF? For that matter, what value does DXO Print DR have if I save as a compressed JPEG for viewing on the web? Are we really just talking about a DXO weighted score, and nothing more? If so, should it really be called Dynamic Range, or is there a better term DXO could use that wouldn't come off as some kind of sketchy maneuvering (real or simply perceived) of their results in favor of a major monetary contributor?

The value of it is to compare cameras more fairly than not using it. If you print those files you'll notice that printed to the same size and viewed from the same distance the same thing happens 40MP source vs 12MP source. Or if you print both at 8MP and then put one print say 10' farther away from the other.

If you don't normalize then if you go by your screen DR all the time then you'd think that  a 20x20" print from a 36MP source cam would have a lot, lot more nasty noise than one from an 18MP camera using the same generation of technology. It doesn't make sense to compare a 36MP source as if it were the same scale as 18MP since you could always filter noise to 18MP scale or filter and downsample to 18MP and trade away your extra detail information to make it look like the same 18MP pixel level performance of the other camera. You are trying to take an advantage that can be traded various ways into a singular resolution advantage but a disadvantage in all other ways which really isn't a fair way to compare.

If you want to take full advantage of the 36MP advantage over say the 22MP of the 5D3 you can't do that and at the same time also give the extra print version boost to DR, true enough, but if you don't want to take advantage of it then you can.

[LetTheRightLensIn]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. Hello DxO!

Can you imagine a food nutrition analysis website published by scientists, where they publish highest scores for palm oil and birthday cake, and lowest scores for raw fish and broccoli. After delving several levels further into their website you find other surprising and almost bizarre scores and numerics. Then after delving another 40 pages into explanatory papers and definitions, if you are sufficiently scientific in mindset, you will discover that there are no flaws in their logic or measurement methods and their system is completely consistent within itself. How useful is this website? How misleading is it?

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.

Sure he knows about many various things and he also takes the time to very carefully write out long posts filled with technical term while being careful to check his spelling and grammar and thus comes off as more authoritative than most of the people finding fault with some of his key points who quickly dash out sloppy posts riddled with spelling errors or poor grammar or less perfectly written posts because English is not their first language so everyone who doesn't know any better decides that jrista must be the one who knows what he is talking about (and the fact that he is also bashing down a site that scored the brand that most readers here use and spent a ton of money on for some scores also makes for a sneaky, and likely not at all insignificant, further bias in his favor in such a regard in minds of many readers). But for all of his knowledge he is making a number of major conceptual errors, conceptual understanding is different than spouting off this and that from various books and technical papers.

What you write may be somewhat true when it comes to their overall sensor scores and overall sub-scores to some extent but it doesn't apply to their normalization stuff or their plots.

[LetTheRightLensIn]

Still assuming that one is perpetually comparing camera sensors. Statistical knowledge about a sensor is not necessarily solely useful for the purpose of comparing them. If I buy a 40mp camera, I have no intention of comparing it to anything else. I have the intention of using it...as a 40mp camera. As such, I want the full stinkin fourty megapixels, not something less. I also want to know what my camera is capable of natively, unscaled, untainted. I want to know what my 40mp camera can do itself, not how well it might do thing X in comparison to cameras A, B, and C! In other words, I want to know its intrinsic capabilities, not its relative capabilities.

Sure you might want to know how it does at 40MP and what the sensor straight out delivers and then you have the screen plot and it's certainly valid to have some interest in that but most people also do want to know how it does relative to other options and to see whether they are maybe also giving up other things compared to other options at different MP counts or not, most people when they want to compare two cameras to each other want to use the print plots. I am gaining the ability to get a ton more res under good lighting but also giving up the ability to ever do as well for SNR or am I actually not giving up anything there compared to my old camera? etc.

[MarkII]

Can you point me to some reading material?

Try http://en.wikipedia.org/wiki/Entropy_(information_theory

(The sections on Entropy as Information Content and Data Compression are the most relevant).

[tnargs]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. ....

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.....

.... it's been my entire point all along, as my complaint is with the fact that DXO sells "Print DR" as something it is not, by its very name, even. ...

You have a strange way of putting people down just before proceeding to agree with them ... but without ever admitting you agree with them.

[jrista]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. Hello DxO!

Can you imagine a food nutrition analysis website published by scientists, where they publish highest scores for palm oil and birthday cake, and lowest scores for raw fish and broccoli. After delving several levels further into their website you find other surprising and almost bizarre scores and numerics. Then after delving another 40 pages into explanatory papers and definitions, if you are sufficiently scientific in mindset, you will discover that there are no flaws in their logic or measurement methods and their system is completely consistent within itself. How useful is this website? How misleading is it?

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.

Sure he knows about many various things and he also takes the time to very carefully write out long posts filled with technical term while being careful to check his spelling and grammar and thus comes off as more authoritative than most of the people finding fault with some of his key points who quickly dash out sloppy posts riddled with spelling errors or poor grammar or less perfectly written posts because English is not their first language so everyone who doesn't know any better decides that jrista must be the one who knows what he is talking about (and the fact that he is also bashing down a site that scored the brand that most readers here use and spent a ton of money on for some scores also makes for a sneaky, and likely not at all insignificant, further bias in his favor in such a regard in minds of many readers). But for all of his knowledge he is making a number of major conceptual errors, conceptual understanding is different than spouting off this and that from various books and technical papers.

What you write may be somewhat true when it comes to their overall sensor scores and overall sub-scores to some extent but it doesn't apply to their normalization stuff or their plots.

Way to gloss over an insult man! You have some serious skill! Why not tell me what you REALLY think, huh? BTW, resorting to personal attack as a means of argument demonstrates weakness in your own opinion. If you truly think people simply listen to me because I'm a dumb guy who just sounds smart, well, might not want to weaken your own position like that. I might dumly conjure up something intelligent in retaliation. Just a thought.

I have no problem with the concepts. The concepts are not the issue. The problem is that we are on entirely different pages about what the issue is. Part of that is a misunderstanding of what DXO DR really refers to, what it is defined as...by MANY people...which in and of itself is WHY we are on different pages (because it is the ambiguity and ease by which readers are mislead by DXO because of the way they describe their results, the terms they choose to use to describe certain scores) that is the problem, and what I dislike. You think I don't understand the need to normalize to properly compare IQ. I DO understand that...if you search these forums, I've argued the normalization point myself many times (and many times well before the D800 came out.)

I've been very clear about exactly what my complaint with DXO is, and very clear about the context within which I voice my complaint. Veiled insults not withstanding, I think it is you who misunderstands the argument we nay-sayers are trying to make. Either that, or you are conveniently ignoring the context, and attempting to use the notion that individuals such as myself "simply don't understand" as a means of trying to strengthen your argument.

Well, ironically, you don't need to strengthen your argument. In the context within which you are arguing, the context of non-isolated results used purely for the purpose of comparisons, DXO's definition of DR is fine and dandy, if confusing based on the way they name their results and scores. Only the people who dig into the mathematical descriptions of their results (which is only the few hypergeeky of the millions who view DXO results and base their purchasing decisions on them) will have the option of gleaning a full understanding. I do not disagree that one needs to normalize image size to determine the difference in how noise levels of a sensor might affect perceived IQ (which, btw, is a subjective or observer-based comparison...SQF would be a better, and standardized, way to provide that information). So, in terms of arguing a narrow point in a narrow context, you guys win. You won a long time ago. People just don't fully understand it because DXO DR is not really how people think about DR in a real-world context...(thus the reason the debate exists in the first place.)

Most people don't understand dynamic range as the ratio between white and black points, or noise in terms of decibels. Neither is DXO's definition of dynamic range the sole valid, concrete definition of dynamic range in the context of a digital image produced by a digital sensor. There are a variety of ways to compute DR, including some offered by standards bodies like ISO (who's ISO 15739-2003 standard aims to provide a standardized way of computing digital image DR in a more "photographic" manner. It clearly defines the types of noise it accounts for, including random or "temporal" noise (photon shot noise) that changes from shot to shot and fixed forms of noise (electronic or read noise) that remain the same from shot to shot.) There are more useful ways to describe dynamic range, and differing contexts within which definitions of dynamic range has meaning (and potentially very different meanings.)

I don't believe there is anything wrong with stating my disdain for the way DXO uses their Print DR score. I don't believe there is anything "dangerous" about me wanting a frequently-quoted resource like DXO to produce more meaningful, more useful results, even if it simply means renaming some of the terms they use to describe their results, for the benefit of potential buyers who will base radical decisions on DXO's scores (such as the dumping of an expensive Canon kit in favor of another expensive Nikon replacement kit, when for their needs they neither need the dynamic range of the D800, nor would benefit from the other traits of the D800 camera body...such as huge image size or slower frame rate when they are an action photographer who needs smaller image size and faster frame rate. Or even for the photographer who does need more DR, but might incorrectly believe the D800 offers an extra 1.2 stops of DR at native, unscaled RAW size than it actually does.)

And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

[LetTheRightLensIn]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. ....

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.....

.... it's been my entire point all along, as my complaint is with the fact that DXO sells "Print DR" as something it is not, by its very name, even. ...

You have a strange way of putting people down just before proceeding to agree with them ... but without ever admitting you agree with them.

Nah I messed up the message and forgot to chop off the last parts and it made it look like his words were my words, my bad, totally messed the post up. I fixed it now.

[LetTheRightLensIn]

And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

You seem to be slowly shifting your position, because you definitely didn't say that you thought it was valid or the best way to go about when comparing between different sensors before.

[AvTvM]

aside of the debate re. comparing pcitures downsized to 8MP and all aspects connected to this ... one more thign that really irks me about DXO

they do NOT state anywhere on their webseite, what lens/es they use in their "sensor" tests, which in reality are not "sensor tests" but rather comparisons of downsized RAW images obtained by taking test pictures using a lens, a camera with sensor, hardware and firmware .. so comparisons of "lens-sensor-hardware-firmware"-combinations. They also state this here:   http://www.dxomark.com/index.php/Publications/DxOMark-Insights/DxOMark-Score

I doubt they are targeting a laser beam at a naked sensor and use their own electronics - hardware and standardized firmware - to measure "pure sensor performance".

There must be a reason, why they are so clandestine about wht lesnes they use for their testeing. Until they fuly dislose all relevant information regarding their test-setup, i completely disregard their results. For all we know, they might be testing Canon cameras with the 50/1.8 and other cameras with a Zeiss 100 Macro.   

[Razor2012]

Hard facts dynamic range

Do you have a link to those figures?