DxOMark Sensor Performance: Nikon vs. Canon

[tnargs]

...As to the continued assertion that downsampling can not increase bit depth, ....

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

[MarkII]

...As to the continued assertion that downsampling can not increase bit depth, ....

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

No - the problem is people who assert that the DXO measurements are "obviously nonsense" because they (wrongly) believe that it is impossible for a down-sampled image to show more per-pixel DR than the original.

DXO use resolution normalised figures in their summaries partly because this is how most people tend to view their images (even the new 15" 'retina' MacBookPro screens are "only" 5MP at full-screen). And if they did not do this, some people would be screaming that they were biased because their numbers favour lower resolution cameras.

It is perfectly reasonable to describe a sensor as having ~13.2 bits at full resolution and ~14.3 bits at a normalised 8MP resolution. Anyone considering buying or using a device like this should be smart enough to understand what this means and take the time and effort to read beyond the headline summaries.

Do people really buy cameras because of one number in a test score? Do they only read the final summary table in a 20-page review before opening their wallets for thousands of dollars???  If so, they probably deserve what they get!

[psolberg]

Wow, site admin flip flops in dxo after saying he would never publish dxo numbers even if canon came ahead. Now it happened and we start to see dxo numbers.....now dxo matters

[elflord]

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

Camera 1 has 36 megapixels and 11 stops of dynamic range per pixel, camera 2 has 20megapixels and 12 stops of dynamic range per pixel.

Which camera has more dynamic range if I display or print their images at the same size ? If I downsample the 36mpx image to 20mpx, I will get more than 11 stops of dynamic range, but do i get more than 12 ?

I think this is a pretty valid question, provided you intend to view the whole image on print or screen, as opposed to just viewing 100% crops.

[TheSuede]

I find it interesting that many would like to make it sound like an "underexposure" is the end of the world, and a photographical failure.

What that stance significates is failing to understand the implications. I spent quite a few years of my life fighting against this erroneous mindset when educating print and pre-press personnel starting out in their first stumbling steps of the "digital revolution". Back then, pre-press personnel was (if possible) an even more tradition-laden and anachronistic group of people than what photographers are today.

As someone mentioned Adams, I have to say that he is a person you really have to admire for the amount of energy and time he spent really getting to know his medium (film), and how you had to expose to get the most out of your envisioned image. He (as he worked with the physicists and chemists at Kodak) set up some really workable testing rules and also inspection criteria to be used when looking at image results.

And he always tried to get the exposure "right".

Now try to understand the implications the sentence in bold above sets. Getting an exposure "right" is NOT the same as trying to make 14-16% gray as you see the image conform to a certain photometric exposure at the film surface! And he never claimed that it was. If there is more pictorial detail that you want to emphasize in the end result (for him; film, development, print) in the shadows - you should really overexpose - compared to the baseline scene average exposure. And if there is "important detail" in the highlights, you need to underexpose [compared to the reference exposure]. He accepted that there was always a compromise between highlight, midtone and shadow local detail. I would recommend a visit to his old home, if his son still has the tours he used to. If you're nice, to him you might even get to see some "behind the scenes" notes and development plans - they're really impressive in attention to detail, and shows how much he mauled his negatives, using both over- and under-exposure, to get "what he wanted" out of them. Regional corrections of + or -3Ev was nothing out of the ordinary either.

In digital, the entire ramification of the process has changed.

Film has more latitude, but much less usable DR within a fixed scene
Film has an bell-shaped MTF modulus, digital is linear
Film clips and blocks gracefully towards both highlight and shadow; and
Digital clips hard against highlights and fades gradually into noise in shadows

This means that you can't treat digital like you did with film, and the most obvious differences are:
a) as long as the entire scene can be contained within the digital DR, you can put the raw-file related exposure (which is not the same as photometric exposure!) wherever you want - it does not effect internal detail contrast.
b) you have to mind the highlights (more) with digital - since when they're clipped, all internal detail contrast is GONE. Impossible to retrieve unless you let software guess what the clipped channel values should be from surrounding color information. This only works for about +0.5Ev from initial clipping.

To underexpose the raw file - again, this is not the same as the photometric exposure! - is the only practically available method to reign in the highlights in the scene. And this is also what Canon does with the "Highlight Tone Priority" or "HTP" option when you shoot jpg. The camera will underexpose the raw file, and then bring image midtone brightness up again - but with smother upper S-curve end of the brightness curve.

Underexposing an image that doesn't have any highlight areas (zones? maybe applicable) you want to protect though - that's only stupid. It hurts image quality.

[LetTheRightLensIn]

...As to the continued assertion that downsampling can not increase bit depth, ....

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

It is described as that but when you want to do a relative comparison you can't compare cameras at different scales otherwise you are comparing noise power at two different frequencies as if they were the same.

[TheSuede]

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

Camera 1 has 36 megapixels and 11 stops of dynamic range per pixel, camera 2 has 20megapixels and 12 stops of dynamic range per pixel.

Which camera has more dynamic range if I display or print their images at the same size ? If I downsample the 36mpx image to 20mpx, I will get more than 11 stops of dynamic range, but do i get more than 12 ?

I think this is a pretty valid question, provided you intend to view the whole image on print or screen, as opposed to just viewing 100% crops.

This is a complex question, since what DxO basically fails to include in their DR vs resolution compensation (the "print" view option) is that no Bayer-based image ever contains equal noise energy all the way up to 1/f. The interpolation stage often called the "de-mosaic" stage necessary (remember, two out of the three channels in each pixel has had to be estimated after the capture...) is effectively filtering the higher noise frequencies out, and tends towards zero at 1/f.

In a layman's terms, you could describe this high-frequency filter as:
-"The noise - or average pixel difference - is stronger in power when you compare two pixels a few pixels apart from each other than if you compare two pixels next to each other."

The end result of this is that the first ~30% of downsampling - down to 70% original scale that is - does not lower image noise power [over the image width] by any significant degree. There wasn't much noise energy in the frequency band that we've filtered away, so what we've basically done is to condense the image information.

But then there's also a more subtle effect. The human eye does not react as strongly to fine-grained noise as it does to coarse-grained noise. This does mean that when you MEASURE the average pixel noise power, it might not have been lowered by any appreciable amount - but when you LOOK at the downsampled image you will perceive the image as less noisy anyway.

So - downsampling 36MP to 20MP would theoretically give you an added:
sqrt(36/20) = 1.34 linear scale
>> convert to log2 >>
log(1.34)/log(2) = 0.42Ev or "bits" of DR

But you wouldn't get 0.42Ev in a real, converted image - you would get maybe 0.1Ev and a much tighter (less objectionable) noise pattern.

From this point on downwards however the noise spectrum could be said to be sufficiently close to a linear 1/f behavior, and you would get the full noise-power lowering effect in practice too. So, continuing down in size would yield the full theoretical gains of log(sqrt(linear scale))/log(2). Together with a lowered resolution per image frame of course... :-)

[dtaylor]

Downsampling doesn't "create detail", but it moves the black point.

* Photographic dynamic range is the range of useful detail.

* As you conceded above, down sampling does not create detail. In fact, it throws detail away.

* Down sampling therefore cannot increase photographic dynamic range.

* DxO's normalized results are therefore false.

Can we move on now?

[dtaylor]

Please feel free to post any unanswered "critiques"

All of them from what I can see.

Far from rebutting, it's not clear to me that you or jrista understand the implications of normalization, or the basic math behind benchmarking,  so aren't in much of a position to criticize it.

http://www.nizkor.org/features/fallacies/

[dtaylor]

Where is your evidence or explanation?

For real, usable DR tests, look to IR and DPReview. (Unfortunately DPReview no longer posts RAW DR, but their tests are still valid for JPEG.)

The explanation has been posted repeatedly, and was posted again just a second ago.

I tested the DR myself and got same results and so have others.

You're making the same mistake as the others. Noise inflates the black point. Down sampling reduces noise, lowering the black point. But the black point is not actually the floor of dynamic range. The floor is the point where usable detail ceases to exist. Down sampling actually throws away detail. It cannot extend DR.

I've seen improvements over that time period. DPReview saw improvements.

On what? 5D3 has actually worse DR than the 1Ds3.

In JPEG the 5D3 is better, as noted by DPReview. Unfortunately they did not post RAW values for the 5D3.

Imaging Resource has Imatest RAW values for both, and the 5D3 is one stop better.

real world, accounting for banding, I'd say D800 does more than 3 better than 5D3 and even ignoring banding it's more than 2.

"Real world"...that would be a switch. Please post real world shots that prove this. Include a transmission step wedge. Oh yes: downloadable RAW files for analysis are mandatory.

And not that I want to go here, but you've finally gotten me partially to this point, and sure even the most brilliant are wrong at times, absolutely true, and I'm not saying who here and on the other forums is who, but there are not only engineers but theoretical physics PhDs and other such, including some who are world renowned, posting in some of these threads and on some other threads and.... not on your side on this.

What is this: an appeal to imaginary authorities? 

http://www.nizkor.org/features/fallacies/

Whether or not the 5D4 has better DR has nothing to do with these stupid threads, and everything to do with their engineers. I have little doubt they are working on it.

Did you know that another division of Canon sent a patent for better DR to the DSLR division and got told to get lost, DR, what?, why? bye. Apparently they didn't even let their engineers look at it! So maybe they do need to be woken up.

Source?

A Canon employee (non-DSLR division) on another forum.

More imaginary friends = no source. Don't repeat the claim without a source.

[dtaylor]

Well, if I have a noisy image (ie with poor DR), it will be usable only at smaller print sizes than a less noisy one. That is a real world example of down-sampling, and if what you claim were true the small print would show exactly the same noise and DR than the larger one.

It does show exactly the same DR. You're making a subjective, aesthetic judgement as to acceptable print/view size due to noise. (And it's not down sampling unless you have a really crummy printer.)

As to the continued assertion that downsampling can not increase bit depth,

No. That's not the argument. (This is like pulling teeth!)

Pay careful attention: down sampling cannot increase *** photographic dynamic range ***

In photography dynamic range is the range of usable detail, not the range between white and black points.

You have a RAW file. It has some noise. Black is not black, but a really dark gray because of the noise. You down sample and kill some of the noise. "Aha! My blacks are blacker! My bit depth is greater!" So what? You didn't gain one drop of detail. In fact you threw some of it away!

BTW, downsampling a 1 bit image is not an artificial example. Early monochrome printing relies on this technique. If you stand up close you see a noisy mess of dots. If you stand further away (making the image smaller - ie downsampling), you start to perceive the image as have graduated tones rather than just patches of plain white or plain black.

Everyone knows this, but it's false analogy.

[dtaylor]

Their dynamic range is their dynamic range. Period. You can reduce it. You can shift tones around, lifting shadows and such. But you can't extend it.

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

Camera 1 has 36 megapixels and 11 stops of dynamic range per pixel, camera 2 has 20megapixels and 12 stops of dynamic range per pixel.

Which camera has more dynamic range if I display or print their images at the same size ? If I downsample the 36mpx image to 20mpx, I will get more than 11 stops of dynamic range, but do i get more than 12 ?

I think this is a pretty valid question, provided you intend to view the whole image on print or screen, as opposed to just viewing 100% crops.

[dtaylor]

This is taken 15 min ago and shows the difference in shadows, low levels 5dmk2 and d800. I  have lifted the motive exactly the same.

RAW files please.

And regarding  to ISO and   "holding iso" Canon 1dx is exposed richer in tests like  Imaging Resource, this means that 6400iso is more like if 1dx has been exposed after 4000Iso compared to Nikon D4 and therefore it is not a valid test

Stop. You have already been told this is not the test scene they use to compute DR, noise, etc. Don't make this mistake again!

[dtaylor]

I find it interesting that many would like to make it sound like an "underexposure" is the end of the world, and a photographical failure.

For the record: I believe that a "correct" exposure is the exposure that maximizes the qualities you desire for the scene. If you wish to capture the maximum possible dynamic range, your exposure may be over/under to take advantage of your particular sensor/film.

Now try to understand the implications the sentence in bold above sets. Getting an exposure "right" is NOT the same as trying to make 14-16% gray as you see the image conform to a certain photometric exposure at the film surface!

Agreed 100%.

Film has more latitude, but much less usable DR within a fixed scene

DR and latitude are simply two ways of looking at the same thing. They are not separate characteristics.

Latitude, or room for exposure "error", is basically sensor or film DR - scene DR.

The rest of your observations are spot on.

[jrista]

It seems there are different interpretations of what dynamic range means, and I've been in line with dtaylor so far in that when I hear "dynamic range" I think of "photographic dynamic range", or this:

But the black point is not actually the floor of dynamic range. The floor is the point where usable detail ceases to exist.

Usable detail ceases to exist almost as soon as you hit the noise floor on a Sony Exmor sensor, since it does not use a bias offset, and because there is so little noise to start with. I would dispute the notion that usable detail ceases to exist the moment you hit the "noise floor" in a Canon sensor. In a Canon sensor, you have to content with a variety of forms of noise, some of which (banding, for example) which might protrude through usable image detail in some areas while leaving other areas entirely untouched, thanks to the fact that Canon sensors DO use a bias offset, and usable detail can extend into negative value territory. Makes it difficult to compare two sensors of fundamentally different design in the same way. (Although it seems obvious that, despite the fact that usable detail exists below the highest amplitude of noise, the use of a bias offset seems to be a very strong detractor of Canon's sensor design.)

To be fair, if DXO defines their DR measurement simply as the ratio between white and black points, and according to their definition downscaling can improve DR beyond the hardware limits, then so be it. That's how DXO defines DR in the context of their tests, so that's what it is when discussing DR in the context of DXO.

Conversely, I am not sure DXO's definition of DR is a particularly realistic one. It may be mathematically sound, but it doesn't address electronic noise (vs. gaussian noise) very well, particularly when useful detail can extend well beyond the upper amplitude of the electronic noise floor (which would indicate that if you could reverse the effect of the stronger forms of noise...FPN/HVBN...you could recover a lot of useful detail and TRULY extend your dynamic range.) I believe this could actually be demonstrated with some of the sample photos posted by sarangiman a while back, either in this thread or another. Shadows in those photos were lifted so many stops that bright red HVBN started to exhibit...however it only exhibited in certain rows and columns of pixels. Between the bands, useful detail extended right down to nearly black. It shouldn't be too difficult to figure out the ratio of the brightest pixels in the banding noise to the darkest pixels of useful image detail, and figure out how many stops of photographic DR might be recoverable if one could remove the banding noise.