DxOMark Sensor Performance: Nikon vs. Canon

[MarkII]

...As to the continued assertion that downsampling can not increase bit depth, ....

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

No - the problem is people who assert that the DXO measurements are "obviously nonsense" because they (wrongly) believe that it is impossible for a down-sampled image to show more per-pixel DR than the original.

DXO use resolution normalised figures in their summaries partly because this is how most people tend to view their images (even the new 15" 'retina' MacBookPro screens are "only" 5MP at full-screen). And if they did not do this, some people would be screaming that they were biased because their numbers favour lower resolution cameras.

It is perfectly reasonable to describe a sensor as having ~13.2 bits at full resolution and ~14.3 bits at a normalised 8MP resolution. Anyone considering buying or using a device like this should be smart enough to understand what this means and take the time and effort to read beyond the headline summaries.

Do people really buy cameras because of one number in a test score? Do they only read the final summary table in a 20-page review before opening their wallets for thousands of dollars??? If so, they probably deserve what they get!

 

[psolberg]

Wow, site admin flip flops in dxo after saying he would never publish dxo numbers even if canon came ahead. Now it happened and we start to see dxo numbers.....now dxo matters

 

[elflord]

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

Camera 1 has 36 megapixels and 11 stops of dynamic range per pixel, camera 2 has 20megapixels and 12 stops of dynamic range per pixel.

Which camera has more dynamic range if I display or print their images at the same size ? If I downsample the 36mpx image to 20mpx, I will get more than 11 stops of dynamic range, but do i get more than 12 ?

I think this is a pretty valid question, provided you intend to view the whole image on print or screen, as opposed to just viewing 100% crops.

 

[TheSuede]

I find it interesting that many would like to make it sound like an "underexposure" is the end of the world, and a photographical failure.

What that stance significates is failing to understand the implications. I spent quite a few years of my life fighting against this erroneous mindset when educating print and pre-press personnel starting out in their first stumbling steps of the "digital revolution". Back then, pre-press personnel was (if possible) an even more tradition-laden and anachronistic group of people than what photographers are today.

As someone mentioned Adams, I have to say that he is a person you really have to admire for the amount of energy and time he spent really getting to know his medium (film), and how you had to expose to get the most out of your envisioned image. He (as he worked with the physicists and chemists at Kodak) set up some really workable testing rules and also inspection criteria to be used when looking at image results.

And he always tried to get the exposure "right".

Now try to understand the implications the sentence in bold above sets. Getting an exposure "right" is NOT the same as trying to make 14-16% gray as you see the image conform to a certain photometric exposure at the film surface! And he never claimed that it was. If there is more pictorial detail that you want to emphasize in the end result (for him; film, development, print) in the shadows - you should really overexpose - compared to the baseline scene average exposure. And if there is "important detail" in the highlights, you need to underexpose [compared to the reference exposure]. He accepted that there was always a compromise between highlight, midtone and shadow local detail. I would recommend a visit to his old home, if his son still has the tours he used to. If you're nice, to him you might even get to see some "behind the scenes" notes and development plans - they're really impressive in attention to detail, and shows how much he mauled his negatives, using both over- and under-exposure, to get "what he wanted" out of them. Regional corrections of + or -3Ev was nothing out of the ordinary either.

In digital, the entire ramification of the process has changed.

Film has more latitude, but much less usable DR within a fixed scene
Film has an bell-shaped MTF modulus, digital is linear
Film clips and blocks gracefully towards both highlight and shadow; and
Digital clips hard against highlights and fades gradually into noise in shadows

This means that you can't treat digital like you did with film, and the most obvious differences are:
a) as long as the entire scene can be contained within the digital DR, you can put the raw-file related exposure (which is not the same as photometric exposure!) wherever you want - it does not effect internal detail contrast.
b) you have to mind the highlights (more) with digital - since when they're clipped, all internal detail contrast is GONE. Impossible to retrieve unless you let software guess what the clipped channel values should be from surrounding color information. This only works for about +0.5Ev from initial clipping.

To underexpose the raw file - again, this is not the same as the photometric exposure! - is the only practically available method to reign in the highlights in the scene. And this is also what Canon does with the "Highlight Tone Priority" or "HTP" option when you shoot jpg. The camera will underexpose the raw file, and then bring image midtone brightness up again - but with smother upper S-curve end of the brightness curve.

Underexposing an image that doesn't have any highlight areas (zones? maybe applicable) you want to protect though - that's only stupid. It hurts image quality.

 

[LetTheRightLensIn]

...As to the continued assertion that downsampling can not increase bit depth, ....

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

It is described as that but when you want to do a relative comparison you can't compare cameras at different scales otherwise you are comparing noise power at two different frequencies as if they were the same.

 

[TheSuede]

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

Camera 1 has 36 megapixels and 11 stops of dynamic range per pixel, camera 2 has 20megapixels and 12 stops of dynamic range per pixel.

Which camera has more dynamic range if I display or print their images at the same size ? If I downsample the 36mpx image to 20mpx, I will get more than 11 stops of dynamic range, but do i get more than 12 ?

I think this is a pretty valid question, provided you intend to view the whole image on print or screen, as opposed to just viewing 100% crops.

This is a complex question, since what DxO basically fails to include in their DR vs resolution compensation (the "print" view option) is that no Bayer-based image ever contains equal noise energy all the way up to 1/f. The interpolation stage often called the "de-mosaic" stage necessary (remember, two out of the three channels in each pixel has had to be estimated after the capture...) is effectively filtering the higher noise frequencies out, and tends towards zero at 1/f.

In a layman's terms, you could describe this high-frequency filter as:
-"The noise - or average pixel difference - is stronger in power when you compare two pixels a few pixels apart from each other than if you compare two pixels next to each other."

The end result of this is that the first ~30% of downsampling - down to 70% original scale that is - does not lower image noise power [over the image width] by any significant degree. There wasn't much noise energy in the frequency band that we've filtered away, so what we've basically done is to condense the image information.

But then there's also a more subtle effect. The human eye does not react as strongly to fine-grained noise as it does to coarse-grained noise. This does mean that when you MEASURE the average pixel noise power, it might not have been lowered by any appreciable amount - but when you LOOK at the downsampled image you will perceive the image as less noisy anyway.

So - downsampling 36MP to 20MP would theoretically give you an added:
sqrt(36/20) = 1.34 linear scale
>> convert to log2 >>
log(1.34)/log(2) = 0.42Ev or "bits" of DR

But you wouldn't get 0.42Ev in a real, converted image - you would get maybe 0.1Ev and a much tighter (less objectionable) noise pattern.

From this point on downwards however the noise spectrum could be said to be sufficiently close to a linear 1/f behavior, and you would get the full noise-power lowering effect in practice too. So, continuing down in size would yield the full theoretical gains of log(sqrt(linear scale))/log(2). Together with a lowered resolution per image frame of course...

 

[dtaylor]

Downsampling doesn't "create detail", but it moves the black point.

* Photographic dynamic range is the range of useful detail.

* As you conceded above, down sampling does not create detail. In fact, it throws detail away.

* Down sampling therefore cannot increase photographic dynamic range.

* DxO's normalized results are therefore false.

Can we move on now?

 

[dtaylor]

Please feel free to post any unanswered "critiques"

All of them from what I can see.

Far from rebutting, it's not clear to me that you or jrista understand the implications of normalization, or the basic math behind benchmarking, so aren't in much of a position to criticize it.

http://www.nizkor.org/features/fallacies/

 

[dtaylor]

Where is your evidence or explanation?

For real, usable DR tests, look to IR and DPReview. (Unfortunately DPReview no longer posts RAW DR, but their tests are still valid for JPEG.)

The explanation has been posted repeatedly, and was posted again just a second ago.

I tested the DR myself and got same results and so have others.

You're making the same mistake as the others. Noise inflates the black point. Down sampling reduces noise, lowering the black point. But the black point is not actually the floor of dynamic range. The floor is the point where usable detail ceases to exist. Down sampling actually throws away detail. It cannot extend DR.

I've seen improvements over that time period. DPReview saw improvements.

On what? 5D3 has actually worse DR than the 1Ds3.

In JPEG the 5D3 is better, as noted by DPReview. Unfortunately they did not post RAW values for the 5D3.

Imaging Resource has Imatest RAW values for both, and the 5D3 is one stop better.

real world, accounting for banding, I'd say D800 does more than 3 better than 5D3 and even ignoring banding it's more than 2.

"Real world"...that would be a switch. Please post real world shots that prove this. Include a transmission step wedge. Oh yes: downloadable RAW files for analysis are mandatory.

And not that I want to go here, but you've finally gotten me partially to this point, and sure even the most brilliant are wrong at times, absolutely true, and I'm not saying who here and on the other forums is who, but there are not only engineers but theoretical physics PhDs and other such, including some who are world renowned, posting in some of these threads and on some other threads and.... not on your side on this.

What is this: an appeal to imaginary authorities? :

http://www.nizkor.org/features/fallacies/

Whether or not the 5D4 has better DR has nothing to do with these stupid threads, and everything to do with their engineers. I have little doubt they are working on it.

Did you know that another division of Canon sent a patent for better DR to the DSLR division and got told to get lost, DR, what?, why? bye. Apparently they didn't even let their engineers look at it! So maybe they do need to be woken up.

Source?

A Canon employee (non-DSLR division) on another forum.

More imaginary friends = no source. Don't repeat the claim without a source.

 

[dtaylor]

Well, if I have a noisy image (ie with poor DR), it will be usable only at smaller print sizes than a less noisy one. That is a real world example of down-sampling, and if what you claim were true the small print would show exactly the same noise and DR than the larger one.

It does show exactly the same DR. You're making a subjective, aesthetic judgement as to acceptable print/view size due to noise. (And it's not down sampling unless you have a really crummy printer.)

As to the continued assertion that downsampling can not increase bit depth,

No. That's not the argument. (This is like pulling teeth!)

Pay careful attention: down sampling cannot increase *** photographic dynamic range ***

In photography dynamic range is the range of usable detail, not the range between white and black points.

You have a RAW file. It has some noise. Black is not black, but a really dark gray because of the noise. You down sample and kill some of the noise. "Aha! My blacks are blacker! My bit depth is greater!" So what? You didn't gain one drop of detail. In fact you threw some of it away!

BTW, downsampling a 1 bit image is not an artificial example. Early monochrome printing relies on this technique. If you stand up close you see a noisy mess of dots. If you stand further away (making the image smaller - ie downsampling), you start to perceive the image as have graduated tones rather than just patches of plain white or plain black.

Everyone knows this, but it's false analogy.

 

[dtaylor]

Their dynamic range is their dynamic range. Period. You can reduce it. You can shift tones around, lifting shadows and such. But you can't extend it.

Is that the assertion? Or is the assertion that a 36 MP sensor with 13.2 bits of DR at 36 MP should be described as having 36 MP of resolution and 13.2 bits of DR?

Camera 1 has 36 megapixels and 11 stops of dynamic range per pixel, camera 2 has 20megapixels and 12 stops of dynamic range per pixel.

Which camera has more dynamic range if I display or print their images at the same size ? If I downsample the 36mpx image to 20mpx, I will get more than 11 stops of dynamic range, but do i get more than 12 ?

I think this is a pretty valid question, provided you intend to view the whole image on print or screen, as opposed to just viewing 100% crops.

 

[dtaylor]

This is taken 15 min ago and shows the difference in shadows, low levels 5dmk2 and d800. I have lifted the motive exactly the same.

RAW files please.

And regarding to ISO and "holding iso" Canon 1dx is exposed richer in tests like Imaging Resource, this means that 6400iso is more like if 1dx has been exposed after 4000Iso compared to Nikon D4 and therefore it is not a valid test

Stop. You have already been told this is not the test scene they use to compute DR, noise, etc. Don't make this mistake again!

 

[dtaylor]

I find it interesting that many would like to make it sound like an "underexposure" is the end of the world, and a photographical failure.

For the record: I believe that a "correct" exposure is the exposure that maximizes the qualities you desire for the scene. If you wish to capture the maximum possible dynamic range, your exposure may be over/under to take advantage of your particular sensor/film.

Now try to understand the implications the sentence in bold above sets. Getting an exposure "right" is NOT the same as trying to make 14-16% gray as you see the image conform to a certain photometric exposure at the film surface!

Agreed 100%.

Film has more latitude, but much less usable DR within a fixed scene

DR and latitude are simply two ways of looking at the same thing. They are not separate characteristics.

Latitude, or room for exposure "error", is basically sensor or film DR - scene DR.

The rest of your observations are spot on.

 

[jrista]

It seems there are different interpretations of what dynamic range means, and I've been in line with dtaylor so far in that when I hear "dynamic range" I think of "photographic dynamic range", or this:

But the black point is not actually the floor of dynamic range. The floor is the point where usable detail ceases to exist.

Usable detail ceases to exist almost as soon as you hit the noise floor on a Sony Exmor sensor, since it does not use a bias offset, and because there is so little noise to start with. I would dispute the notion that usable detail ceases to exist the moment you hit the "noise floor" in a Canon sensor. In a Canon sensor, you have to content with a variety of forms of noise, some of which (banding, for example) which might protrude through usable image detail in some areas while leaving other areas entirely untouched, thanks to the fact that Canon sensors DO use a bias offset, and usable detail can extend into negative value territory. Makes it difficult to compare two sensors of fundamentally different design in the same way. (Although it seems obvious that, despite the fact that usable detail exists below the highest amplitude of noise, the use of a bias offset seems to be a very strong detractor of Canon's sensor design.)

To be fair, if DXO defines their DR measurement simply as the ratio between white and black points, and according to their definition downscaling can improve DR beyond the hardware limits, then so be it. That's how DXO defines DR in the context of their tests, so that's what it is when discussing DR in the context of DXO.

Conversely, I am not sure DXO's definition of DR is a particularly realistic one. It may be mathematically sound, but it doesn't address electronic noise (vs. gaussian noise) very well, particularly when useful detail can extend well beyond the upper amplitude of the electronic noise floor (which would indicate that if you could reverse the effect of the stronger forms of noise...FPN/HVBN...you could recover a lot of useful detail and TRULY extend your dynamic range.) I believe this could actually be demonstrated with some of the sample photos posted by sarangiman a while back, either in this thread or another. Shadows in those photos were lifted so many stops that bright red HVBN started to exhibit...however it only exhibited in certain rows and columns of pixels. Between the bands, useful detail extended right down to nearly black. It shouldn't be too difficult to figure out the ratio of the brightest pixels in the banding noise to the darkest pixels of useful image detail, and figure out how many stops of photographic DR might be recoverable if one could remove the banding noise.

 

[ishdakuteb]

The read noise is much higher, the QE is much lower in a MF ccd , (so even if the surface is bigger in MF than a 24x36mm Sony sensor ) the score is what it is because of FWC , read noise.The same with d7000 and 5dmk3 there d7000 is a APS and has higher DR than 5dmk3 because of much lower read out noise at base iso
In other words, nothing strange

i am not sure what you are trying to do, learn or even get what kinda information, but within one year, from 2011-2012, you have gone to number of sites to write your name on them, starting with nikon d7000 vs. canon 5d mark ii (how do i know this? it is pretty easy to me). however, if nikon is that good enough to you, then buy more nikon, sell all your canon gears and go to nikon site to learn more about that camera as well as technical things right? it makes more sense that way, at least to me. i learn from both sites since i do not favor one and hate the other. and i have learned quitely instead of making noise.

in conclusion, if dr. roger clark is loving nikon that much, why he is using canon? this is what i think, have not verified the fact, but he have, at least and recently, used canon....

in short, get a life mr. mikael risedal and your images cannot draw my attention even though i am a newbie to dslr, but drooling to have images like joe buissink even though he is mr. p mode and probably know less about camera technical things....

 

[CanonGirl]

I must say that all of this “my camera is better than your camera” talk gives me a headache, and I’m not sure I want to get in the middle of this, um, whatever contest. I have read in detail the DxO results, along with many other test results and feel this debate is “full of sound and fury signifying nothing!” Bottom line to me is whether or not the camera in question is the best match for my needs. If you’re priorities are different, you should select a camera accordingly. DxO has performed a series of tests showing the D800 to have a wider dynamic range than other cameras, provided that you are at iso 800 or below. Also their results show the 5D Mark III to have maybe 0.5 stops more headroom on the highlights than the D800, but the D800 has well over a stop more headroom in the shadows. I love the slightly greater headroom in the highlights, as I shoot white wedding dresses and want to ensure I retain detail at that end. I also see in their charts that the S/N of the 5D Mark III is better than the D800, which is important for low-light photography to me, but clearly the D800 has more resolution. C’t Digital Photography (Fall Issue, 9, 2012) just did a very quantitative review of the D800, Canon 5D Mark III, Olympus OM-D, Nikon D4, and Pentax K-01, and their results are similar to DxO, but add other interesting comparisons. The D800 wins for resolution & dynamic range at lower iso, but the Canon 5D Mark III has superior S/N (low noise). I shoot weddings, which often involve low-light shooting, so for my needs, the 5D is superior. Would I like greater dynamic range? Sure, but not at the expense of S/N. DxO results, in their raw form, just provide another piece of useful information to help camera buyers evaluate equipment against their needs. Looking at their scoring system, it’s clear to me that they consider resolution and dynamic range to be most important, and therefore, rate these two parameters much higher than others to achieve their ratings. Yes, I find it amusing that they quote dynamic range in their final model for the D800 (14.4 Evs) wider than the 14-bit converter, which tells me that their models are not quite 100% accurate. I also find this true in that they rate the Canon 5D Mark III and the Canon 7D as both 11.7 Evs, whereas, when I shoot a Sekonic target using both cameras & the same lens/lighting conditions, camera settings, I get about 1.5 stops more dynamic range with the 5D over my 7D, gaining both in the highlights and shadows over the 7D, using Sekonic’s light meter calibration software. So I see DxO testing results to be useful, but not the absolute determinate of overall camera performance.

In the end, I don’t see either camera as absolutely the best in all ways, as they each have their strengths and weaknesses. For me, the Canon 5D Mark III is a wonderful tool, meeting my present needs beautify, and I wouldn’t trade it for the D800 at this time, as the pros do not outweigh the cons for me. As for others, their needs, interests, concerns might be completely different—LOL, some are obviously drastically different, reading the comments on this blog, but that’s perfectly ok with me, and I hope you’ll share some of your beautiful work, but I will smile if I find the photo acclaimed to be the best was taken with a cell phone! ;-)

 

[Razor2012]

I must say that all of this “my camera is better than your camera” talk gives me a headache, and I’m not sure I want to get in the middle of this, um, whatever contest. I have read in detail the DxO results, along with many other test results and feel this debate is “full of sound and fury signifying nothing!” Bottom line to me is whether or not the camera in question is the best match for my needs. If you’re priorities are different, you should select a camera accordingly. DxO has performed a series of tests showing the D800 to have a wider dynamic range than other cameras, provided that you are at iso 800 or below. Also their results show the 5D Mark III to have maybe 0.5 stops more headroom on the highlights than the D800, but the D800 has well over a stop more headroom in the shadows. I love the slightly greater headroom in the highlights, as I shoot white wedding dresses and want to ensure I retain detail at that end. I also see in their charts that the S/N of the 5D Mark III is better than the D800, which is important for low-light photography to me, but clearly the D800 has more resolution. C’t Digital Photography (Fall Issue, 9, 2012) just did a very quantitative review of the D800, Canon 5D Mark III, Olympus OM-D, Nikon D4, and Pentax K-01, and their results are similar to DxO, but add other interesting comparisons. The D800 wins for resolution & dynamic range at lower iso, but the Canon 5D Mark III has superior S/N (low noise). I shoot weddings, which often involve low-light shooting, so for my needs, the 5D is superior. Would I like greater dynamic range? Sure, but not at the expense of S/N. DxO results, in their raw form, just provide another piece of useful information to help camera buyers evaluate equipment against their needs. Looking at their scoring system, it’s clear to me that they consider resolution and dynamic range to be most important, and therefore, rate these two parameters much higher than others to achieve their ratings. Yes, I find it amusing that they quote dynamic range in their final model for the D800 (14.4 Evs) wider than the 14-bit converter, which tells me that their models are not quite 100% accurate. I also find this true in that they rate the Canon 5D Mark III and the Canon 7D as both 11.7 Evs, whereas, when I shoot a Sekonic target using both cameras & the same lens/lighting conditions, camera settings, I get about 1.5 stops more dynamic range with the 5D over my 7D, gaining both in the highlights and shadows over the 7D, using Sekonic’s light meter calibration software. So I see DxO testing results to be useful, but not the absolute determinate of overall camera performance.

In the end, I don’t see either camera as absolutely the best in all ways, as they each have their strengths and weaknesses. For me, the Canon 5D Mark III is a wonderful tool, meeting my present needs beautify, and I wouldn’t trade it for the D800 at this time, as the pros do not outweigh the cons for me. As for others, their needs, interests, concerns might be completely different—LOL, some are obviously drastically different, reading the comments on this blog, but that’s perfectly ok with me, and I hope you’ll share some of your beautiful work, but I will smile if I find the photo acclaimed to be the best was taken with a cell phone! ;-)

As much as people have tried to convey this reasoning, it falls to deaf ears. We have to concede that the only real way to take any kind of good pictures is to have DR. :

 

[CanonGirl]

Razor2012, I'm sure slide film users would be delighted to hear that! :

 

[LetTheRightLensIn]

Where is your evidence or explanation?

For real, usable DR tests, look to IR and DPReview. (Unfortunately DPReview no longer posts RAW DR, but their tests are still valid for JPEG.)

The explanation has been posted repeatedly, and was posted again just a second ago.

I tested the DR myself and got same results and so have others.

You're making the same mistake as the others. Noise inflates the black point. Down sampling reduces noise, lowering the black point. But the black point is not actually the floor of dynamic range. The floor is the point where usable detail ceases to exist. Down sampling actually throws away detail. It cannot extend DR.

I've seen improvements over that time period. DPReview saw improvements.

On what? 5D3 has actually worse DR than the 1Ds3.

In JPEG the 5D3 is better, as noted by DPReview. Unfortunately they did not post RAW values for the 5D3.

Imaging Resource has Imatest RAW values for both, and the 5D3 is one stop better.

real world, accounting for banding, I'd say D800 does more than 3 better than 5D3 and even ignoring banding it's more than 2.

"Real world"...that would be a switch. Please post real world shots that prove this. Include a transmission step wedge. Oh yes: downloadable RAW files for analysis are mandatory.

And not that I want to go here, but you've finally gotten me partially to this point, and sure even the most brilliant are wrong at times, absolutely true, and I'm not saying who here and on the other forums is who, but there are not only engineers but theoretical physics PhDs and other such, including some who are world renowned, posting in some of these threads and on some other threads and.... not on your side on this.

What is this: an appeal to imaginary authorities? :

http://www.nizkor.org/features/fallacies/

Whether or not the 5D4 has better DR has nothing to do with these stupid threads, and everything to do with their engineers. I have little doubt they are working on it.

Did you know that another division of Canon sent a patent for better DR to the DSLR division and got told to get lost, DR, what?, why? bye. Apparently they didn't even let their engineers look at it! So maybe they do need to be woken up.

Source?

A Canon employee (non-DSLR division) on another forum.

More imaginary friends = no source. Don't repeat the claim without a source.

1. It's pointless since you don't get it.
2. Suede added some nice finer details above due to the Bayer read-out already partly pre-averaged thing, which is a good point that almost everyone else has left out, so if you want to quibble with DxO quibble a tiny bit over that (but as he notes, real world, to the eye, it doesn't matter so much, since the changed character looks better over that regime anyway)
3. some of the posters in this thread alone, are engineers and/or have PhD in the hard sciences, myself included and the others have websites up that have been quoted in plenty of threads such as this one and have posted on these topics in other threads on other forums
4. I private msged you

 

[tnargs]

Just want to post my thanks for posts #360-364, the additional perspective is useful to me.