DxOMark Sensor Performance: Nikon vs. Canon

[LetTheRightLensIn]

jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. ....

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.....

.... it's been my entire point all along, as my complaint is with the fact that DXO sells "Print DR" as something it is not, by its very name, even. ...

You have a strange way of putting people down just before proceeding to agree with them ... but without ever admitting you agree with them.

Nah I messed up the message and forgot to chop off the last parts and it made it look like his words were my words, my bad, totally messed the post up. I fixed it now.

 

[LetTheRightLensIn]

And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

You seem to be slowly shifting your position, because you definitely didn't say that you thought it was valid or the best way to go about when comparing between different sensors before.

 

[AvTvM]

aside of the debate re. comparing pcitures downsized to 8MP and all aspects connected to this ... one more thign that really irks me about DXO

they do NOT state anywhere on their webseite, what lens/es they use in their "sensor" tests, which in reality are not "sensor tests" but rather comparisons of downsized RAW images obtained by taking test pictures using a lens, a camera with sensor, hardware and firmware .. so comparisons of "lens-sensor-hardware-firmware"-combinations. They also state this here: http://www.dxomark.com/index.php/Publications/DxOMark-Insights/DxOMark-Score

I doubt they are targeting a laser beam at a naked sensor and use their own electronics - hardware and standardized firmware - to measure "pure sensor performance".

There must be a reason, why they are so clandestine about wht lesnes they use for their testeing. Until they fuly dislose all relevant information regarding their test-setup, i completely disregard their results. For all we know, they might be testing Canon cameras with the 50/1.8 and other cameras with a Zeiss 100 Macro.

 

[Razor2012]

Hard facts dynamic range

Do you have a link to those figures?

 

[jrista]

And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

You seem to be slowly shifting your position, because you definitely didn't say that you thought it was valid or the best way to go about when comparing between different sensors before.

"It", being normaizing scale between images, to compare "the amount of noise" in a sensor, is something I do not have a problem with. I don't normally equate the amount of noise in an image with dynamic range, not the kind of dynamic range I think about when I hear the term. Based on elflord's comments so far, it seems based on DXO's definition of Print DR that normalizing images for comparison is just that, but nothing really any more than that. I do have a problem with referring to the difference between two normalized images a change in dynamic range, and I've already explained in extensive detail why. I won't be explaining it again.

 

[MarkII]

I doubt they are targeting a laser beam at a naked sensor and use their own electronics - hardware and standardized firmware - to measure "pure sensor performance".

Actually, it is quite possible that they perform the public sensor tests without any lens mounted at all. All you need is to be able to do is to illuminate the sensor in a consistent and calibrated manner, and the less that you put in the way the better.

 

[rpt]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

 

[Razor2012]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

 

[tron]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

In his pic Mikael lifted the shadows at a specific place ALOT! You will have to present a similar case to prove your point and not compare apples to oranges.

 

[rpt]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

 

[rpt]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

In his pic Mikael lifted the shadows at a specific place ALOT! You will have to present a similar case to prove your point and not compare apples to oranges.

Sure, I will post my images. I took the pictures as described and looked at the RAW images in 100% view and it looked like Mikael's images were taken by my old Olympus P&S from 1999...

 

[LetTheRightLensIn]

And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

You seem to be slowly shifting your position, because you definitely didn't say that you thought it was valid or the best way to go about when comparing between different sensors before.

"It", being normaizing scale between images, to compare "the amount of noise" in a sensor, is something I do not have a problem with. I don't normally equate the amount of noise in an image with dynamic range, not the kind of dynamic range I think about when I hear the term. Based on elflord's comments so far, it seems based on DXO's definition of Print DR that normalizing images for comparison is just that, but nothing really any more than that. I do have a problem with referring to the difference between two normalized images a change in dynamic range, and I've already explained in extensive detail why. I won't be explaining it again.

1. You just used the term PrintDR above.
2. if you are now going back on using to compare between different sensors then that is a shame
3. numerous people have pointed out to you a thousand times that are off-base on this (and recall back this past spring how you spent 15 pages trashing me to bits when I stated that the 5D3 didn't improve ISO100 DR one bit over the 5D2 and how my numbers were absurd and my methodology ridiculous and that there was no way DxO would ever report the number I got. And then.... weeks later DxO released there number and it was within one tenth of a stop of the number I reported!)

 

[LetTheRightLensIn]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

why did you use iso 12k and 25k when he is talking 100? 5d3 does as well for DR at 25k as exmor

 

[LetTheRightLensIn]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

super high iso has the signal boosted past the noisy stage of the canon sensor read electronics, your tests have nothing to do with the discussion

 

[rpt]

OK, don't mean to offend anybody but if a part time enthusiast's camera can shoot this I am sure most of you can do better...

Here are the screen shots of 100% view of my raw files.

The 25k6 iso image...

The 12k8 iso image...

 

[rpt]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

super high iso has the signal boosted past the noisy stage of the canon sensor read electronics, your tests have nothing to do with the discussion

So there is a way to boost signal and not noise? Wow! You got me! Sorry, I am confused. Obviously I am not as knowledgeable as you chaps. I thought amps amped irrespective of signal or noise. Never mind... Lets move on.

What ISO should I be testing at? Where will I be doing the most damage to the signal while shooting in a dark room? And dont tell me to set EC to +3 or -3 unless the exposure requires it because I wont...

BTW, the EC in the images above is +2...

 

[LetTheRightLensIn]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

super high iso has the signal boosted past the noisy stage of the canon sensor read electronics, your tests have nothing to do with the discussion

So there is a way to boost signal and not noise? Wow! You got me! Sorry, I am confused. Obviously I am not as knowledgeable as you chaps. I thought amps amped irrespective of signal or noise. Never mind... Lets move on.

What ISO should I be testing at? Where will I be doing the most damage to the signal while shooting in a dark room? And dont tell me to set EC to +3 or -3 unless the exposure requires it because I wont...

BTW, the EC in the images above is +2...

There are two stages in the sensor read out, one stage is quite noisy for Canon, this stage dumps a lot of read noise on lower ISO shadows, when you shoot high iso, the first stage has already boosted the shadows above the level where much of this read noise gets dumped so it doesn't hurt things much thus Canon 5D3 can actually do as well or even a trace better at the super crazy high ISO you are testing in shadows and yet do much worse at super low ISO in the shadows than Exmor Nikon cameras.

You are posting things here that have nothing to do with what is under discussion. Nobody has ever said the Canon sensors have bad DR at very high ISO. The 5D3 does at least as well as the D600/D800 there and once you get to like 25k maybe even a tiny bit better and the 1DX seems like it probably can match the D4 and D3s for the cleanest super high iso shadows yet. So yeah you get comparably good results at those super high iso, but nobody ever said you wouldn't. Canon is doing fine up there. The D4/D3s/1DX (and the D3s is so much lower res that the larger scale noise makes it not as good as the other two IMO) probably have the best super high ISO DR ever, by perhaps a stop over the next best, which is probably the 5D3, which has a small advantage over the next best ones whch might be D800 and such. So Canon is right there or maybe even with the lead for best high ISO DR at each price point. But this thread is about low ISO DR.

 

[rpt]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

super high iso has the signal boosted past the noisy stage of the canon sensor read electronics, your tests have nothing to do with the discussion

So there is a way to boost signal and not noise? Wow! You got me! Sorry, I am confused. Obviously I am not as knowledgeable as you chaps. I thought amps amped irrespective of signal or noise. Never mind... Lets move on.

What ISO should I be testing at? Where will I be doing the most damage to the signal while shooting in a dark room? And dont tell me to set EC to +3 or -3 unless the exposure requires it because I wont...

BTW, the EC in the images above is +2...

There are two stages in the sensor read out, one stage is quite noisy for Canon, this stage dumps a lot of read noise on lower ISO shadows, when you shoot high iso, the first stage has already boosted the shadows above the level where much of this read noise gets dumped so it doesn't hurt things much thus Canon 5D3 can actually do as well or even a trace better at the super crazy high ISO you are testing in shadows and yet do much worse at super low ISO in the shadows than Exmor Nikon cameras.

You are posting things here that have nothing to do with what is under discussion. Nobody has ever said the Canon sensors have bad DR at very high ISO. The 5D3 does at least as well as the D600/D800 there and once you get to like 25k maybe even a tiny bit better and the 1DX seems like it probably can match the D4 and D3s for the cleanest super high iso shadows yet. So yeah you get comparably good results at those super high iso, but nobody ever said you wouldn't. Canon is doing fine up there. The D4/D3s/1DX (and the D3s is so much lower res that the larger scale noise makes it not as good as the other two IMO) probably have the best super high ISO DR ever, by perhaps a stop over the next best, which is probably the 5D3, which has a small advantage over the next best ones whch might be D800 and such. So Canon is right there or maybe even with the lead for best high ISO DR at each price point. But this thread is about low ISO DR.

So what you are saying is I should shoot the same scenario at ISO 100?

 

[Loswr]

But this thread is about low ISO DR.

Is it? Why? Is it because everyone shoots at ISO 100 all the time? Or because DxO bases their Score on ISO 100? :

 

[Razor2012]

OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?

Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

Lol, rpt the reference wasn't for you.