Canon EOS-1D X DXOMark Sensor Scores

[heptagon]

Well they do matter for a few specific use cases. E.g. if you can't get the lighting right and need to brighten your pictures.

 

[neuroanatomist]

Well they do matter for a few specific use cases. E.g. if you can't get the lighting right and need to brighten your pictures.

I'll buy that. But the examples provided recently in other threads don't show 'can't get the lighting right', they show intentionally choosing to get the lighting wrong, then brightening the image that should not have needed brightening if properly exposed to begin with.

 

[Viggo]

Well they do matter for a few specific use cases. E.g. if you can't get the lighting right and need to brighten your pictures.

I'll buy that. But the examples provided recently in other threads don't show 'can't get the lighting right', they show intentionally choosing to get the lighting wrong, then brightening the image that should not have needed brightening if properly exposed to begin with.

+1 I expose 5/8 to the right by moving my 0 ev, and this leads to pulling shadows DOWN or even the whole exposure, if need. I haven't pulled UP a single image with the 1d x, the metering is fantastic. Leaving me with perfect exposure and no need to do it wrong and try to rescue it later. Learn how to expose instead....

 

[elflord]

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

That's not what he said at all.

He said that Canon didn't design their camera with DXO's benchmarks in mind. Instead, the manufacturer (Canon) have their own design goals, performance goals, use cases, etc in mind. They also almost certainly do some kind of internal benchmarking.

That doesn't mean that DxOMark's performance measurement is uncorrelated with real world performance (or for that matter, benchmarks that Canon might use internally to test their products).

It also most certainly doesn't mean that DxOMark's performance measures are useless.

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

 

[Woody]

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

Ultimately, real world results are what counts, yes? Photographers buy cameras to capture images, not to pass specific testing processes or specific testing regimes, ya?

So, can you point out specific real world scenarios that Canon's cameras perform really well, outside of what DXOMark can ever reveal? Perhaps Canon's internal test regimes are tailored for such instances?

 

[elflord]

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

Ultimately, real world results are what counts, yes? Photographers buy cameras to capture images, not to pass specific testing processes or specific testing regimes, ya?

So, can you point out specific real world scenarios that Canon's cameras perform really well, outside of what DXOMark can ever reveal? Perhaps Canon's internal test regimes are tailored for such instances?

To address your first question(s) --

I'm not sure what you think you're disagreeing with. Better equipment helps us get "real world results", and the sensor is a key part (but not all of) our camera equipment. Photographers have gotten great results before the latest sensors were available, indeed, before digital was available.

If you're happy to use the same equipment that produced great results for photographers who didn't have access to the latest gear, then you have absolutely no need to pursue the latest technology.

However, if you are in the market for the best technology the market has to offer, benchmark results count.

You can wait until the technology in question has an established track record, to see which products get the best "real world results" but there are least two problems with this -- one is that by the time this established track record is realized, it is not the latest tech any more.

The other is that "real world results" depend more on the skill of the photographer than the technology. You can buy the same equipment used by the photographer produces the greatest "real world results", but the photographers talent does not come in the box with the equipment.

In answer to your second question, DxOMark only test sensors. The camera is much more than a sensor. Canon almost certainly include considerable testing of features including but not limited to ergonomics, durability (weather proofing and longevity). and autofocus performance. And there are yet other factors that can't really be tested, but are important all the same -- Canon's unique ability and willingness to stand behind their products. This is a combination of a first rate support system, and complete control over their product line. For example, there is no other full format DSLR manufacturer that completely manage their product development (cameras, sensors, lenses). Nikon's performance numbers are impressive but one could reasonably doubt Sony's commitment to Nikon's DSLR line. Nobody doubts Canon's commitment to Canon. Canon's strategic decision to avoid taking the easy way out and just buying the sensor might seem to hurt them in the short run, but it is a strategic decision that takes the long view.

 

[jrista]

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

Ultimately, real world results are what counts, yes? Photographers buy cameras to capture images, not to pass specific testing processes or specific testing regimes, ya?

So, can you point out specific real world scenarios that Canon's cameras perform really well, outside of what DXOMark can ever reveal? Perhaps Canon's internal test regimes are tailored for such instances?

To address your first question(s) --

I'm not sure what you think you're disagreeing with. Better equipment helps us get "real world results", and the sensor is a key part (but not all of) our camera equipment. Photographers have gotten great results before the latest sensors were available, indeed, before digital was available.

If you're happy to use the same equipment that produced great results for photographers who didn't have access to the latest gear, then you have absolutely no need to pursue the latest technology.

However, if you are in the market for the best technology the market has to offer, benchmark results count.

You can wait until the technology in question has an established track record, to see which products get the best "real world results" but there are least two problems with this -- one is that by the time this established track record is realized, it is not the latest tech any more.

The other is that "real world results" depend more on the skill of the photographer than the technology. You can buy the same equipment used by the photographer produces the greatest "real world results", but the photographers talent does not come in the box with the equipment.

In answer to your second question, DxOMark only test sensors. The camera is much more than a sensor. Canon almost certainly include considerable testing of features including but not limited to ergonomics, durability (weather proofing and longevity). and autofocus performance. And there are yet other factors that can't really be tested, but are important all the same -- Canon's unique ability and willingness to stand behind their products. This is a combination of a first rate support system, and complete control over their product line. For example, there is no other full format DSLR manufacturer that completely manage their product development (cameras, sensors, lenses). Nikon's performance numbers are impressive but one could reasonably doubt Sony's commitment to Nikon's DSLR line. Nobody doubts Canon's commitment to Canon. Canon's strategic decision to avoid taking the easy way out and just buying the sensor might seem to hurt them in the short run, but it is a strategic decision that takes the long view.

Very well said! +1

 

[Fishnose]

'DxO has no credibility'.... who says? You and a couple of others here who don't like the results they publish, and just don't understand that DxO CANNOT AFFORD TO BULLSHIT...

Anyway, the 1Dx is an absolutely unbelievably good camera. It's not made to get high scores, it's made to get fantastic results in almost impossible situations, especially low light of course.

I hope you see the obvious self-contradiction in what you wrote

Wow, you don't have a clue, do you. I write that the 1Dx is "not made to get high scores, it's made to get fantastic results in almost impossible situations".
In what way does that contradict anything else I wrote?

I'll write it again, to make sure you get it this time: Canon did not make the 1Dx to score one way or the other in any test, Canon made it to be an incredibly good camera at what it does. And Canon succeeded.

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

Wow, you really are confused. :

If the DxO figures rumored are in fact the official DxO results, then the numbers mentioned are exceptionally high and the 1Dx gets fantastic results.
Remember, DxO are testing the SENSOR - not the camera.
The camera obviously is fantastic - and, according to DxO, so is the sensor. The numbers are stellar.

That the D600 and D800 get even higher marks (for their SENSORS) has got nothing to do with anything in this situation - they're a different kind of camera.

The 1Dx is made specifically for low light work and other difficult situations. The D600 and D800 are definitely not made for this.

So - what was your point again? In what way are DxO not credible? Be specific now, explain your point with some kind of useful argument other than 'DxO is a joke' - as infared put it.
Very erudite, to say the least

 

[jrista]

'DxO has no credibility'.... who says? You and a couple of others here who don't like the results they publish, and just don't understand that DxO CANNOT AFFORD TO BULLSHIT...

Anyway, the 1Dx is an absolutely unbelievably good camera. It's not made to get high scores, it's made to get fantastic results in almost impossible situations, especially low light of course.

I hope you see the obvious self-contradiction in what you wrote

Wow, you don't have a clue, do you. I write that the 1Dx is "not made to get high scores, it's made to get fantastic results in almost impossible situations".
In what way does that contradict anything else I wrote?

I'll write it again, to make sure you get it this time: Canon did not make the 1Dx to score one way or the other in any test, Canon made it to be an incredibly good camera at what it does. And Canon succeeded.

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

Wow, you really are confused. :

If the DxO figures rumored are in fact the official DxO results, then the numbers mentioned are exceptionally high and the 1Dx gets fantastic results.
Remember, DxO are testing the SENSOR - not the camera.
The camera obviously is fantastic - and, according to DxO, so is the sensor. The numbers are stellar.

That the D600 and D800 get even higher marks (for their SENSORS) has got nothing to do with anything in this situation - they're a different kind of camera.

The 1Dx is made specifically for low light work and other difficult situations. The D600 and D800 are definitely not made for this.

So - what was your point again? In what way are DxO not credible? Be specific now, explain your point with some kind of useful argument other than 'DxO is a joke' - as infared put it.
Very erudite, to say the least

I think it the conversation has become rather confused. In general, DXO is a good resource when it comes to IQ. Its not that DXO is entirely non-credible. The problem many of us have with DXO is specifically with their "Print DR" statistic. It is not a measurement, it is a rating, and it is a weighted rating at that. Not only that, but their "Print DR" statistic seems to have an overbearing weight on overall sensor score, making it seem like the only thing that matters in a sensor is dynamic range. The fact that Canon sensors are capable of considerably better high ISO performance these days is so weakly rated that it doesn't seem to matter...at least according to DXO.

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

Everyone needs to keep in mind, these are not yet posted as official results. Assuming they ARE official results, something else to keep in mind:

A DR of 12.8 is the Print DR. As everyone here who has read the debates about DXO before, particularly from Neuro and myself, Print DR statistics from DXO are very misleading. These numbers are not "impossible" like the D800's 14.4, however to remain objective and consistent in my argument:

Just as you couldn't actually capture a scene with 14.4 literal stops of DR with a D800 IN-CAMERA, neither will anyone be able to capture a scene with 12.8 literal stops of DR with a 1D X. For the exact same reasons, the PHYSICAL capabilities of the HARDWARE simply won't allow it. The hardware is rated by DXO's actual measurements, which fall under their Screen DR statistics.

My guess is that the 1D X will still have 11.something stops of real-world HARDWARE DR. The 12.8 stops is only something that might potentially be possible with the right kind of scaling algorithm, and all it will do is allow you to utilize a little more headroom that is normally consumed by noise in a 100% image (however at the tradeoff of detail and resolution...potentially a LOT of detail and resolution, since it required downscaling). The 12.8 stop DR rating tells you about what SOFTWARE can do if it normalizes (bins) noise, but it does not tell you anything about the physical capabilities of the hardware. (Although if 12.8 stops really is the Print DR, it sounds like the QUALITY of the 1D X's noise is really quite good.)

Just to put things in perspective, and maintain a level playing field with consistent arguments: Screen DR tells you about the hardware (and we don't know this yet, probably won't until DXO actually posts the 1D X results on their site.) Print DR tells you about how clever DXO's testing software is and how capable it is at normalizing noise when downscaling.

 

[JR]

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

I would tend to agree there. Having both the 1DX and the D4 (and I guess the D800), the only number in those DxO "rumored" scores that surprised me was the 12.8 DR. Overall I do think the 1DX is a superior camera then the D4, however I still find from real life shooting that my D4 gives me more dynamic range when I shoot outside and I dont find the difference marginal either...

One of the biggest difference between both camera also seems to be metering which could explain in part my perceived preference for one camera over the other in certain shooting condition...

 

[peederj]

I've come to the conclusion that the 1DX is worth paying double for vs. the 5D3. With Big Megapixel Talk on the horizon though, and Canon needing an answer to the D800e (which has a damn fine sensor and that's about it), maybe it's better I wait a bit before trading in this camera that I only got this year.

 

[jrista]

I've come to the conclusion that the 1DX is worth paying double for vs. the 5D3. With Big Megapixel Talk on the horizon though, and Canon needing an answer to the D800e (which has a damn fine sensor and that's about it), maybe it's better I wait a bit before trading in this camera that I only got this year.

It would indeed be interesting if Canon provided a 46.1mp FF camera without a low pass filter. I'd be very interested in that for landscape work.

 

[elflord]

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

You keep repeating it, but it's still wrong. It's wrong because the low end of the dynamic range is determined by SNR, and SNR changes when you downsample. The lower end of the dynamic range is NOT some point at which the signal is clipped. It is not in general true that a pixel is completely insensitive outside the dynamic range. The baseline of dynamic range is defined in terms of SNR, not detectability.

You do run into a "quantization limit" at the number of bits in the ADC (assuming that both the sensor response and the ADC itself are linear), but even then, you do eliminate shadow noise and therefore increase your usable dynamic range. But this is not an issue with the new canon bodies because their reported dynamic range does not exceed that of the ADC.

I should add further that the screen "dynamic range" is also in some sense "just a score". A SNR of 0db is extremely noisy (as a point of comparison the powershot S90's noise at max ISO 18% gray is 13db), so depending on your criteria, the "usable dynamic range" might be a few stops less than that indicated by the screen score.

 

[jrista]

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

You keep repeating it, but it's still wrong. It's wrong because the low end of the dynamic range is determined by SNR, and SNR changes when you downsample. The lower end of the dynamic range is NOT some point at which the signal is clipped. It is not in general true that a pixel is completely insensitive outside the dynamic range. The baseline of dynamic range is defined in terms of SNR, not detectability.

You do run into a "quantization limit" at the number of bits in the ADC (assuming that both the sensor response and the ADC itself are linear), but even then, you do eliminate shadow noise and therefore increase your usable dynamic range. But this is not an issue with the new canon bodies because their reported dynamic range does not exceed that of the ADC.

I should add further that the screen "dynamic range" is also in some sense "just a score". A SNR of 0db is extremely noisy (as a point of comparison the powershot S90's noise at max ISO 18% gray is 13db), so depending on your criteria, the "usable dynamic range" might be a few stops less than that indicated by the screen score.

In the case of the 1D X, what you describe might indeed be the case. It is a bit different than the D800, wherein 14.4 stops of DR surpasses the bit depth of the ADC and the limit imposed by quantization. If we were purely talking about the Gaussian type of noise produced by the Poisson distribution of photons, I'd probably agree. I am not sure the same simplistic rules that apply to gaussian noise apply to the non-random forms of electronic/read noise, which is the primary cause of a LOSS of DR at low ISO in Canon sensors.

 

[peederj]

Mikael,

FWIW, you're even more nuts.

Hugs and Kisses... :-*

 

[jrista]

The Canon brand fanatics writes about arguments regarding why something suddenly not important anymore (when "their" brand suddenly is not the best ) this take quite comical proportions ... Some very active writers, among others here at Canonrumors's are direct rabid - this despite the notice that they are intelligent enough to see the connections, there is an instinctive religious barrier that prevents them to see what's right in front of their eyes ...

You might carry a bit more weight with your arguments if you actually posted facts, undoctored evidence, and did not resort to personal attacks. I have respect for elflord, even if I don't fully agree with his assessments (which I primarily do agree, I just disagree on some of the finer points, as do a number of others here).

You, on the other hand, well to be frank I don't have a shred of respect for you. I'd also point out my fanaticism isn't really about Canon. It is about DXO's scoring system and my general dislike of it, and the often unenlightened application of their scores by die-hard Nikon fanatics. ;P I apply my same skepticism for DXO's scoring in regards to Canon as I do in regards to Nikon (or any other brand, for that matter).

I'll freely admit that I greatly dislike the notion that simply because the D800 is a great camera, then all other cameras (and particularly Canon cameras) suck donkey danglers, and that the D800 is the end-all, be-all of cameras everywhere, infinitely capable of everything to everyone (a notion you would certainly think was true if you believed all the drivel from the majority of the raging Nikon fanbase). The advent of new & improved technology does not immediately and entirely negate previous technology, or diminish the quality it previously attained. The regular arguments to the contrary, and the bogus doctored, contrived or staged evidence to try and prove that bogus fact, are the fundamental basis of my tirade against such inanity. I've never disagreed that Exmor is some phenomenal technology, nor have I ever said Canon sensors do as well (although I certainly believe they can do as well or better, and probably will in coming Canon sensor generations). I simply deny that the difference between Canon cameras and Nikon cameras is as big as it is usually made out to be by the fanatical branch of the Nikonites. I also believe a periodic reality check to that fact is beneficial to those potentially...gullible enough?...to waste time, effort, and money jumping ship and switching brands when in most cases (there are a few outliers) it is unnecessary, and people could save their hard-earned cash.

 

[jocau]

The 1D X scores still can't be found on the DXOMark website. 12,8 stops of DR out of a Canon sensor? I won't believe it until I see it.

 

[peederj]

The conspiracy theorists would suggest DxO is waiting the completion of "sponsorship negotiations" as to what the final score will be.

 

[LetTheRightLensIn]

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

You keep repeating it, but it's still wrong. It's wrong because the low end of the dynamic range is determined by SNR, and SNR changes when you downsample. The lower end of the dynamic range is NOT some point at which the signal is clipped. It is not in general true that a pixel is completely insensitive outside the dynamic range. The baseline of dynamic range is defined in terms of SNR, not detectability.

You do run into a "quantization limit" at the number of bits in the ADC (assuming that both the sensor response and the ADC itself are linear), but even then, you do eliminate shadow noise and therefore increase your usable dynamic range. But this is not an issue with the new canon bodies because their reported dynamic range does not exceed that of the ADC.

I should add further that the screen "dynamic range" is also in some sense "just a score". A SNR of 0db is extremely noisy (as a point of comparison the powershot S90's noise at max ISO 18% gray is 13db), so depending on your criteria, the "usable dynamic range" might be a few stops less than that indicated by the screen score.

In the case of the 1D X, what you describe might indeed be the case. It is a bit different than the D800, wherein 14.4 stops of DR surpasses the bit depth of the ADC and the limit imposed by quantization. If we were purely talking about the Gaussian type of noise produced by the Poisson distribution of photons, I'd probably agree. I am not sure the same simplistic rules that apply to gaussian noise apply to the non-random forms of electronic/read noise, which is the primary cause of a LOSS of DR at low ISO in Canon sensors.

advice:

just quit before you dig an even deeper hole on this issue

Elford has it right.

 

[thepancakeman]

The 1D X scores still can't be found on the DXOMark website. 12,8 stops of DR out of a Canon sensor? I won't believe it until I see it.

And that's what cracks me up about this whole thing. If it were that significant or that drastic, you wouldn't need a test to tell you about it, you'd see it in the photos. Just my $.02.