Canon EOS-1D X DXOMark Sensor Scores

[Viggo]

... The 1d X lacks the f8 focusing and the superlow light af lock and red lit up points. Those things doesn't matter to me, and everything else it just doesn't do better than anything else, it does it to perfection. A real tool I can trust for 100% of my pictures. Only eyes will capture a moment better..

Unless the camera is in the high-speed mode 

That is true. I was thinking about IQ..

[Larry]

'DxO has no credibility'.... who says? You and a couple of others here who don't like the results they publish, and just don't understand that DxO CANNOT AFFORD TO BULLSHIT...
 
Anyway, the 1Dx is an absolutely unbelievably good camera. It's not made to get high scores, it's made to get fantastic results in almost impossible situations, especially low light of course.

I hope you see the obvious self-contradiction in what you wrote

Wow, you don't have a clue, do you. I write that the 1Dx is "not made to get high scores, it's made to get fantastic results in almost impossible situations".
In what way does that contradict anything else I wrote?

I'll write it again, to make sure you get it this time: Canon did not make the 1Dx to score one way or the other in any test, Canon made it to be an incredibly good camera at what it does. And Canon succeeded.

+1    No contradiction!

[Woody]

'DxO has no credibility'.... who says? You and a couple of others here who don't like the results they publish, and just don't understand that DxO CANNOT AFFORD TO BULLSHIT...
 
Anyway, the 1Dx is an absolutely unbelievably good camera. It's not made to get high scores, it's made to get fantastic results in almost impossible situations, especially low light of course.

I hope you see the obvious self-contradiction in what you wrote

Wow, you don't have a clue, do you. I write that the 1Dx is "not made to get high scores, it's made to get fantastic results in almost impossible situations".
In what way does that contradict anything else I wrote?

I'll write it again, to make sure you get it this time: Canon did not make the 1Dx to score one way or the other in any test, Canon made it to be an incredibly good camera at what it does. And Canon succeeded.

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

[heptagon]

Well they do matter for a few specific use cases. E.g. if you can't get the lighting right and need to brighten your pictures.

[neuroanatomist]

Well they do matter for a few specific use cases. E.g. if you can't get the lighting right and need to brighten your pictures.

I'll buy that. But the examples provided recently in other threads don't show 'can't get the lighting right', they show intentionally choosing to get the lighting wrong, then brightening the image that should not have needed brightening if properly exposed to begin with.

[Viggo]

Well they do matter for a few specific use cases. E.g. if you can't get the lighting right and need to brighten your pictures.

I'll buy that. But the examples provided recently in other threads don't show 'can't get the lighting right', they show intentionally choosing to get the lighting wrong, then brightening the image that should not have needed brightening if properly exposed to begin with.

+1 I expose 5/8 to the right by moving my 0 ev, and this leads to pulling shadows DOWN or even the whole exposure, if need. I haven't pulled UP a single image with the 1d x, the metering is fantastic. Leaving me with perfect exposure and no need to do it wrong and try to rescue it later. Learn how to expose instead....

[elflord]

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

That's not what he said at all.

He said that Canon didn't design their camera with DXO's benchmarks in mind. Instead, the manufacturer (Canon) have their own design goals, performance goals, use cases, etc in mind. They also almost certainly do some kind of internal benchmarking.

That doesn't mean that DxOMark's performance measurement is uncorrelated with real world performance (or for that matter, benchmarks that Canon might use internally to test their products).

It also most certainly doesn't mean that DxOMark's performance measures are useless.

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

[Woody]

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

Ultimately, real world results are what counts, yes? Photographers buy cameras to capture images, not to pass specific testing processes or specific testing regimes, ya?

So, can you point out specific real world scenarios that Canon's cameras perform really well, outside of what DXOMark can ever reveal? Perhaps Canon's internal test regimes are tailored for such instances?

[elflord]

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

Ultimately, real world results are what counts, yes? Photographers buy cameras to capture images, not to pass specific testing processes or specific testing regimes, ya?

So, can you point out specific real world scenarios that Canon's cameras perform really well, outside of what DXOMark can ever reveal? Perhaps Canon's internal test regimes are tailored for such instances?

To address your first question(s) --

I'm not sure what you think you're disagreeing with. Better equipment helps us get "real world results", and the sensor is a key part (but not all of) our camera equipment. Photographers have gotten great results before the latest sensors were available, indeed, before digital was available.

If you're happy to use the same equipment that produced great results for photographers who didn't have access to the latest gear, then you have absolutely no need to pursue the latest technology.

However, if you are in the market for the best technology the market has to offer, benchmark results count.

You can wait until the technology in question has an established track record, to see which products get the best "real world results" but there are least two problems with this -- one is that by the time this established track record is realized, it is not the latest tech any more.

The other is that "real world results" depend more on the skill of the photographer than the technology. You can buy the same equipment used by the photographer  produces the greatest "real world results", but the photographers talent does not come in the box with the equipment.

In answer to your second question, DxOMark only test sensors. The camera is much more than a sensor. Canon almost certainly include considerable testing of features including but not limited to ergonomics, durability (weather proofing and longevity). and autofocus performance. And there are yet other factors that can't really be tested, but are important all the same -- Canon's unique ability and willingness to stand behind their products. This is a combination of a first rate support system, and complete control over their product line. For example, there is no other full format DSLR manufacturer that completely manage their product development (cameras, sensors, lenses). Nikon's performance numbers are impressive but one could reasonably doubt Sony's commitment to Nikon's DSLR line. Nobody doubts Canon's commitment to Canon.  Canon's strategic decision to avoid taking the easy way out and just buying the sensor might seem to hurt them in the short run, but it is a strategic decision that takes the long view.

[jrista]

There are at least two compelling advantages of DxOMark here -- one is the simple practical one -- as we do not have access to Canon's internal processes, we can't use their internal benchmarking results to appraise the 1DX or compare it to competing products (again, assuming Canon have an internal testing regimen, one would expect that they would run all of their own, and several competing products through it).

Second, even if we could see those results, it would be misleading to compare cameras that had been tuned to those specific testing processes with those that hadn't (basically, it would be analogous to in-sample versus out-of-sample testing)

Ultimately, real world results are what counts, yes? Photographers buy cameras to capture images, not to pass specific testing processes or specific testing regimes, ya?

So, can you point out specific real world scenarios that Canon's cameras perform really well, outside of what DXOMark can ever reveal? Perhaps Canon's internal test regimes are tailored for such instances?

To address your first question(s) --

I'm not sure what you think you're disagreeing with. Better equipment helps us get "real world results", and the sensor is a key part (but not all of) our camera equipment. Photographers have gotten great results before the latest sensors were available, indeed, before digital was available.

If you're happy to use the same equipment that produced great results for photographers who didn't have access to the latest gear, then you have absolutely no need to pursue the latest technology.

However, if you are in the market for the best technology the market has to offer, benchmark results count.

You can wait until the technology in question has an established track record, to see which products get the best "real world results" but there are least two problems with this -- one is that by the time this established track record is realized, it is not the latest tech any more.

The other is that "real world results" depend more on the skill of the photographer than the technology. You can buy the same equipment used by the photographer  produces the greatest "real world results", but the photographers talent does not come in the box with the equipment.

In answer to your second question, DxOMark only test sensors. The camera is much more than a sensor. Canon almost certainly include considerable testing of features including but not limited to ergonomics, durability (weather proofing and longevity). and autofocus performance. And there are yet other factors that can't really be tested, but are important all the same -- Canon's unique ability and willingness to stand behind their products. This is a combination of a first rate support system, and complete control over their product line. For example, there is no other full format DSLR manufacturer that completely manage their product development (cameras, sensors, lenses). Nikon's performance numbers are impressive but one could reasonably doubt Sony's commitment to Nikon's DSLR line. Nobody doubts Canon's commitment to Canon.  Canon's strategic decision to avoid taking the easy way out and just buying the sensor might seem to hurt them in the short run, but it is a strategic decision that takes the long view.

Very well said! +1

[Fishnose]

'DxO has no credibility'.... who says? You and a couple of others here who don't like the results they publish, and just don't understand that DxO CANNOT AFFORD TO BULLSHIT...
 
Anyway, the 1Dx is an absolutely unbelievably good camera. It's not made to get high scores, it's made to get fantastic results in almost impossible situations, especially low light of course.

I hope you see the obvious self-contradiction in what you wrote

Wow, you don't have a clue, do you. I write that the 1Dx is "not made to get high scores, it's made to get fantastic results in almost impossible situations".
In what way does that contradict anything else I wrote?

I'll write it again, to make sure you get it this time: Canon did not make the 1Dx to score one way or the other in any test, Canon made it to be an incredibly good camera at what it does. And Canon succeeded.

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

Wow, you really are confused. 

If the DxO figures rumored are in fact the official DxO results, then the numbers mentioned are exceptionally high and the 1Dx gets fantastic results.
Remember, DxO are testing the SENSOR - not the camera.
The camera obviously is fantastic - and, according to DxO, so is the sensor. The numbers are stellar.

That the D600 and D800 get even higher marks (for their SENSORS) has got nothing to do with anything in this situation - they're a different kind of camera.

The 1Dx is made specifically for low light work and other difficult situations. The D600 and D800 are definitely not made for this.

So - what was your point again? In what way are DxO not credible? Be specific now, explain your point with some kind of useful argument other than 'DxO is a joke' - as infared put it.
Very erudite, to say the least 

[jrista]

'DxO has no credibility'.... who says? You and a couple of others here who don't like the results they publish, and just don't understand that DxO CANNOT AFFORD TO BULLSHIT...
 
Anyway, the 1Dx is an absolutely unbelievably good camera. It's not made to get high scores, it's made to get fantastic results in almost impossible situations, especially low light of course.

I hope you see the obvious self-contradiction in what you wrote

Wow, you don't have a clue, do you. I write that the 1Dx is "not made to get high scores, it's made to get fantastic results in almost impossible situations".
In what way does that contradict anything else I wrote?

I'll write it again, to make sure you get it this time: Canon did not make the 1Dx to score one way or the other in any test, Canon made it to be an incredibly good camera at what it does. And Canon succeeded.

So, according to you, (i) DXOMark results are accurate (ii) Canon 1DX produces fantastic results REGARDLESS of its DXOMark scores.

This means whatever DXOMark measures is USELESS since their results have no correlation to real world results. Correct?

Wow, you really are confused. 

If the DxO figures rumored are in fact the official DxO results, then the numbers mentioned are exceptionally high and the 1Dx gets fantastic results.
Remember, DxO are testing the SENSOR - not the camera.
The camera obviously is fantastic - and, according to DxO, so is the sensor. The numbers are stellar.

That the D600 and D800 get even higher marks (for their SENSORS) has got nothing to do with anything in this situation - they're a different kind of camera.

The 1Dx is made specifically for low light work and other difficult situations. The D600 and D800 are definitely not made for this.

So - what was your point again? In what way are DxO not credible? Be specific now, explain your point with some kind of useful argument other than 'DxO is a joke' - as infared put it.
Very erudite, to say the least 

I think it the conversation has become rather confused. In general, DXO is a good resource when it comes to IQ. Its not that DXO is entirely non-credible. The problem many of us have with DXO is specifically with their "Print DR" statistic. It is not a measurement, it is a rating, and it is a weighted rating at that. Not only that, but their "Print DR" statistic seems to have an overbearing weight on overall sensor score, making it seem like the only thing that matters in a sensor is dynamic range. The fact that Canon sensors are capable of considerably better high ISO performance these days is so weakly rated that it doesn't seem to matter...at least according to DXO.

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

Everyone needs to keep in mind, these are not yet posted as official results. Assuming they ARE official results, something else to keep in mind:

A DR of 12.8 is the Print DR. As everyone here who has read the debates about DXO before, particularly from Neuro and myself, Print DR statistics from DXO are very misleading. These numbers are not "impossible" like the D800's 14.4, however to remain objective and consistent in my argument:

Just as you couldn't actually capture a scene with 14.4 literal stops of DR with a D800 IN-CAMERA, neither will anyone be able to capture a scene with 12.8 literal stops of DR with a 1D X. For the exact same reasons, the PHYSICAL capabilities of the HARDWARE simply won't allow it. The hardware is rated by DXO's actual measurements, which fall under their Screen DR statistics.

My guess is that the 1D X will still have 11.something stops of real-world HARDWARE DR. The 12.8 stops is only something that might potentially be possible with the right kind of scaling algorithm, and all it will do is allow you to utilize a little more headroom that is normally consumed by noise in a 100% image (however at the tradeoff of detail and resolution...potentially a LOT of detail and resolution, since it required downscaling). The 12.8 stop DR rating tells you about what SOFTWARE can do if it normalizes (bins) noise, but it does not tell you anything about the physical capabilities of the hardware. (Although if 12.8 stops really is the Print DR, it sounds like the QUALITY of the 1D X's noise is really quite good.)

Just to put things in perspective, and maintain a level playing field with consistent arguments: Screen DR tells you about the hardware (and we don't know this yet, probably won't until DXO actually posts the 1D X results on their site.) Print DR tells you about how clever DXO's testing software is and how capable it is at normalizing noise when downscaling.

[JR]

I mentioned this before, but perhaps it was lost in other conversation. My guess is that the 12.8 stops of DR for the 1D X is the "Print DR". It doesn't matter the camera, Canon, Nikon, or anyone else...I think that the Print DR figure is exceptionally misleading and falsely indicative of a sensor's capabilities. My guess is that the 1D X still only has 11.something stops of DR, like all the rest of Canon's cameras.

I would tend to agree there.  Having both the 1DX and the D4 (and I guess the D800), the only number in those DxO "rumored" scores that surprised me was the 12.8 DR.  Overall I do think the 1DX is a superior camera then the D4, however I still find from real life shooting that my D4 gives me more dynamic range when I shoot outside and I dont find the difference marginal either...

One of the biggest difference between both camera also seems to be metering which could explain in part my perceived preference for one camera over the other in certain shooting condition...

[peederj]

I've come to the conclusion that the 1DX is worth paying double for vs. the 5D3. With Big Megapixel Talk on the horizon though, and Canon needing an answer to the D800e (which has a damn fine sensor and that's about it), maybe it's better I wait a bit before trading in this camera that I only got this year.

[jrista]

I've come to the conclusion that the 1DX is worth paying double for vs. the 5D3. With Big Megapixel Talk on the horizon though, and Canon needing an answer to the D800e (which has a damn fine sensor and that's about it), maybe it's better I wait a bit before trading in this camera that I only got this year.

It would indeed be interesting if Canon provided a 46.1mp FF camera without a low pass filter. I'd be very interested in that for landscape work.