DxOMark Sensor Performance: Nikon vs. Canon

[Meh]

Everybody plays with the same laws of physics: I'd be very surprised if Canon does not have high-DR tech in their back pocket.

Those pesky laws of physics always getting in the way.  Not sure if Canon would have some advancement that they're holding back but it could be the sensor engineers have not been given the mandate/budget to improve low-ISO DR or they find it sacrifices something else.  Hard to say.

[Meh]

I'm just surprised at how many people call foul at DXO without taking the effort to really analyze what the measurements/results show.

+1

confirmation bias, thats why you didnt also get a response over your comment @ dtaylor and his "the right way" DR test.

+1 confirmation bias is detestable!

[Meh]

Sensor performance isn't everything...

Not only not everything, but far less than even remotely close to everything.  Consider all the glass the light must pass through before it even reaches the sensor.

Also consider that DxOMark is evaluating only the sensor, and also that their "Overall Score" is composite of three arbitrarily chosen "Use Case Scores" that are combined in a 'weighted' manner, but the weighting is not disclosed.  Furthermore, their use case scores are normalized to an 8 MP file size, which explains how a camera with 14 bits per pixel can, according to DxOMark, actually deliver a dynamic range greater than 14 bits of EV. 

IMO, their Measurements (screen) are valid and quite useful.  Their Scores are steaming pile of misleading cow excrement.

Just a little reality check...   

That's about right 

[sarangiman]

+1

confirmation bias, thats why you didnt also get a response over your comment @ dtaylor and his "the right way" DR test.

Well, let's give @dtaylor some more time to respond. If I remember correctly, he said he uses transmission wedges for his tests, so I'd be very surprised if he got different comparative results between bodies. Absolute results may vary based on what you set as your acceptable SNR on the low end. But even then, the D800 should consistently show ~2.5EV higher DR (pixel-level) than 5DIII. I've tested this numerous times, b/c I was also incredulous at first.

I even remember being incredulous about the D800 beating the D4 in DR, but the data is the data. The D800, despite having a lower saturation capacity per-pixel, simply has lower read noise on a per-pixel basis. Low enough that its DR even trumps the D4. That's also why it's not hard to imagine the D800 beating medium format in DR (though I agree with neuroanatomist that I'm not sure how fair the normalization formula is | e.g. D800 @36MP goes up 1.1EV from screen to print, Hasselblad @50MP goes up 1.35EV, D600 @24MP goes up 0.8EV... I really can't comment one way or the other if it's fair as I haven't delved into their normalization formula yet).

Interesting point about DXO getting the D600 scores out before the 1Dx. Then again, if I had a D600 vs. a 1Dx in my hands, I'd be more excited about testing the D600 b/c the 1Dx has already been shown to not have much more DR than the 5DIII (e.g. here: http://home.comcast.net/~NikonD70/Charts/PDR.htm#EOS%205D%20Mark%20III,EOS%201D%20X)

[jthomson]

Strange that they can get the Nikon D600 out on the day that it is released but haven't yet got around to the 1DX.

That speaks to an entirely different potential bias, but without knowledge about the rationale for the discrepancy, no conclusions can be drawn.  Maybe Nikon overnight shipped them the first D600 off the line, and they're still waiting on a retail back ordered 1D X. Heck, noted bird photographer and Canon Explorer of Light, Art Morris, just recently got his 1D X - I've had mine longer than him.

Good point about the availability of the 1DX but  I was just at their site and they don't even have a rating for the  T4i/ 650D.   

[Zlatko]

A lot of denial in the forum about inferiority of Canon's sensors. The knee jerk argument is that Dxomark:
1) is biased, or
2) their tests are bad because "I" get superior results out of my Canon, unlike "my" Nikon friend who is struggling, or
3) is meaningless because I can bracket and get high DR, so low DR is not big deal

DXOMarks is simply giving empirical evidence to the inferiority of Canon's decade old tech versus modern tech of Sonikon. If we as Canon consumers (whether we like the brand or, more importantly, stuck to it due to sunk costs) live in denial, Canon has no incentive to improve. Result? we get shafted while competition has better cameras. Period!

The problem with DxO's scoring is that it doesn't match what I can see with my own eyes.  Last year I shot two Canon 5D Mark II's and two Nikon D7000's.  The Canon is in a higher price category, but DxO actually scores the D7000 higher:

Canon 5D Mark II score = 79
Nikon D7000 score = 80

So the D7000 sensor is better, right?  Not at all.  Just compare the color in portraits from each camera and judge which one shows humans looking more like actual humans.  The 5D Mark II's won easily; the D7000's were actually worse in color and more prone to blowing highlights.  Of course, that result is to be expected based on the rather big price difference, but DxO would have you believe the those cameras' sensors are a close match and that the D7000's is a tiny bit better.  Well, they are very wrong.

So whatever DxO is measuring, they aren't measuring everything that's important to me or scoring in a way that matches my experience.  I heard a similar experience from a wedding photographer who had moved to a higher end Nikon and missed the color of his 5D Mark II.

[Meh]

1. their overall scores are weird, how do you combine so many different performance factors into one number that would work for everyone? you can't

Precisely.  They have arbitrarily designed one of many possible algorithms to combine their various measurements into a "score".

[sarangiman]

The 5D Mark II's won easily; the D7000's were actually worse in color and more prone to blowing highlights.

A camera being prone to blowing highlights says more about its metering than its dynamic range. The dynamic range of the D7000 is demonstrably worlds above that of any of the 5D cameras. Yes, despite its price. It's all about saturation well capacity & read noise (& megapixel count of course, if we're talking about normalized results).

[Zlatko]

agree...if you look at the scores of dxo on low light iso the 1d mark 3 is destroied by a d5100; but maybe i am blind

There is something absurd about those scores.  If the D5100 is anything like the D7000, then I would much prefer the sensor of the 1D Mark III.

[Meh]

Question for you: my understanding of Bayer pattern sensors is that a group of 4 monochrome {R|G|B} are de-mosaiced to form one color RGB pixel.  If I combine two 14-bit wells I get (roughly) one 15-bit well.  If I combine four 14-bit wells I should get (roughly) one 16-bit well.  Certainly they wouldn't do a crude addition of wells, but it seems very plausible that you can get >14EV DR, even without re-normalizing.  What's wrong with this interpretation?

I don't believe it's correct to say that 4 are combined into 1.  The de-mosaicing is an interpolation algorithm that uses the values from adjacent pixels to estimate the intensity and color of each pixel.  The demosaicing algorithm doesn't do any binning in the way you suggest.   IIRC the Canon C300 does perform the binning that way to give better color accuracy for each pixel.

[Zlatko]

The 5D Mark II's won easily; the D7000's were actually worse in color and more prone to blowing highlights.

A camera being prone to blowing highlights says more about its metering than its dynamic range. The dynamic range of the D7000 is demonstrably worlds above that of any of the 5D cameras. Yes, despite its price. It's all about saturation well capacity & read noise (& megapixel count of course, if we're talking about normalized results).

As I wrote, the problem was the color in portraits was distinctly worse.  The blown highlights were an added problem.  I'm talking about using the cameras for actual photography, not lab tests.  So, yes, consistent with its price, the D7000 was distinctly worse in actual photography.  The D7000 was a bit like a kid who scores great on a standardized test, but is lacking in essential people skills.

[gilmorephoto]

The 5D Mark II's won easily; the D7000's were actually worse in color and more prone to blowing highlights.

A camera being prone to blowing highlights says more about its metering than its dynamic range. The dynamic range of the D7000 is demonstrably worlds above that of any of the 5D cameras. Yes, despite its price. It's all about saturation well capacity & read noise (& megapixel count of course, if we're talking about normalized results).

As I wrote, the problem was the color in portraits was distinctly worse.  The blown highlights were an added problem.  I'm talking about using the cameras for actual photography, not lab tests.  So, yes, consistent with its price, the D7000 was distinctly worse in actual photography.  The D7000 was a bit like a kid who scores great on a standardized test, but is lacking in essential people skills.

+1

[sarangiman]

As I wrote, the problem was the color in portraits was distinctly worse.  The blown highlights were an added problem.  I'm talking about using the cameras for actual photography, not lab tests.  So, yes, consistent with its price, the D7000 was distinctly worse in actual photography.  The D7000 was a bit like a kid who scores great on a standardized test, but is lacking in essential people skills.

I understand, but my point is simply that the 'blown highlights', in this case, is not indicative of anything wrong with DXOs sensor scores for the D7000. The (Sony) sensor in the D7000/D5100 really does have significantly more DR than any Canon sensor.

If you found that it blows highlights too often, then perhaps you disagree with Nikon's metering algorithms in those situations.

As for Nikon/Canon color, I'm inclined to agree with you & say my personal taste is for Canon's default colors. But I bet if you really wanted to you could build a DNG profile that makes a Nikon camera match Canon's default colors. Whether or not you wish to do so is another matter altogether.

[Martin]

Actually I think that canon makes everthing to postpone the 1dx scoring by dxo. It would be funny if d600 scored better than flagship canon. Unfortunately thats probably the reality.

[weixing]

Hi,
   IMHO, DxOMark is not testing the sensor, but the camera itself because they use the RAW file produce by the DSLR (they didn't remove the sensor from the body and test, right?) and if the RAW file produce by the camera is not the RAW data, then it's not testing the sensor and the sensor compare between brands is somehow not valid. I believe all RAW file is not RAW data and Nikon RAW file are well know for it.

   Anyway, I was wondering how do Nikon D800 and D600 get a dynamic range of more than 14EV when there is only 14-bit of data? Compress the dynamic range or non-linear data?

   Have a nice day.