DxOMark Sensor Performance: Nikon vs. Canon

[dtaylor]

dtaylor: How do you test DR?

Stouffer step wedge and visually inspect the results.

Furthermore, your results match DPReview? DPReview doesn't test RAW dynamic range...

They used to report RAW and JPEG.

My own 'real-world' tests also show ~3EV better DR on the D800 when I do side-by-side shots of high DR sunsets with my 5DIII vs. D800;

I have a hard time believing 3 stops, though I must admit I have not formally tested these bodies.

Put another way: I have to overexpose my 5DIII by 2 to 3 stops at the very least to get its shadows to match the cleanliness of lifted shadows of the D800 file that was underexposed to maintain highlights.

"Match the cleanliness" is a wide open question. Are you matching at 200% in PS or in a 20" print? And what constitutes a "match"? Does the 5D3 not "match" if there's a hint of noise that's irrelevant to 99% of uses? And to what degree does color play a role? (When you push RAW converters you can often recover detail that is correct in terms of tone, but incorrect in terms of color. How much of this you're willing to accept will alter the final judgement on DR.)

So, respectfully, I fail to see how DXOs DR & SNR, etc., numbers are the 'odd ball out'.

When I've compared their results to other sites, or to my own experience, they have not matched. One example: according to DxO the 7D (Canon's 18 MP sensor) has little DR gain over the 10D / 20D. I could tell you before formally testing them that it was large, 2 stops easily.

Now maybe I'm being too harsh. Maybe their current tests are better, or maybe it just so happens that the cameras I compared were the odd balls, not the entire testing methodology. I'll take another look. But DxO seems easily thrown by small factors, or easily gamed. Michael Reichmann was a big fan when they first started, then dropped them later because really tiny things would shove one score well above another, and not just on DR.

To clarify: I don't at all mean this to be a personal attack; just looking for clarification.

As was evident from the tone of your post, and I appreciate that. I'll look more carefully at DxO's latest results.

[mystic_theory]

As for dpreview, actually look at what they had to say about Canon & the 6D:

"Overall, though, it's difficult to shake the feeling that the EOS 6D simply lacks the 'wow' factor of its main rival. Whereas Nikon seems to have taken the approach of taking away as little as possible from D800 when creating the D600, Canon appears almost to have gone the other way, removing as much as it thinks it can get away with at the price. The result is the kind of conservative, slightly unimaginative design that's become the company's hallmark. It's still bound to be a very good camera, of course; just perhaps not quite as good as it could be."

http://www.dpreview.com/previews/canon-eos-6d/6

I think dpreview does a good job of trying to remain unbiased.

I see: well, I have to admit that I haven't read a dpreview review in a long while, since I read one on the 600D that I found very biased. One excerpt says:
"The Highlight Tone Priority option (Custom Function II.6) is a method for capturing more information in the brightest parts of the scene. It does this by applying ... Turn this on and the 600D captures an extra stop in the highlights, resulting in an overall range that at least matches that of Sony and Nikon models."
Notice that Nikon models have similar options, and the 5100D has about 3 stops more DR. But maybe things have changed at dpreview.

Notice also that I own a 550D, since I wanted to keep my awesome 10-22mm Canon but didn't want to spend extra bucks to have the same sensor for the 650D (or for a 60D, a 7D, an eos M, and the list goes on and on and on). I hope that within a few months there will be a new crop sensor from Canon with decent DR and low noise at high ISO: such a 70D would be perfect for me.

[Fishnose]

I love when idiots see some test, and then jump all over it.

Looks exactly like what you just did, since DXO is about sensors and not about cameras. When choosing a camera surely many other factors are to be considered, but that wasn't the point of these tests to begin with.

All your arguments make therefore little to no sense.

+1

Maui5150, before you call other people idiots (and you actually name them by quoting), make sure you know where you're treading - so you don't step in your own turds

[weixing]

Hi,

   DxO said they test the sensor, but IMHO, they are actually testing the camera, because they test using the RAW file from the camera not the RAW data directly from the sensor. If the RAW file is not RAW data, then you are not testing the sensor, but how camera handle the image.

   Have a nice day.

[Fishnose]

Hi,

   DxO said they test the sensor, but IMHO, they are actually testing the camera, because they test using the RAW file from the camera not the RAW data from directly the sensor. If the RAW file is not RAW data, then you are not testing the sensor, but how camera handle the image.

   Have a nice day.

Yes, they're testing the camer'a sensor and the part of the electronics that gets data off the sensor and turns it into a RAW image file. Nothing else is tested - not build quality, not sealing, not AF or metering, not any other functions, not anything except the sensor and the electronics directly involved in creating an image from it.
So what you say is only partly true.

[Mt Spokane Photography]

...Are the DxO tests rigorous enough to be published in a serious scientific journal? Very likely, no. But for what they are meant to do (publicize a software), they are outstanding.

As I stated, their testing methodology is rigorous, and the data are of high quality.  It's their data interpretation that's flawed.

Yes, this.

+1
How they arrive at their numbers is very questionable.  However, if you look at their data rather than their sensor score, you can find out what you need to know.
Trying to give weight to numerous important parameters and put them into one score doesn't work, it does not tell you if a parameter you value highly is being given a low weighting.

[bdunbar79]

I love when idiots see some test, and then jump all over it.

Looks exactly like what you just did, since DXO is about sensors and not about cameras. When choosing a camera surely many other factors are to be considered, but that wasn't the point of these tests to begin with.

All your arguments make therefore little to no sense.

+1

Maui5150, before you call other people idiots (and you actually name them by quoting), make sure you know where you're treading - so you don't step in your own turds

I agree.  Now, I've gotten into some disagreements on here that I wish I hadn't and maybe emphasized my point a little too strongly, but it's never been personal and I look at it like a debate on one of those Sunday morning political shows.  It's all in good fun and respect.  I don't believe it is in good taste to quote someone and then put the word idiot in there.  Afterall, Dx0Mark is only talking sensors, not the complete overall camera.  Whether their sensor scores make sense or not, that's another issue.

[nightbreath]

As for Nikon/Canon color, I'm inclined to agree with you & say my personal taste is for Canon's default colors. But I bet if you really wanted to you could build a DNG profile that makes a Nikon camera match Canon's default colors. Whether or not you wish to do so is another matter altogether.

I don't see possibility to do that. Just imagine two curves represented by diferent equations. There's no way you can easily come up with algorithm that makes transformation of one curve to another. It's easier to draw the required curve from the start 

[marekjoz]

As for Nikon/Canon color, I'm inclined to agree with you & say my personal taste is for Canon's default colors. But I bet if you really wanted to you could build a DNG profile that makes a Nikon camera match Canon's default colors. Whether or not you wish to do so is another matter altogether.

I don't see possibility to do that. Just imagine two curves represented by diferent equations. There's no way you can easily come up with algorithm that makes transformation of one curve to another. It's easier to draw the required curve from the start 

You only need three 14 bits long tables/matrixes/hashtables

[nightbreath]

As for Nikon/Canon color, I'm inclined to agree with you & say my personal taste is for Canon's default colors. But I bet if you really wanted to you could build a DNG profile that makes a Nikon camera match Canon's default colors. Whether or not you wish to do so is another matter altogether.

I don't see possibility to do that. Just imagine two curves represented by diferent equations. There's no way you can easily come up with algorithm that makes transformation of one curve to another. It's easier to draw the required curve from the start 

You only need three 14 bits long tables/matrixes/hashtables

It's not that simple 

Color of each individual pixel from my point of view can be result of evaluation of:
1. R-G-B channels of current pixel.
2. Colors of pixels-neighbors.
3. Overall picture color.
4. Exposure (?)

So if there's complex equation that is used to process specific pixel color, you won't be able to easily switch from one color evaluation scheme to another.

[marekjoz]

As for Nikon/Canon color, I'm inclined to agree with you & say my personal taste is for Canon's default colors. But I bet if you really wanted to you could build a DNG profile that makes a Nikon camera match Canon's default colors. Whether or not you wish to do so is another matter altogether.

I don't see possibility to do that. Just imagine two curves represented by diferent equations. There's no way you can easily come up with algorithm that makes transformation of one curve to another. It's easier to draw the required curve from the start 

You only need three 14 bits long tables/matrixes/hashtables

It's not that simple 

Color of each individual pixel from my point of view can be result of evaluation of:
1. R-G-B channels of current pixel.
2. Colors of pixels-neighbors.
3. Overall picture color.
4. Exposure (?)

So if there's complex equation that is used to process specific pixel color, you won't be able to easily switch from one color evaluation scheme to another.

I always thought it's simple color shift. Is it not the way simple camera calibration like Color Checker works like?
Has anyone viewed calibration file?

[Chuck Alaimo]

From my point of view it is evident that sony/nikon sensors are superior to canon's. I would love to have my 5d III with the D600 sensor.
It is hard to believe that the majority of canon users are not disappointed with the increasing gap in sensor technology between Nikon and Canon.

I'm not a fan of either brand. I'm just a Canon user with a lot of glass and therefore married for good or bad. Right now, things are going bad.

I can see wanting the D800 sensor, but the d600?  Actually, I can't even really see wanting that sensor.  I've said it before and will say it again ---  reading this makes one wonder whether I should do things like, present a wedding gallery to a client filled with images or have all the images sent to dxo, have them rated and send the test results to the client cause obviously that's what matters? 

I honestly don't see why people keep beating this dead horse.  It's dead, and its been beaten, now your beating on it more ---  But I gess it needs to be said again---for this round of bodies canon seems to have chosen to leap ahead in high ISO performance (while also putting a ton into lens dev, putting out a dedicated cinema line.  And if the recent rumor is true, canon does have a 46.1 mp body in the final dev stages), AF and processor power.   Nikon/sony went for MP's and DR.  Like it or not, thats where we are NOW.  Will it be that way forever, no, canon will catch up in DR and nikon will catch up in high ISO, then we'll wait for the next round of dev.  Until then, go take some freaking pictures!!!!

[elflord]

When I've compared their results to other sites, or to my own experience, they have not matched. One example: according to DxO the 7D (Canon's 18 MP sensor) has little DR gain over the 10D / 20D. I could tell you before formally testing them that it was large, 2 stops easily.

Now maybe I'm being too harsh. Maybe their current tests are better, or maybe it just so happens that the cameras I compared were the odd balls, not the entire testing methodology. I'll take another look.

You keep posting that DxO claim that the 10D has about the same dynamic range as the 7D but the graphs on DxO's website don't support this.

The 7D appears to have about a 2 stop advantage over the 10D. For example, the 7D at ISO 400 beats the 10D at ISO 100. The same holds all the way through the ISO range -- ISO 800,1600,3200 and 6400 on 7D are better than 200, 400, 800, 1600 respectively on the 10D. 

So the 7D gives me about 2 more stops of ISO for any given dynamic range over the 10D. That's a pretty substantial difference in my book.

[Sony]

To make it clear, there is no new Nikon sensor. They are Sony sensors in Nikon cameras. One of my friends cried with me last night that his Sony A900's mirror fell out so he has to let it sleep. Sony's technology !!!

[naskatar]

I do not believe in those DxO results.

Just compare their DxO Optics Pro software raw conversion results to those from Adobe software regarding detail extraction and image sharpness.
(At least with Canon raw files. I have never tried any Nikon files due to the lack of a Nikon camera...)

While the DxO software is pretty good for a quick conversion, they are miles behind in other aspects.
So how should they be able to judge about sensor quality if they do not know how to get the optimum results of them?