DxOMark vs. Reality

[neuroanatomist]

There are lots of posts about DxOMark, lots of support, lots of bashing, some of it fact-based, much of it hearsay and knee-jerk reaction. There are many posts about sales figures, generally fewer than about DxOMark, and generally with even less of them with real facts.

So...let's do a little reality check, about the past 5 years in DxOMark-land and the past 5 years in the real world.

The year is 2007. Nikon has the D3 as a flagship and the D300 as the prosumer APS-C body. Canon has the 1DsIII and D40, respectively. DxOMark scores the D3 as 81, the 1DsIII as 80 - a meaningless difference, by their standards. The D300 scores a 67, while the 40D scores a 64 - that is starting to look like a real difference.

2008 saw Nikon enter the 'affordable' FF market with the D700, which DxOMark scores an insignificant 1 point higher than Canon's 5DII at 80 vs. 79. 2009 had updates to the high-end prosumer APS-C segment, and Nikon's D300S scores a 70, while Canon's 7D scores a 66. 2010 brought us mid-range APS-C updates, and while the 60D with the 7D's sensor scores a 66, the D7000 scores an 80, beating Canon's then-current FF sensors.

This year, the somewhat-less-affordable FF battle continues, with the 5DIII scoring 81, barely better than it's predecessor, while the D800/E scores a very impressive 95/96, soundly beating all Canon cameras as well as every other sensor DxOMark has ever tested, including medium format bodies with sensors with 2.5-times larger sensors.

So, it seems quite clear that over the past 5 years in DxOMark-land, Nikon has been dramatically improving, Canon has been stagnating, and Nikon is way ahead. But that's DxOMark-land. What about the real world?

Canon and Nikon are publicly traded companies, and as such, they publish their annual and quarterly reports (in the Investor Relations sections of their corporate websites). IDC (a market analysis firm) also publishes annual summaries of the market as a whole.

In 2007, Canon had 43% of the dSLR market, Nikon had 40%. In 2010, Canon had 44.5% of the dSLR market, Nikon had 29.8%. So, over years while Nikon was bringing us better sensors than Canon, Nikon was losing market share while Canon was gaining it. Ok, fine, but that's 2010. What about this year?

In 2Q2012 (by calendar year, they report it as 1QFY2013), Nikon had an 18% y/y growth of unit sales of dSLRs and lenses. In 2Q2012, (they report by CY, not FY), Canon had a 47% y/y growth of unit sales of dSLRs and lenses.

So, DxOMark has said Nikon has had better sensors for years, and the sales data show that Canon has sold more dSLRs and lenses for those same years, and continues to do so, as of the most recent data available. The straightforward conclusion from the above is that while DxOMark's Scores have a huge impact on the number of inflammatory posts on Internet discussion boards, they have no meaningful impact on the real world aggregate buying decisions of consumers.

 

[Mt Spokane Photography]

Part of this is because lenses play a very important role in photography, and Nikon has lagged behind with FF lenses. They have very few really good lenses, their 14-24 being the best example followed by their new 85mm f/1.4. Their 24-70mm has horible CA, their 70-200mm is very good. There are a ton of "D" lenses that are good, but no one seems to want them, and they do not have coatings that are well suited to digital.

Then, Nikon nothing that matches Canons 24-105mm L, 135mmL, or 100-400mmL much less the Canon 400mm f/5.6.

Nikon has been cranking out new low cost DX lenses as though the lens side does not believe FF is the way to go. I get the feeliing that they do not have a coordinated plan. If they are bring excellent new FF bodies to the market while developing DX lenses, customers like me who went out and bought a D800 and 24-70 f/2.8, 80-200mm f/2.8, 200-400mm f/4 and discovered that I'd have to buy a Sigma lens if I wanted a decent low cost 400mm lens that could not match the older Canon designs, and found nothing to match my favorite Canon lenses. I finally sold the body and the lenses, and bought another new Canon body.
The Nikon lens prices are generally too high for what you get.
Its a shame, but Nikon does seem to be struggling, and I'd credit their being behind in good lens availability at low prices for a big part of it.

With Sigma agressively churning out ever new and improved lens designs, I'd think this helps Nikon but buyers generally prefer to stick with OEM lenses.

 

[PackLight]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

 

[bornshooter]

DXO mark means nothing to me... i will say glass is what matters and canon's ef range of lenses are untouchable they just make my mouth water i want them all but i will need to sell my girlfriend lol

 

[Mt Spokane Photography]

DXO mark means nothing to me... i will say glass is what matters and canon's ef range of lenses are untouchable they just make my mouth water i want them all but i will need to sell my girlfriend lol

Could you post a photo (of her)

 

[well_dunno]

Regarding the glass, DxO is probably the only source that suggests 70-200 f/2.8 IS mkII is less sharp than mark I (suggesting mkII is better in the center but overall mkI is better, if my memory is not failing me). I do not think that affected the 70-200 f/2.8 mkII sales either, rather the effect was on the credibility of the DxO...

Cheers!

 

[willis]

Regarding the glass, DxO is probably the only source that suggests 70-200 f/2.8 IS mkII is less sharp than mark I (suggesting mkII is better in the center but overall mkI is better, if my memory is not failing me). I do not think that affected the 70-200 f/2.8 mkII sales either, rather the effect was on the credibility of the DxO...

Cheers!

And I've always heard that MK2 is superb sharp even sharper than MK1.

 

[lfg530]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

Seriously? Their 24-70 is really great, way better than the mk1 was. Their 1.4 primes are about as good as the canons are (the canon 24 is excellent tough). Both 70-200 are great. Nikon recently came out with 3 new 1.8g lenses that are awesome and affordable... We're still not talking about the 14-24 or the ridiculous amount of old F mount lenses (e.g the 135 DC or the 55 micro) that are still awesome pieces of glass. Yes Canon also has great glass and some lenses that don't have good nikon counteparts, but nikon is not out of the game in that regard.

I'm really having a problem seeing where the nikon glass is unworthy outside of a fanboy perspective.

 

[PackLight]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

Seriously? Their 24-70 is really great, way better than the mk1 was. Their 1.4 primes are about as good as the canons are (the canon 24 is excellent tough). Both 70-200 are great. Nikon recently came out with 3 new 1.8g lenses that are awesome and affordable... We're still not talking about the 14-24 or the ridiculous amount of old F mount lenses (e.g the 135 DC or the 55 micro) that are still awesome pieces of glass. Yes Canon also has great glass and some lenses that don't have good nikon counteparts, but nikon is not out of the game in that regard.

I'm really having a problem seeing where the nikon glass is unworthy outside of a fanboy perspective.

Thats funny, lets go back to talking about sensors. That way the Nikonians can particapate in the conversation.

 

[Gothmoth]

why should i, as photographer, care about how well nikon or canon sells?

even as a shop owner i don´t care about THEIR numbers.. i care about MY numbers.

as a customer i care what the camera offers me for my money... nothing more nothing less.

canon can outsell nikon 10x times.. that will still not make their current lineup better.

 

[PackLight]

why should i, as photographer, care about how well nikon or canon sells?

even as a shop owner i don´t care about THEIR numbers.. i care about MY numbers.

as a customer i care what the camera offers me for my money... nothing more nothing less.

canon can outsell nikon 10x times.. that will still not make their current lineup better.

The shop owner point is an interesting one.

The local shop pushes Nikon. I asked why, what the owner told me is that Canon controls what he can sell his lenses and cameras for. He gets a bigger margin out of selling Nikon's.

After the purchase they push the off brand lenses even harder because the margin is greater. There is one color that Nikon processes better than Canon, and keeps them in the game. It is green.

 

[Chewy734]

That is interesting... my local AD has lots of banners and large signs outside their storefront... all are Nikon, none are Canon. Inside the two biggest display shelves are Nikon and Canon. Every time I have gone in (probably a couple times a week, to once a month), I've never seen anyone actually buy anything Nikon. But, I've seen a lot of people buy Canon gear. I'm not saying that they aren't selling a lot of Nikon gear, because I'm sure they are, but those are just my observations.

I once asked the rental section of the store a similar question... what do you rent out more often? The answer was Canon equipment 3x as often. Actually, Nikon was only rented slightly more often than Mamiya, Leaf, and the Hassys. In fact Zeiss lenses were more often rented than Nikon lenses.

What's interesting is that when I went to the Browns game last month, there were only two photographers shooting Nikon. The other dozen were shooting Canon. Why?

But then, last year I went to Belize for vacation. While walking around, going on excursions, etc, I did not see one other Canon... not one... not even a tiny PowerShot. All I saw were Nikon dSLRs, compacts, etc. I was the only one I saw shooting with my Canon (5D2 at the time).

So, what's this all mean? Who knows.

 

[lfg530]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

Seriously? Their 24-70 is really great, way better than the mk1 was. Their 1.4 primes are about as good as the canons are (the canon 24 is excellent tough). Both 70-200 are great. Nikon recently came out with 3 new 1.8g lenses that are awesome and affordable... We're still not talking about the 14-24 or the ridiculous amount of old F mount lenses (e.g the 135 DC or the 55 micro) that are still awesome pieces of glass. Yes Canon also has great glass and some lenses that don't have good nikon counteparts, but nikon is not out of the game in that regard.

I'm really having a problem seeing where the nikon glass is unworthy outside of a fanboy perspective.

Thats funny, lets go back to talking about sensors. That way the Nikonians can particapate in the conversation.

Nice to see you're open minded and bringing proof to the conversation; you don't seem to be a mindless Canonian at all.

Just sayin': I'm the proud owner of a 7d and macro 100L wich I loove, I tried several canon primes and L zooms and most of them were great, but I also compared to equivalents in nikon and their glass has nothing to be ashamed of. It's really cute to see some of you guys acting like canon glass is like an affordable leica kind of thing, but it's not and nikon is serious competition in that matter (at least for what I tested MYSELF and any review site you'll find that compare both).

 

[bvukich]

But then, last year I went to Belize for vacation. While walking around, going on excursions, etc, I did not see one other Canon... not one... not even a tiny PowerShot. All I saw were Nikon dSLRs, compacts, etc. I was the only one I saw shooting with my Canon (5D2 at the time).

So, what's this all mean? Who knows.

It's Ashton Kutcher... he's so dreamy...

 

[PackLight]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

Seriously? Their 24-70 is really great, way better than the mk1 was. Their 1.4 primes are about as good as the canons are (the canon 24 is excellent tough). Both 70-200 are great. Nikon recently came out with 3 new 1.8g lenses that are awesome and affordable... We're still not talking about the 14-24 or the ridiculous amount of old F mount lenses (e.g the 135 DC or the 55 micro) that are still awesome pieces of glass. Yes Canon also has great glass and some lenses that don't have good nikon counteparts, but nikon is not out of the game in that regard.

I'm really having a problem seeing where the nikon glass is unworthy outside of a fanboy perspective.

Thats funny, lets go back to talking about sensors. That way the Nikonians can particapate in the conversation.

Nice to see you're open minded and bringing proof to the conversation; you don't seem to be a mindless Canonian at all.

Just sayin': I'm the proud owner of a 7d and macro 100L wich I loove, I tried several canon primes and L zooms and most of them were great, but I also compared to equivalents in nikon and their glass has nothing to be ashamed of. It's really cute to see some of you guys acting like canon glass is like an affordable leica kind of thing, but it's not and nikon is serious competition in that matter (at least for what I tested MYSELF and any review site you'll find that compare both).

It doesn't take long to see where Nikon is lacking. Just consult the ISO charts. There are sites that give you the side by side. No Lecia comparison, just side by side Nikon vs Canon.
Case in point, 6D vs the D600. Comparable offerings right?
Compare the kit lenses that will be offered with both cameras. Nikon gives you a better sensor, Canon does it with better glass. In three years the bodies are old news, obsolete by gear head standards. At that point who is ahead, Canon or Nikon? It seems the sensor race goes back and forth. Good Glass is Good Glass no matter which sensor is better.

 

[Meh]

why should i, as photographer, care about how well nikon or canon sells?

even as a shop owner i don´t care about THEIR numbers.. i care about MY numbers.

as a customer i care what the camera offers me for my money... nothing more nothing less.

canon can outsell nikon 10x times.. that will still not make their current lineup better.

Very true. However, if the reason why Canon outsells Nikon is because they have an overall better offering and better value proposition then your point is fully consistent with the market sentiment. On the other hand, if Canon is outselling other brands because of simple popularity or a larger marketing budget then perhaps you could conclude they are not better despite the numbers. Markets can get it wrong, but typically the markets get it right over the long-term.

 

[dtaylor]

I, for one, can't take DxOMark seriously or trust any of their numbers when they...

* Rank $40,000 medium format digital backs lower than consumer APS-C DSLRs.

* Report physically unachievable values for dynamic range (i.e. >14 stops from a 14-bit ADC).

* Report values for dynamic range that I know to be false from both personal experience and testing. (They rank the 10D, 20D, and 7D about the same. The 7D is a good 2 stops better.)

For all the critics of DxOMark critics, I would like to point out that no less a professional and respected figure than Michael Reichmann stopped using DxOMark because of the obvious errors he observed in their results.

All that said...I wish Canon would lower their prices

 

[elflord]

So, DxOMark has said Nikon has had better sensors for years, and the sales data show that Canon has sold more dSLRs and lenses for those same years, and continues to do so, as of the most recent data available. The straightforward conclusion from the above is that while DxOMark's Scores have a huge impact on the number of inflammatory posts on Internet discussion boards, they have no meaningful impact on the real world aggregate buying decisions of consumers.

There's much more to a camera system than sensor technology.

The willingness and ability to stand behind ones product is very important if you're taking the long view. Orphaned products are almost worthless. No-one besides Nikon comes close and when their main advantage over Canon is a product that is sourced from a competitor, you can't give them too much credit for it.

Sensor technology does count for something, Canon need to be respectable but the evidence is that other factors are more important.

 

[lfg530]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

Seriously? Their 24-70 is really great, way better than the mk1 was. Their 1.4 primes are about as good as the canons are (the canon 24 is excellent tough). Both 70-200 are great. Nikon recently came out with 3 new 1.8g lenses that are awesome and affordable... We're still not talking about the 14-24 or the ridiculous amount of old F mount lenses (e.g the 135 DC or the 55 micro) that are still awesome pieces of glass. Yes Canon also has great glass and some lenses that don't have good nikon counteparts, but nikon is not out of the game in that regard.

I'm really having a problem seeing where the nikon glass is unworthy outside of a fanboy perspective.

Thats funny, lets go back to talking about sensors. That way the Nikonians can particapate in the conversation.

Nice to see you're open minded and bringing proof to the conversation; you don't seem to be a mindless Canonian at all.

Just sayin': I'm the proud owner of a 7d and macro 100L wich I loove, I tried several canon primes and L zooms and most of them were great, but I also compared to equivalents in nikon and their glass has nothing to be ashamed of. It's really cute to see some of you guys acting like canon glass is like an affordable leica kind of thing, but it's not and nikon is serious competition in that matter (at least for what I tested MYSELF and any review site you'll find that compare both).

It doesn't take long to see where Nikon is lacking. Just consult the ISO charts. There are sites that give you the side by side. No Lecia comparison, just side by side Nikon vs Canon.
Case in point, 6D vs the D600. Comparable offerings right?
Compare the kit lenses that will be offered with both cameras. Nikon gives you a better sensor, Canon does it with better glass. In three years the bodies are old news, obsolete by gear head standards. At that point who is ahead, Canon or Nikon? It seems the sensor race goes back and forth. Good Glass is Good Glass no matter which sensor is better.

Still dodging proofs and bringing empty assumptions. You're talking about ONE example and it's a kit lens, the 24-105 is great and probably (didn't test it myself) better than the 24-85 (wich is no slouch either at half the price....). But I tought we were talking about the lens lineup in general, not specific cases. And it's possible with both brands to have a really nice lens lineup if you invest in glass instead of bodies and both brands will offer great bodies when the times comes to upgrade...

 

[PackLight]

Neuro your post could have been sumed up in just a few words.

"Canon Glass Rules"

We can debate the sensors of the day and in a few months the conversation will change, but in 8 years we will still be debating the same glass. Maybe in those 8 years Nikon will come up with some glass worthy of a debate.

Seriously? Their 24-70 is really great, way better than the mk1 was. Their 1.4 primes are about as good as the canons are (the canon 24 is excellent tough). Both 70-200 are great. Nikon recently came out with 3 new 1.8g lenses that are awesome and affordable... We're still not talking about the 14-24 or the ridiculous amount of old F mount lenses (e.g the 135 DC or the 55 micro) that are still awesome pieces of glass. Yes Canon also has great glass and some lenses that don't have good nikon counteparts, but nikon is not out of the game in that regard.

I'm really having a problem seeing where the nikon glass is unworthy outside of a fanboy perspective.

Thats funny, lets go back to talking about sensors. That way the Nikonians can particapate in the conversation.

Nice to see you're open minded and bringing proof to the conversation; you don't seem to be a mindless Canonian at all.

Just sayin': I'm the proud owner of a 7d and macro 100L wich I loove, I tried several canon primes and L zooms and most of them were great, but I also compared to equivalents in nikon and their glass has nothing to be ashamed of. It's really cute to see some of you guys acting like canon glass is like an affordable leica kind of thing, but it's not and nikon is serious competition in that matter (at least for what I tested MYSELF and any review site you'll find that compare both).

It doesn't take long to see where Nikon is lacking. Just consult the ISO charts. There are sites that give you the side by side. No Lecia comparison, just side by side Nikon vs Canon.
Case in point, 6D vs the D600. Comparable offerings right?
Compare the kit lenses that will be offered with both cameras. Nikon gives you a better sensor, Canon does it with better glass. In three years the bodies are old news, obsolete by gear head standards. At that point who is ahead, Canon or Nikon? It seems the sensor race goes back and forth. Good Glass is Good Glass no matter which sensor is better.

Still dodging proofs and bringing empty assumptions. You're talking about ONE example and it's a kit lens, the 24-105 is great and probably (didn't test it myself) better than the 24-85 (wich is no slouch either at half the price....). But I tought we were talking about the lens lineup in general, not specific cases. And it's possible with both brands to have a really nice lens lineup if you invest in glass instead of bodies and both brands will offer great bodies when the times comes to upgrade...

See how empty this is.

Lets look at the top side of the line up. This will take it from talking about 1 lens to 10.
Hop over to TDP's ISO charts and compare any of the big supertele primes 200mm or up, version I or version II.
If you find a Nikon that outperforms any of its Canon's matches old version or new let me know. I haven't compared one yet that did.
A point to consider, the Nikon lenses ISO on the site were shot with the "superior" Nikon sensor.

Sure you can point out, the Canon lenses cost more. Canon knows there long lenses are better, they demand a premium and people pay it.

Which direction should we go next, I am sure there is a Nikon nugget somewhere. But, I know there are good lenses made for Nikon cameras. Just look up Zeiss.