DxOMark vs. Reality

[verysimplejason]

Hi Mike, but sorry we can't see anything... Can you re-post the links of your sources?  Thanks.

[neuroanatomist]

What I show you  is that there a world out side yours

I'm well aware of that, thanks.  I'm quoting sales figures from Canon and Nikon reflecting global sales, and aggregated market reports, and those show that Canon has a much larger market share than Nikon, that Canon has outsold Nikon year over year for the past 5 years, with the exception of 3Q12, and that means that Canon is the world leader in dSLR sales.  "Best and most beautiful" are subjective, billions of ¥ in revenue and number of units sold are not. 

Nikon SLR NR 1 selling in Japan...in  2011

Ok, so let's look at that.  There are two ways to interpret that statement, either Nikon was #1 in Japan in 2011, or Nikon had the #1 selling model in Japan in 2011.  The D3100 was the best selling dSLR in Japan in 2011.  The fact that the D3100 was the best selling dSLR in Japan in 2011 is misleading - the D3100 was the current model in it's segment for the entire year; in that segment, Canon's Kiss X4 was the current model at the beginning of the year, and was replaced by the Kiss X5 during 2011.  If you add the Kiss X4 and Kiss X5 sales together, that far exceeds D3100 sales.

Moreover, 6 of the top 20 cameras on that list are Canon models compared to 4 Nikon models on the list, and if you add up the sales by vendor, Canon dSLRs had 27% of the Japan market in 2011, compared to 22.5% for Nikon.  So...was Nikon really #1 in Japan in 2011?  No.

[neuroanatomist]

rather uninteresting who sells the most SLRs - right?

To shareholders, such things are very interesting...and because publicly-traded companies have an obligation to their shareholders (it's their primary obligation, in fact), such things are of paramount importance to those companies.  If it's important to them, it should be important to consumers of their products.

[PackLight]

So I guess DxO scores have a negative impact on CAJ share prices, and the dividend CAJ pays.
We should watch DxO's new scoring system for lenses closely, we may want to short the stock.

[sdsr]

Well it is hard to see Canon maintaining its sales numbers without continued fire sales of (for example) the 5D3 at a discount of over $500 of the MSRP. Or one might say that such fire sales are evidence that vendors are struggling to sell 5D3 stock at MSRP and that Canon sales numbers will suffer if they don't happen.

If "fire sales" matter, what would you have us infer from the current $700 discount on the D600 + kit lens via certain (all?) US (only US?) vendors?

[weekendshooter]

Well it is hard to see Canon maintaining its sales numbers without continued fire sales of (for example) the 5D3 at a discount of over $500 of the MSRP. Or one might say that such fire sales are evidence that vendors are struggling to sell 5D3 stock at MSRP and that Canon sales numbers will suffer if they don't happen.

If "fire sales" matter, what would you have us infer from the current $700 discount on the D600 + kit lens via certain (all?) US (only US?) vendors?

Probably that they have way too many of the terrible kit lens left over.

[thepancakeman]

There is some nitpicking about the way those measurements are aggregated but that's about it.

Subaru Legacy, Overall Score = 92
BMW 760Li xDrive, Overall Score = 84

Preposterous?  Well...the Overall Score is based on a weighted composite of two Use Case Scores, Winter Utility and Summer Utility. Those are based, respectively, on accurate and reliable Measurements of the ability of just the left rear wheel to push the car up a 20-degree incline, and the towing capacity.  But those details are just nitpicking. The Overall Scores clearly show that the Subaru is better.

 

Well, IMO it's actually worse than this.  Because they would publish overall scores, but those scores are only reflective of the engine performance.  There is no consideration for handling, traction, braking, interior features, etc.  I don't know anyone who would buy a car based on engine performance charts, yet somehow this makes sense in a camera? 

"My car has more ponies than yours and is therefore better even though I can't corner over 15mph, and it becomes airborn over 85mph."

[marinien]

http://www.techradar.com/news/photography-video-capture/cameras/nikon-d3100-tops-japans-best-seller-list-1051269
said nothing else

http://www.bjp-online.com/british-journal-of-photography/news/2040291/nikon-adds-digital-slr-range-d5100
said nothing else.

rather uninteresting who sells the most SLRs - right?
or , as someone else member told  you, Trabants sold most of all cars in the old East. Toyota is best selling cars today.
yard stick?
over and out regarding this subject

So Mikael, if someone posts an article solely about high ISO performance, your conclusion would be: DR at low ISO is "uninteresting", right? Right!

Your example with Toyota just doesn't work. Canon and Nikon are in the same market segments for DSRL, and they have , I dare to say, comparable distribution networks in most countries, so sale figure does mean something about user preference. While Toyota and, say, Mercedes are not in the same market segment, or Toyota and Peugeot do not have comparable global distribution networks, so their sale figures do not have the same meanings in user preference as Canon vs. Nikon.

Nikon SLR NR 1 selling in Japan...in  2011

Ok, so let's look at that.  There are two ways to interpret that statement, either Nikon was #1 in Japan in 2011, or Nikon had the #1 selling model in Japan in 2011.  The D3100 was the best selling dSLR in Japan in 2011.  The fact that the D3100 was the best selling dSLR in Japan in 2011 is misleading - the D3100 was the current model in it's segment for the entire year; in that segment, Canon's Kiss X4 was the current model at the beginning of the year, and was replaced by the Kiss X5 during 2011.  If you add the Kiss X4 and Kiss X5 sales together, that far exceeds D3100 sales.

Mikael, next time when you want to argue, try to learn from Neuro. Above is what I consider a good argument (throwing in some numbers would make it better)

Last advice: how about deciding once and for all on having a space or not following a comma in your signature? It's a detail, but it does reveal something ... 

[PackLight]

Mikael, next time when you want to argue, try to learn from Neuro. Above is what I consider a good argument (throwing in some numbers would make it better)

Last advice: how about deciding once and for all on having a space or not following a comma in your signature? It's a detail, but it does reveal something ... 

What would that reveal?
Not real sure on this one?

[marinien]

Mikael, next time when you want to argue, try to learn from Neuro. Above is what I consider a good argument (throwing in some numbers would make it better)

Last advice: how about deciding once and for all on having a space or not following a comma in your signature? It's a detail, but it does reveal something ... 

What would that reveal?
Not real sure on this one?

I am a researcher in applied mathematics. I work with a lot of data and stats. Being rigorous in every step is a key element. A misinterpretation here, a bad format there and the results may mean nothing. Our Mikael posted and repeated again and again his tests on this forum. But by looking at his signature, I'd not have a great confidence in his conclusions  .

[elflord]

Well, IMO it's actually worse than this.  Because they would publish overall scores, but those scores are only reflective of the engine performance.  There is no consideration for handling, traction, braking, interior features, etc.

If they tried to incorporate all of those features into their score, it would make things worse (and make the score more open to criticism).

It's better that they stick with a single thing -- benchmarking sensor performance.

I don't know anyone who would buy a car based on engine performance charts, yet somehow this makes sense in a camera? 

This is a straw man. No one has proposed that sensor performance is the only factor a buyer should consider.

You seem to be suggesting that no-one should measure engine performance (and therefore no-one should publish engine power/torque  output and curves)

[PackLight]

I am a researcher in applied mathematics. I work with a lot of data and stats. Being rigorous in every step is a key element. A misinterpretation here, a bad format there and the results may mean nothing. Our Mikael posted and repeated again and again his tests on this forum. But by looking at his signature, I'd not have a great confidence in his conclusions  .

I see,

This is why I do not trust doctors. Have you seen their signatures.

[thepancakeman]

Well, IMO it's actually worse than this.  Because they would publish overall scores, but those scores are only reflective of the engine performance.  There is no consideration for handling, traction, braking, interior features, etc.

If they tried to incorporate all of those features into their score, it would make things worse (and make the score more open to criticism).

It's better that they stick with a single thing -- benchmarking sensor performance.

I don't know anyone who would buy a car based on engine performance charts, yet somehow this makes sense in a camera? 

This is a straw man. No one has proposed that sensor performance is the only factor a buyer should consider.

You seem to be suggesting that no-one should measure engine performance (and therefore no-one should publish engine power/torque  output and curves)

Not saying no one should measure, but that in the camera world these measures are blown way out of proportion for what they should be.  And if they're truly just measuring sensors, then shouldn't be the rating be identified to the sensor, and not the camera??  Pretty sure multiple camera's use the same sensor, but you don't find the sensor score, you find the camera score.

My gripe is not in their testing, it's in the presentation and marketing layer.  For example, they say "The Overall Sensor Score is based on all characteristics of a camera sensor, independent of the camera lens..."  Umm, it's independent of a whole lot more than just the lens.  My point is there is a certain amount of deceptiveness in their marketing (for lack of a better word.)  They present scores in "Portrait" and "Landscape" and "Sports" even though their criteria do not apply to many critical elements of those photographic activities.  Yes, they explain what they mean by those terms, but to the average consumer the name is going to mean a whole lot more than the actual test criteria.  I know for my sports photography, the sensor itself is the least of my concerns and yet they present low-light ISO as a singular element for their "sports" score.  Again, not an outright lie, but it sure seems intentional to mislead people that aren't willing or interested in reading the fine print.

[elflord]

Not saying no one should measure, but that in the camera world these measures are blown way out of proportion for what they should be.  And if they're truly just measuring sensors, then shouldn't be the rating be identified to the sensor, and not the camera??  Pretty sure multiple camera's use the same sensor, but you don't find the sensor score, you find the camera score.

Under the "cameras" tab on their page, I see "camera sensor database", "camera sensor ratings" and "compare camera sensors".

I don't think they're making it a secret that they are benchmarking sensors.

They present scores in "Portrait" and "Landscape" and "Sports" even though their criteria do not apply to many critical elements of those photographic activities.  Yes, they explain what they mean by those terms, but to the average consumer the name is going to mean a whole lot more than the actual test criteria.

What descriptions would be more appropriate in your opinion ?

I know for my sports photography, the sensor itself is the least of my concerns and yet they present low-light ISO as a singular element for their "sports" score.  Again, not an outright lie, but it sure seems intentional to mislead people that aren't willing or interested in reading the fine print.

How precisely are they trying to "mislead" people ? This is a step beyond simply criticising their choice of naming, here you appear to be accusing them of choosing the names in such a way as to favour their business.

Well, please do substantiate this accusation a little. For example, how do they benefit from calling their high ISO use case "Sports" instead of "event photography" or "wedding photography" ?

Also, it's clear that you don't like their choice of names. It's not clear (a) what you would choose instead, or even (b) if your choices, with the benefit of a few years of hindsight (e.g. after everyone has had their chance to publically critique DxO's), are any better.

[tnargs]

This is a straw man. No one has proposed that sensor performance is the only factor a buyer should consider.

You are joking of course. The number of posts and comments on the internet basically saying "I have to jump ship from Canon to Nikon NOW, and so do you, because the Nikon (Sony) sensor has a higher DR" is fast approaching infinity.