November 23, 2014, 03:11:33 PM

Author Topic: DxOMark vs. Reality  (Read 82671 times)

weekendshooter

  • Guest
Re: DxOMark vs. Reality
« Reply #150 on: December 17, 2012, 03:48:17 PM »

Well it is hard to see Canon maintaining its sales numbers without continued fire sales of (for example) the 5D3 at a discount of over $500 of the MSRP. Or one might say that such fire sales are evidence that vendors are struggling to sell 5D3 stock at MSRP and that Canon sales numbers will suffer if they don't happen.

If "fire sales" matter, what would you have us infer from the current $700 discount on the D600 + kit lens via certain (all?) US (only US?) vendors?

Probably that they have way too many of the terrible kit lens left over.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #150 on: December 17, 2012, 03:48:17 PM »

thepancakeman

  • Canon 7D MK II
  • *****
  • Posts: 457
  • If at first you don't succeed, don't try skydiving
    • View Profile
Re: DxOMark vs. Reality
« Reply #151 on: December 17, 2012, 04:20:56 PM »
There is some nitpicking about the way those measurements are aggregated but that's about it.

Subaru Legacy, Overall Score = 92
BMW 760Li xDrive, Overall Score = 84

Preposterous?  Well...the Overall Score is based on a weighted composite of two Use Case Scores, Winter Utility and Summer Utility. Those are based, respectively, on accurate and reliable Measurements of the ability of just the left rear wheel to push the car up a 20-degree incline, and the towing capacity.  But those details are just nitpicking. The Overall Scores clearly show that the Subaru is better.

 ::)

Well, IMO it's actually worse than this.  Because they would publish overall scores, but those scores are only reflective of the engine performance.  There is no consideration for handling, traction, braking, interior features, etc.  I don't know anyone who would buy a car based on engine performance charts, yet somehow this makes sense in a camera? 

"My car has more ponies than yours and is therefore better even though I can't corner over 15mph, and it becomes airborn over 85mph."

marinien

  • Rebel SL1
  • ***
  • Posts: 80
    • View Profile
Re: DxOMark vs. Reality
« Reply #152 on: December 17, 2012, 06:57:57 PM »
http://www.techradar.com/news/photography-video-capture/cameras/nikon-d3100-tops-japans-best-seller-list-1051269
said nothing else

http://www.bjp-online.com/british-journal-of-photography/news/2040291/nikon-adds-digital-slr-range-d5100
said nothing else.

rather uninteresting who sells the most SLRs - right?
or , as someone else member told  you, Trabants sold most of all cars in the old East. Toyota is best selling cars today.
yard stick?
over and out regarding this subject

So Mikael, if someone posts an article solely about high ISO performance, your conclusion would be: DR at low ISO is "uninteresting", right? Right!

Your example with Toyota just doesn't work. Canon and Nikon are in the same market segments for DSRL, and they have , I dare to say, comparable distribution networks in most countries, so sale figure does mean something about user preference. While Toyota and, say, Mercedes are not in the same market segment, or Toyota and Peugeot do not have comparable global distribution networks, so their sale figures do not have the same meanings in user preference as Canon vs. Nikon.

Nikon SLR NR 1 selling in Japan...in  2011
Ok, so let's look at that.  There are two ways to interpret that statement, either Nikon was #1 in Japan in 2011, or Nikon had the #1 selling model in Japan in 2011.  The D3100 was the best selling dSLR in Japan in 2011.  The fact that the D3100 was the best selling dSLR in Japan in 2011 is misleading - the D3100 was the current model in it's segment for the entire year; in that segment, Canon's Kiss X4 was the current model at the beginning of the year, and was replaced by the Kiss X5 during 2011.  If you add the Kiss X4 and Kiss X5 sales together, that far exceeds D3100 sales.

Mikael, next time when you want to argue, try to learn from Neuro. Above is what I consider a good argument (throwing in some numbers would make it better) ;)

Last advice: how about deciding once and for all on having a space or not following a comma in your signature? It's a detail, but it does reveal something ...  ::)
« Last Edit: December 17, 2012, 07:11:10 PM by marinien »
7D | EF-S 17-55 | EF 100mm f/2.8 L IS Macro | 580EX II | Benro C3780T + Markins M20

PackLight

  • Guest
Re: DxOMark vs. Reality
« Reply #153 on: December 17, 2012, 07:21:27 PM »

Mikael, next time when you want to argue, try to learn from Neuro. Above is what I consider a good argument (throwing in some numbers would make it better) ;)

Last advice: how about deciding once and for all on having a space or not following a comma in your signature? It's a detail, but it does reveal something ...  ::)

What would that reveal?
Not real sure on this one?


marinien

  • Rebel SL1
  • ***
  • Posts: 80
    • View Profile
Re: DxOMark vs. Reality
« Reply #154 on: December 17, 2012, 07:33:22 PM »

Mikael, next time when you want to argue, try to learn from Neuro. Above is what I consider a good argument (throwing in some numbers would make it better) ;)

Last advice: how about deciding once and for all on having a space or not following a comma in your signature? It's a detail, but it does reveal something ...  ::)

What would that reveal?
Not real sure on this one?

I am a researcher in applied mathematics. I work with a lot of data and stats. Being rigorous in every step is a key element. A misinterpretation here, a bad format there and the results may mean nothing. Our Mikael posted and repeated again and again his tests on this forum. But by looking at his signature, I'd not have a great confidence in his conclusions  ;).
7D | EF-S 17-55 | EF 100mm f/2.8 L IS Macro | 580EX II | Benro C3780T + Markins M20

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #155 on: December 17, 2012, 07:35:52 PM »
Well, IMO it's actually worse than this.  Because they would publish overall scores, but those scores are only reflective of the engine performance.  There is no consideration for handling, traction, braking, interior features, etc.

If they tried to incorporate all of those features into their score, it would make things worse (and make the score more open to criticism).

It's better that they stick with a single thing -- benchmarking sensor performance.

Quote
I don't know anyone who would buy a car based on engine performance charts, yet somehow this makes sense in a camera? 

This is a straw man. No one has proposed that sensor performance is the only factor a buyer should consider.

You seem to be suggesting that no-one should measure engine performance (and therefore no-one should publish engine power/torque  output and curves)

PackLight

  • Guest
Re: DxOMark vs. Reality
« Reply #156 on: December 17, 2012, 07:38:39 PM »

I am a researcher in applied mathematics. I work with a lot of data and stats. Being rigorous in every step is a key element. A misinterpretation here, a bad format there and the results may mean nothing. Our Mikael posted and repeated again and again his tests on this forum. But by looking at his signature, I'd not have a great confidence in his conclusions  ;).

I see,

This is why I do not trust doctors. Have you seen their signatures.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #156 on: December 17, 2012, 07:38:39 PM »

thepancakeman

  • Canon 7D MK II
  • *****
  • Posts: 457
  • If at first you don't succeed, don't try skydiving
    • View Profile
Re: DxOMark vs. Reality
« Reply #157 on: December 18, 2012, 06:25:14 PM »
Well, IMO it's actually worse than this.  Because they would publish overall scores, but those scores are only reflective of the engine performance.  There is no consideration for handling, traction, braking, interior features, etc.

If they tried to incorporate all of those features into their score, it would make things worse (and make the score more open to criticism).

It's better that they stick with a single thing -- benchmarking sensor performance.

Quote
I don't know anyone who would buy a car based on engine performance charts, yet somehow this makes sense in a camera? 

This is a straw man. No one has proposed that sensor performance is the only factor a buyer should consider.

You seem to be suggesting that no-one should measure engine performance (and therefore no-one should publish engine power/torque  output and curves)

Not saying no one should measure, but that in the camera world these measures are blown way out of proportion for what they should be.  And if they're truly just measuring sensors, then shouldn't be the rating be identified to the sensor, and not the camera??  Pretty sure multiple camera's use the same sensor, but you don't find the sensor score, you find the camera score.

My gripe is not in their testing, it's in the presentation and marketing layer.  For example, they say "The Overall Sensor Score is based on all characteristics of a camera sensor, independent of the camera lens..."  Umm, it's independent of a whole lot more than just the lens.  My point is there is a certain amount of deceptiveness in their marketing (for lack of a better word.)  They present scores in "Portrait" and "Landscape" and "Sports" even though their criteria do not apply to many critical elements of those photographic activities.  Yes, they explain what they mean by those terms, but to the average consumer the name is going to mean a whole lot more than the actual test criteria.  I know for my sports photography, the sensor itself is the least of my concerns and yet they present low-light ISO as a singular element for their "sports" score.  Again, not an outright lie, but it sure seems intentional to mislead people that aren't willing or interested in reading the fine print.

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #158 on: December 18, 2012, 07:01:27 PM »
Not saying no one should measure, but that in the camera world these measures are blown way out of proportion for what they should be.  And if they're truly just measuring sensors, then shouldn't be the rating be identified to the sensor, and not the camera??  Pretty sure multiple camera's use the same sensor, but you don't find the sensor score, you find the camera score.

Under the "cameras" tab on their page, I see "camera sensor database", "camera sensor ratings" and "compare camera sensors".

I don't think they're making it a secret that they are benchmarking sensors.

Quote
They present scores in "Portrait" and "Landscape" and "Sports" even though their criteria do not apply to many critical elements of those photographic activities.  Yes, they explain what they mean by those terms, but to the average consumer the name is going to mean a whole lot more than the actual test criteria.

What descriptions would be more appropriate in your opinion ?

Quote
I know for my sports photography, the sensor itself is the least of my concerns and yet they present low-light ISO as a singular element for their "sports" score.  Again, not an outright lie, but it sure seems intentional to mislead people that aren't willing or interested in reading the fine print.

How precisely are they trying to "mislead" people ? This is a step beyond simply criticising their choice of naming, here you appear to be accusing them of choosing the names in such a way as to favour their business.

Well, please do substantiate this accusation a little. For example, how do they benefit from calling their high ISO use case "Sports" instead of "event photography" or "wedding photography" ?

Also, it's clear that you don't like their choice of names. It's not clear (a) what you would choose instead, or even (b) if your choices, with the benefit of a few years of hindsight (e.g. after everyone has had their chance to publically critique DxO's), are any better.


tnargs

  • Rebel T5i
  • ****
  • Posts: 138
    • View Profile
Re: DxOMark vs. Reality
« Reply #159 on: December 18, 2012, 08:04:38 PM »
This is a straw man. No one has proposed that sensor performance is the only factor a buyer should consider.

You are joking of course. The number of posts and comments on the internet basically saying "I have to jump ship from Canon to Nikon NOW, and so do you, because the Nikon (Sony) sensor has a higher DR" is fast approaching infinity.   8)

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #160 on: December 18, 2012, 09:43:08 PM »
I do not pretend to be a technical expert in camera gear....BUT....it's interesting to notice that the nikon d600, WITH THE 2ND HIGHEST RATED SENSOR SCORE EVER BY DXO, (only behind the d800) WITH A SCORE OF 94, is now being heavily discounted by nikon, not by retailers, but by nikon....just months after it's launch.  (a couple months ago you could have had the camera for $2k, now you can get the camera and the nikon 24-85 ($600) for $2k.   

Yes, and these have  higher than the D4, which costs how much ?

People buy cameras, not sensors.

Quote
it's just amazing to me that people believe that somehow a $2000 nikon entry level full frame camera somehow has a sensor(according to the experts at dxo...lol)  that is SIGNIFICANTLY HIGHER than the $6800 canon flagship 1dx.

all around the world people understand this phenomena...it's called $$$$$ under the table.

So should DxO instead award higher sensor benchmark scores to cameras that cost more, so as not to offend snobs who own or worship expensive equipment ?

It's not surprising at all that the D600 gets a similar score to the D800. The sensor is the same size, same vintage, and made by the same manufacturer. Why would it not have similar performance characteristics ?

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: DxOMark vs. Reality
« Reply #161 on: December 18, 2012, 10:12:15 PM »
hey ELF...i already implied why....read again the last part of my post.

as I've already stated on this forum...it's of my opinion, that DXO is a group of lazy and unorganized....enough said.
I do understand that you're casting aspersions as opposed to stating your views in a more forthright manner (which might create an expectation that you substantiate your allegations).

What you do not understand is that I'm asking you to put up or shut up. If you have some insight or knowledge that supports your view, please do not be shy -- share it with us. If you don't have any such insight or knowledge, it does raise some questions as to why you are prone to make unsubstantiated and unprovoked attacks on the reputation and character of DxO.
« Last Edit: December 18, 2012, 10:13:59 PM by elflord »

Northstar

  • 1D X
  • *******
  • Posts: 1503
    • View Profile
Re: DxOMark vs. Reality
« Reply #162 on: December 18, 2012, 11:13:43 PM »
hey ELF...i already implied why....read again the last part of my post.

as I've already stated on this forum...it's of my opinion, that DXO is a group of lazy and unorganized....enough said.
I do understand that you're casting aspersions as opposed to stating your views in a more forthright manner (which might create an expectation that you substantiate your allegations).

What you do not understand is that I'm asking you to put up or shut up. If you have some insight or knowledge that supports your view, please do not be shy -- share it with us. If you don't have any such insight or knowledge, it does raise some questions as to why you are prone to make unsubstantiated and unprovoked attacks on the reputation and character of DxO.


look up my recent hisotory on posting about DXO...i just don't believe they have their "act" together.
Sport Shooter

1dX and 5d3... 24-70 2.8ii, 70-200 2.8ii, 1.4xiii and 2xiii, 85, 40mm, 300 2.8L IS....430ex

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #162 on: December 18, 2012, 11:13:43 PM »

neuroanatomist

  • CR GEEK
  • **********
  • Posts: 14929
    • View Profile
Re: DxOMark vs. Reality
« Reply #163 on: December 19, 2012, 08:50:53 AM »
What you do not understand is that I'm asking you to put up or shut up.

That has been asked of others on this thread, with no 'putting up' evident. 

you have the right to think what you want.
we are some who do not share your opinion, you must also accept that

+1  Without supporting evidence, statements are merely opinion, and as you say, we are all entitled to our own with no expectation that others will agree.

Just to remind people of the central thesis of this thread - it's not that DxOMark scores are 'cooked' or 'false', nor even that the scores are biased in favor one brand over another.  I've seen no evidence of the former (quite the opposite), and while the latter may be true, there's no evidence it's driven by anything other than logic - we generally try to shoot at the lowest ISO possible, so there's some logic to biasing the scores to base ISO. 

Rather, the central point is that DxOMark's scores seem to have had no meaningful impact on the real world aggregate buying decisions of consumers, as demonstrated by Canon's and Nikon's own financial reports detailing their global sales figures.  Put succinctly as a statement comprising fact, Nikon dSLRs have beaten Canon's in DxOMark scores for several years, and Canon's dSLRs have outsold Nikon's for those same several years. 
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

PackLight

  • Guest
Re: DxOMark vs. Reality
« Reply #164 on: December 19, 2012, 08:58:33 AM »
make unsubstantiated and unprovoked attacks on the reputation and character of DxO.

Interesting statement;

I think anytime a company presents information to the public, if it is misleading in nature it calls in to question the character of the company.

The tests DXO performs on sensors, if the data is presented for what it is I would not question their presentation or motive. When they take the data and manipulate it to arrive at a score they created with a description that they perceive or make us want to believe is correct, I have to question why they would do this.

How I see DXO is they are a company that manipulates their findings to arrive at an arbitrary score they established, this is their reputation as I see it, and I hear it from others as well. You may have a different view of their reputation as many do. Again when a company starts receiving a bad reputation it calls in to question their character.

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #164 on: December 19, 2012, 08:58:33 AM »