June 19, 2013, 08:12:56 AM

Author Topic: DxOMark vs. Reality  (Read 60160 times)

neuroanatomist

  • 1D X
  • *******
  • Posts: 8929
    • View Profile
Re: DxOMark vs. Reality
« Reply #195 on: December 19, 2012, 12:58:09 PM »
And frankly, I don't think that non-obsesed consumers care about making truly personally informed decisions.

Not sure I agree with that...I think most consumers make personally informed decisions - it's just that since it's personally informed, and what constitutes adequate information for that is different for different people.  But that idea is consistent with what I'm saying - DOMark data are not a factor in the buying decisions of most consumers when it omes to a dSLR purchase.
EOS 1D X, EOS 7D, and lots of lenses
_____________________________
Flickr | TDP Profile/Gear List

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #195 on: December 19, 2012, 12:58:09 PM »

thepancakeman

  • 7D
  • *****
  • Posts: 348
  • There are absolutely no absolutes.
    • View Profile
Re: DxOMark vs. Reality
« Reply #196 on: December 19, 2012, 02:09:56 PM »
And frankly, I don't think that non-obsesed consumers care about making truly personally informed decisions.

Not sure I agree with that...I think most consumers make personally informed decisions - it's just that since it's personally informed, and what constitutes adequate information for that is different for different people.  But that idea is consistent with what I'm saying - DOMark data are not a factor in the buying decisions of most consumers when it omes to a dSLR purchase.

Ah, but DXOMark doesn't just do dSLR's.   ;)
7D, 40D, XTi, 24-105 4.0L, 70-200 2.8 IS L, 85 1.8, 50 1.8, 100 2.8 macro, Tamron 17-50mm f/2.8, and an S100 that is more complicated than any of them.

Maui5150

  • EOS M
  • ****
  • Posts: 312
    • View Profile
Re: DxOMark vs. Reality
« Reply #197 on: December 19, 2012, 03:02:23 PM »

your answer is  clever like  as -  all Canons gear is worthless
Please point why and  how the DXO sensor measurement is faulty.

Which is a better overall camera?  The D4 or the D600?

Which will take the best picture?  The D4 or the D600?

Which has a better Dx0Mark Score? 

The answer to the first 2 is the D4, and any HONEST person would agree

Yet the Dx0 Mark for the D600 is over 5% higher? 

Sort of PROVES my point the the Dx0Mark score is MEANINGLESS

Or perhaps you are trying to say that Nikon throws poor sensors in their flagship camera (meaning that the D4 is SUPER over priced)

elflord

  • 5D Mark III
  • ******
  • Posts: 701
    • View Profile
Re: DxOMark vs. Reality
« Reply #198 on: December 19, 2012, 06:55:21 PM »
look up my recent hisotory on posting about DXO...i just don't believe they have their "act" together.

I did. It looks like you've gone on a fishing expedition to find some kind of perceived wrongdoing.  I suppose this is consistent with casting aspersions and not being forthright about what your true objections are. One recent post was a complaint about typos, the other about their lens benchmarking. None of this has any relevance to or casts any doubt on their status as the industry leader in sensor benchmarking.

It seems to me that you didn't like it that Canon cameras did not score well on their benchmarks and decided to smear them on that basis.

elflord

  • 5D Mark III
  • ******
  • Posts: 701
    • View Profile
Re: DxOMark vs. Reality
« Reply #199 on: December 19, 2012, 06:58:57 PM »
Umm, how about "Color depth", "Dynamic Range", and "Low-light ISO"?

But they use those labels as well.

Quote
I guess I am just of the mind that their scores (not their measurements) seem to have a bent towards Nikon.  I don't know this for a fact or why they would do such; maybe they own stock.  But the net result (pun intended) is that an "I'm looking for a good sports camera" type of query returns values that are actually only good "low-light ISO" cameras and completely suck for sports.  Why would they perpetrate this unless they are either lazy (as one person suggested), have a hidden agenda, or just kinda clueless (which I doubt, because they seem pretty bright.)

Well, they don't benchmark AF systems.

elflord

  • 5D Mark III
  • ******
  • Posts: 701
    • View Profile
Re: DxOMark vs. Reality
« Reply #200 on: December 19, 2012, 07:03:33 PM »
How I see DXO is they are a company that manipulates their findings to arrive at an arbitrary score they established, this is their reputation as I see it, and I hear it from others as well. You may have a different view of their reputation as many do. Again when a company starts receiving a bad reputation it calls in to question their character.

How are they fudging the data ? Do you have any evidence to support such "manipulation" besides hearsay ?

As to their reputation -- there are some who for whatever reason have a very strong affinity for certain camera brands. Let's call them "fans". All major brands have "fans". The fans due to their enthusiasm are often on internet forums. DxO does have a bad reputation among "fans" of brands that get low scores on their benchmark. This says more about said "fans" than anything else. They are overwhelmingly seen as the leader in sensor benchmarking.

Even the DxO sceptics here acknowledge and take seriously their measurements -- most of the complaints are with the methods used to aggregate those measurements.

elflord

  • 5D Mark III
  • ******
  • Posts: 701
    • View Profile
Re: DxOMark vs. Reality
« Reply #201 on: December 19, 2012, 07:15:33 PM »

your answer is  clever like  as -  all Canons gear is worthless
Please point why and  how the DXO sensor measurement is faulty.

Which is a better overall camera?  The D4 or the D600?

Which will take the best picture?  The D4 or the D600?

DxO do not attempt to measure which is "a better overall camera" (whatever that means though I think most would agree that it's the D4) nor do they measure "which will take the best picture" (what does "the best picture" mean ?)

They benchmark sensors.

Now, your answer to the question is ?

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #201 on: December 19, 2012, 07:15:33 PM »

verysimplejason

  • 1D Mark IV
  • ******
  • Posts: 819
    • View Profile
    • My Flickr Account
Re: DxOMark vs. Reality
« Reply #202 on: December 19, 2012, 08:04:42 PM »
If  we  shall discuss a specific topic (DXO measurements vs reality) let's us do that without involving other parameters.
it is very difficult to follow  when people are involving   other features in the cameras who has nothing to do with DXO measurements.

But REALITY does involve other parameters... Don't tell me you are taking pictures with just your sensor?  That is why this is titled DXO vs Reality.  Reality is what it is.  You need to consider everything and not only the sensor.  It's elementary my dear Watson...
Please visit my flickr.
http://www.flickr.com/photos/10307637@N08/
Equipment: Canon 500D, 28mm F1.8 USM, 50mm F1.8 II, 100mm F2.8 USM, 55-250mm IS

neuroanatomist

  • 1D X
  • *******
  • Posts: 8929
    • View Profile
Re: DxOMark vs. Reality
« Reply #203 on: December 19, 2012, 08:21:29 PM »
If  we  shall discuss a specific topic (DXO measurements vs reality) let's us do that without involving other parameters.
it is very difficult to follow  when people are involving   other features in the cameras who has nothing to do with DXO measurements.


But REALITY does involve other parameters... Don't tell me you are taking pictures with just your sensor?  That is why this is titled DXO vs Reality.  Reality is what it is.  You need to consider everything and not only the sensor.  It's elementary my dear Watson...


Nope, DXO mark vs reality, you can read  at DXO what the marks includes  and which  sensor parameters  are measured. http://www.dxomark.com/index.php/About/Sensor-scores


Yep, reality is the whole point.  If DxOMark = Reality, and their Scores were all that mattered, no one would be buying Canon dSLRs, only Nikon dSLRs.  Has that been happening?  Or does Canon still have a greater market share?
EOS 1D X, EOS 7D, and lots of lenses
_____________________________
Flickr | TDP Profile/Gear List

verysimplejason

  • 1D Mark IV
  • ******
  • Posts: 819
    • View Profile
    • My Flickr Account
Re: DxOMark vs. Reality
« Reply #204 on: December 19, 2012, 08:32:25 PM »
If  we  shall discuss a specific topic (DXO measurements vs reality) let's us do that without involving other parameters.
it is very difficult to follow  when people are involving   other features in the cameras who has nothing to do with DXO measurements.


But REALITY does involve other parameters... Don't tell me you are taking pictures with just your sensor?  That is why this is titled DXO vs Reality.  Reality is what it is.  You need to consider everything and not only the sensor.  It's elementary my dear Watson...


Nope, DXO mark vs reality, you can read  at DXO what the marks includes  and which  sensor parameters  are measured. http://www.dxomark.com/index.php/About/Sensor-scores

To help photographers rank and compare photographic equipment, DxOMark provides four scores showing camera sensor image quality performance:  nada more.


DXO scores sensors while in REALITY there's more to a camera than a sensor.  Is it that hard to understand?  That's the problem with DXO and what's almost everybody is pointing out.  DXO seems content on grading the sensor but not the whole camera body.  That's why this is titled DXOMARK vs REALITY.
Please visit my flickr.
http://www.flickr.com/photos/10307637@N08/
Equipment: Canon 500D, 28mm F1.8 USM, 50mm F1.8 II, 100mm F2.8 USM, 55-250mm IS

TheSuede

  • PowerShot G1 X
  • ***
  • Posts: 52
    • View Profile
Re: DxOMark vs. Reality
« Reply #205 on: December 19, 2012, 08:44:35 PM »
The DxO "scores" are in no way to be mistaken for "reality". They are what they are, an end sum of an arbitrarily weighed set of measurement points that in their constitution may (or may not!) in some way be representative of how you as an individual use your camera.

As stated so many times before, you cannot put a "score" on a camera as a complete system for any sort of user average, you have to specify a quite tight definition of the score scope - what it does include and what it doesn't. And it does get tricky when you try to include more than one parameter in a "score".

You may (in my opinion...) for instance put a reasonably real-world relevant score on "low light noise performance". This score will then preferably accurately represent how much noise a camera will show in a shot given an average exposure of exactly "X" lumen seconds per mm2. The measurement "noise per lm*s/mm2" is just a strict definition of what a photographer would call "I want 1/200s at F2.8 in this light".

-But it (the score) will not say anything at all about how well the camera can AF in that same situation. Neither will it say anything about the viewfinder, the fps rate or the color accuracy. The score will be "low light NOISE performance", not "low light performance".
...

But the individual measurement scales and results in themselves that DxO use to build their total "score" are almost beyond reproach. You could possibly fault their presentation mode in some of the results, but not the results in themselves.
The few individuals and organizations that can replicate the accuracy and scientific stringency of their tests all arrive at the same conclusion - there's not much to be said about it except for the fact that DxO are almost 100% repeatable and accurate. The numbers just fit, and anyone measuring the same thing on the same cameras will arrive at the same result. And anyone comparing two cameras out in the real world can - if they adhere to reality in stead of maker-defined bogus ISO values and so on - also visualize exactly what that number means in an image.

So it's up to the reader of the results to make use of the information in a way that he or she can relate to, and here's where most people fall flat on their faces. Unfortunately DxO trip themselves up by trying to apply an overall "score" on a camera, but as far as anyone in the publishing business can tell - you NEED a promise of an end score or other simple graphical or single-digit grading system to get 95% of the readers to even bother starting to read.

If the average readers realizes that they actually might have to think a bit for themselves to get something out of something, they quite often just bugger off somewhere else before even starting to assimilate any real information.

TheSuede

  • PowerShot G1 X
  • ***
  • Posts: 52
    • View Profile
Re: DxOMark vs. Reality
« Reply #206 on: December 19, 2012, 08:53:38 PM »
Isn't trying to use the argument:
-"They're wrong because Canon actually sell quite a lot of cameras"

kind of like saying
:"Everything Apple makes right now has to be crap, they're constantly dropping in market share"
???

Neither of those arguments are worth paying any attention to, since there's more to sales and market shares than measurable product quality.

neuroanatomist

  • 1D X
  • *******
  • Posts: 8929
    • View Profile
Re: DxOMark vs. Reality
« Reply #207 on: December 19, 2012, 09:12:48 PM »
Isn't trying to use the argument:
-"They're wrong because Canon actually sell quite a lot of cameras"

Actually, the argument is, "They're irrelevant (or at least, not very relevant) because Canon sells quite a lot more cameras than their competitors who score much higher in DxOMark's rankings."

The DxO "scores" are in no way to be mistaken for "reality". They are what they are, an end sum of an arbitrarily weighed set of measurement points that in their constitution may (or may not!) in some way be representative of how you as an individual use your camera.

I could not agree more.  This was what I was driving at, initially.  At the time I originated this thread, there was a plethora of posts/threads proclaiming the superiority of the D800 (and other Nikon bodies) as cameras because of the DxOMark scores, and nearly as many posts/threads suggesting that Canon was doomed as a corporate entity because of their subpar sensors.  The former claim is clearly unsupported by DxOMark, and the latter claim is equally untrue.

Thanks, TheSuede, for your cogent post.   :)
EOS 1D X, EOS 7D, and lots of lenses
_____________________________
Flickr | TDP Profile/Gear List

thepancakeman

  • 7D
  • *****
  • Posts: 348
  • There are absolutely no absolutes.
    • View Profile
Re: DxOMark vs. Reality
« Reply #208 on: December 19, 2012, 10:10:04 PM »
If  we  shall discuss a specific topic (DXO measurements vs reality) let's us do that without involving other parameters.
it is very difficult to follow  when people are involving   other features in the cameras who has nothing to do with DXO measurements.


But REALITY does involve other parameters... Don't tell me you are taking pictures with just your sensor?  That is why this is titled DXO vs Reality.  Reality is what it is.  You need to consider everything and not only the sensor.  It's elementary my dear Watson...


Nope, DXO mark vs reality, you can read  at DXO what the marks includes  and which  sensor parameters  are measured. http://www.dxomark.com/index.php/About/Sensor-scores

To help photographers rank and compare photographic equipment, DxOMark provides four scores showing camera sensor image quality performance:  nada more.


The reality is that low-light ISO does not = a good sports camera, but that is how it's being portrayed.
7D, 40D, XTi, 24-105 4.0L, 70-200 2.8 IS L, 85 1.8, 50 1.8, 100 2.8 macro, Tamron 17-50mm f/2.8, and an S100 that is more complicated than any of them.

elflord

  • 5D Mark III
  • ******
  • Posts: 701
    • View Profile
Re: DxOMark vs. Reality
« Reply #209 on: December 19, 2012, 10:14:53 PM »
Yep, reality is the whole point.  If DxOMark = Reality, and their Scores were all that mattered, no one would be buying Canon dSLRs, only Nikon dSLRs.  Has that been happening?  Or does Canon still have a greater market share?

It is not as the thread title suggests, a "competitor" or an "alternative" to reality but rather a subset thereof.

BTW, if the scores do not substantially affect sales, why all the concern about them "misleading" the "consumer" ?

canon rumors FORUM

Re: DxOMark vs. Reality
« Reply #209 on: December 19, 2012, 10:14:53 PM »