DxOMark vs. Reality

[PackLight]

How I see DXO is they are a company that manipulates their findings to arrive at an arbitrary score they established, this is their reputation as I see it, and I hear it from others as well. You may have a different view of their reputation as many do. Again when a company starts receiving a bad reputation it calls in to question their character.

How are they fudging the data ? Do you have any evidence to support such "manipulation" besides hearsay ?

As to their reputation -- there are some who for whatever reason have a very strong affinity for certain camera brands. Let's call them "fans". All major brands have "fans". The fans due to their enthusiasm are often on internet forums. DxO does have a bad reputation among "fans" of brands that get low scores on their benchmark. This says more about said "fans" than anything else. They are overwhelmingly seen as the leader in sensor benchmarking.

Even the DxO sceptics here acknowledge and take seriously their measurements -- most of the complaints are with the methods used to aggregate those measurements.

Fudging? It wasn't my word.

I have all the evidence I need to may an informed decision. Thing is, I figured this out for myself several years back before I read a single thread about how bad DxO scoring is. I did it by reading how they do the testing and how they arrive at there scores. DxO provided this information to determine the flaw to their madness. It was easy with just a little intelligent reasoning to realize the overall scores are flawed. Did I need to consult someone other than DxO to make this determination?

Do you not read the posts you are responding to or do you just go in making accusations?
I am not sure, I say there method of testing is ok but the method they use to "aggregate those measurements" (your words) is flawed. And you go off on a different tangent.

Why do you feel the need to defend DxO. Did you buy your Nikon based on these scores and are afraid they may be make believe now?

[elflord]

Do you not read the posts you are responding to or do you just go in making accusations?
I am not sure, I say there method of testing is ok but the method they use to "aggregate those measurements" (your words) is flawed. And you go off on a different tangent.

"Manipulating the data" with the intent to mislead sounds like fudging to me, though it wasn't your word. As far as I can tell they are simply summarizing / aggregating the data and it just happens that with the current sensors, the way they do this doesn't work out well for Canon.

Why do you feel the need to defend DxO. Did you buy your Nikon based on these scores and are afraid they may be make believe now?

I don't own a Nikon. I own a 5D Mk II and a Panasonic GF2. Before that I owned a Rebel. I am not a "fan" of either Canon, Panasonic or any other brand.

I defend them because in my opinion they do an excellent job at what they do, and most of the attacks are not well founded (the vast majority of criticism comes from camera "fans" who are unhappy that their manufacturer doesn't do well on the test scores. That and I don't have a very high opinion of camera "fans".

[PackLight]

"Manipulating the data" with the intent to mislead sounds like fudging to me, though it wasn't your word. As far as I can tell they are simply summarizing / aggregating the data and it just happens that with the current sensors, the way they do this doesn't work out well for Canon.

It has nothing to do with the Nikon/Canon debate as far as I am concerned.

You can read at DxO which numbers and factors they use. There are factors they leave out of the evaluation as well and they tell you this. The weight they apply to the three factors they use to arrive at the number score, is arrived at and determined by DxO. Why should we believe that the way they arrive at this average score is correct for how a sensor should be evaulated and scored? Is it because they "claim" to be the leader in sensor testing?

When I first went to DxO to use the information they provide to compare Canon cameras it was years ago and, Nikon only had few in the top ten. At that time when I looked at the scores they had just on Canon cameras the scores just didn't line up with reality with Canon vs Canon. If the final results do not line up with the real world situation, perhaps they should look at how they combine all this data to arrive at a final number score.

Then there is the description "landscape" or "sports" they use, rather than calling the test what it is actually for. Why don't they describe the test for what it is, and then imply  this is important for that activity. With landscaping DR is not the only thing that matters, why apply the name landscape to high DR?

I find DxO's presentation of the data to miss leading. Any time a large company chooses to put out data that on the surface appears to miss lead the general public I think you have to question their motives.

I do not believe the "simply summarizing" view. But then maybe in their minds the believe they are.

[tnargs]

I do not believe the "simply summarizing" view. But then maybe in their minds they believe they are.

Have you seen their new lens scoring thingy?   

Clearly they haven't learnt their lesson. Blinkers? 

[PackLight]

I will se that day  Canon produces a camera with as good or better sensor  measurements than the Sony etc , how will you detractors of DXO look at DXO  measurements then??

The exact same way I have for years. Even before Nikon released all of the recent cameras that scored so high and Canon was on the top. Their method was wrong in the past, it is now and unless they change it will be wrong in the future no matter who's sensor scores a better make believe score.

Will you switch your view of DxO when Canon is on top of the chart?

[elflord]

You can read at DxO which numbers and factors they use. There are factors they leave out of the evaluation as well and they tell you this. The weight they apply to the three factors they use to arrive at the number score, is arrived at and determined by DxO. Why should we believe that the way they arrive at this average score is correct for how a sensor should be evaulated and scored? Is it because they "claim" to be the leader in sensor testing?

The way they choose to weight those factors is subjective. Therefore it is neither correct, nor incorrect. If you demand hard numbers as opposed to the wishy-washy subjective stuff, you will insist on seeing the measurements. I would feel very differently about DxO if they only showed us their summary numbers and not the underlying measurements.

As far as I'm concerned, they are welcome to publish whatever subjective summaries they like, though I don't care for these myself. Ultimately these I see as just another opinion -- well informed of course, but still just an opinion. There are other smart people who will have different opinions. The only opinion I care much for is my own after seeing the data.
 

When I first went to DxO to use the information they provide to compare Canon cameras it was years ago and, Nikon only had few in the top ten.

So I don't understand how they are trying to be "misleading" or what their alleged purpose is.

Then there is the description "landscape" or "sports" they use, rather than calling the test what it is actually for. Why don't they describe the test for what it is, and then imply  this is important for that activity. With landscaping DR is not the only thing that matters, why apply the name landscape to high DR?

I think the full name of the tests are (the parenthetical remarks are part of the name)

portrait (color depth),
landscape (dynamic range),
ports (low-light ISO)

I don't see what the problem is -- those who know what the technical terms mean, you can simply look past the non-technical ones.  For those who don't, omitting the non-technical terms doesn't help a whole lot.

I find DxO's presentation of the data to miss leading. Any time a large company chooses to put out data that on the surface appears to miss lead the general public I think you have to question their motives.

I still don't get who they are misleading and what they are trying to get their "victims" to do. So far we don't have a plausible motive for this alleged deception. According to the originator of this thread, no-one is fooled anyway, so not only we don't have a cause, we don't see an effect either. All we have is some grumbling from disgruntled camera fans, and additionally the fact that you take issue with some perceived inaccuracies in the way they summarize their data.

[elflord]

Will you switch your view of DxO when Canon is on top of the chart?

No, but I might upgrade my 5DII.

[TheSuede]

Have you seen their new lens scoring thingy?   

Clearly they haven't learnt their lesson. Blinkers? 

I very much do NOT like their new lens score "thingy", and I especially dislike the fact that they removed the underlying base data from public view. But I know how to read MTF / image height and MTF / frequency diagrams - something I (without meaning to sound condescending) honestly believe very few average camera buyers do. I understand why they did it, but I don't like it.

[TheSuede]

Isn't trying to use the argument:
-"They're wrong because Canon actually sell quite a lot of cameras"

Actually, the argument is, "They're irrelevant (or at least, not very relevant) because Canon sells quite a lot more cameras than their competitors who score much higher in DxOMark's rankings."

(some stuff of the later part of the quote removed)

Actually (actually x3 now! ) your argument needs to be refined even more... To reduce ambiguity to a level where the sentence actually makes any sense, it needs to be:
-"DxO scores in themselves are almost 100% irrelevant to camera market shares"

And that does not in any way mean that they are irrelevant to camera image quality. Which is what most people writing with a hostile intent towards DxO want it to mean.

[PackLight]

I will se that day  Canon produces a camera with as good or better sensor  measurements than the Sony etc , how will you detractors of DXO look at DXO  measurements then??

The exact same way I have for years. Even before Nikon released all of the recent cameras that scored so high and Canon was on the top. Their method was wrong in the past, it is now and unless they change it will be wrong in the future no matter who's sensor scores a better make believe score.

Will you switch your view of DxO when Canon is on top of the chart?

your answers and statements are not worthy to be treated with a answer
there are plenty  answers already.

Why not? I answered your worthless question with a legitimate response.

[Hobby Shooter]

Isn't trying to use the argument:
-"They're wrong because Canon actually sell quite a lot of cameras"

Actually, the argument is, "They're irrelevant (or at least, not very relevant) because Canon sells quite a lot more cameras than their competitors who score much higher in DxOMark's rankings."

(some stuff of the later part of the quote removed)

Actually (actually x3 now! ) your argument needs to be refined even more... To reduce ambiguity to a level where the sentence actually makes any sense, it needs to be:
-"DxO scores in themselves are almost 100% irrelevant to camera market shares"

And that does not in any way mean that they are irrelevant to camera image quality. Which is what most people writing with a hostile intent towards DxO want it to mean.

I would argue that irrelevant can not be graded in %, it's either relevant or irrelevant, binary. On the other hand something can be more or less relevant, but something can not be more or less irrelevant.

[neuroanatomist]

Actually (actually x3 now! ) your argument needs to be refined even more... To reduce ambiguity to a level where the sentence actually makes any sense, it needs to be:
-"DxO scores in themselves are almost 100% irrelevant to camera market shares"

And that does not in any way mean that they are irrelevant to camera image quality. Which is what most people writing with a hostile intent towards DxO want it to mean.

Perhaps one more refinement: irrelevant to camera sensor image quality.  A camera with a great sensor and lousy AF is likely to deliver poor image quality in many situations.

[PackLight]

I still don't get who they are misleading and what they are trying to get their "victims" to do. So far we don't have a plausible motive for this alleged deception. According to the originator of this thread, no-one is fooled anyway, so not only we don't have a cause, we don't see an effect either. All we have is some grumbling from disgruntled camera fans, and additionally the fact that you take issue with some perceived inaccuracies in the way they summarize their data.

Not everyone researches as in depth as those on Canon Rumors. I know many with DSLR's that wouldn't have a clue how to interpret DxO's information, and they would just take the overall number at face value. What DxO's motive is I wouldn't know, but when ever a public company offers information there is usually a motive and it almost always involves monetary gain. Why do they post this information at all? Really I don't care.

[Duprant]

Not everyone researches as in depth as those on Canon Rumors. I know many with DSLR's that wouldn't have a clue how to interpret DxO's information, and they would just take the overall number at face value. What DxO's motive is I wouldn't know, but when ever a public company offers information there is usually a motive and it almost always involves monetary gain. Why do they post this information at all? Really I don't care.

Their motive is to publicize the DxO brand and sell DxO software. Nothing nefarious here.

[PackLight]

Their motive is to publicize the DxO brand and sell DxO software. Nothing nefarious here.

What?? You don't think Nikon sent a can of Christmas popcorn or a care package over to DxO this year for all of their help?