DxOMark vs. Reality

[elflord]

Perhaps one more refinement: irrelevant to camera sensor image quality.  A camera with a great sensor and lousy AF is likely to deliver poor image quality in many situations.

Well if your point is that the final product depends on aggregate performance of the whole system, that's fine, but what are they to do about it ?

They could aggregate AF performance into an even more subjective and debatable score, but you have repeatedly said that this is precisely what they shouldn't be doing.

Are you simply opposed to them publishing any sensor benchmarks because they expose the technological stagnation of your favourite camera brand ?

[TheSuede]

Perhaps one more refinement: irrelevant to camera sensor image quality.  A camera with a great sensor and lousy AF is likely to deliver poor image quality in many situations.

Hehe... Yup - true.

But when you can 'assume with reasonable confidence' that two cameras have similar AF accuracy and sensitivity, the static image quality metric still rules.
-And you might have noticed that I deliberately MENTIONED the autofocus as being left out of the metric in the first place - so no modification is needed.

Better image quality = better image quality.
No modifier, no disclaimer, no BS sidetracking.

[neuroanatomist]

Perhaps one more refinement: irrelevant to camera sensor image quality.  A camera with a great sensor and lousy AF is likely to deliver poor image quality in many situations.

Well if your point is that the final product depends on aggregate performance of the whole system, that's fine, but what are they to do about it ?

They could aggregate AF performance into an even more subjective and debatable score, but you have repeatedly said that this is precisely what they shouldn't be doing.

Are you simply opposed to them publishing any sensor benchmarks because they expose the technological stagnation of your favourite camera brand ?

Sorry, but my reply wasn't directed generally, nor to you specifically. If you read my previous posts, hopefully you'd realize your questions are tangential and I'm saying none of the statements you seem to be expressing on my behalf.

[neuroanatomist]

Perhaps one more refinement: irrelevant to camera sensor image quality.  A camera with a great sensor and lousy AF is likely to deliver poor image quality in many situations.

Hehe... Yup - true.

But when you can 'assume with reasonable confidence' that two cameras have similar AF accuracy and sensitivity, the static image quality metric still rules.
-And you might have noticed that I deliberately MENTIONED the autofocus as being left out of the metric in the first place - so no modification is needed.

Better image quality = better image quality.
No modifier, no disclaimer, no BS sidetracking.

Well, perhaps a modifier or disclaimer is in order.      For example, how about better image quality at ISO 12800?  That may be important to some, but not others, and it's not represented in DxOMark's Scores at all.  But that's why I've stated several times that I applaud them for making the measurement data available.  While I feel it's improper to publish Scores without disclosing full details of how those Scores are determined, personally, it doesn't matter to me for my own buying decisions.

[PackLight]

it doesn't matter to me for my own buying decisions.

Obviously, or you would have bought Nikon 

[elflord]

Sorry, but my reply wasn't directed generally, nor to you specifically. If you read my previous posts, hopefully you'd realize your questions are tangential and I'm saying none of the statements you seem to be expressing on my behalf.

After rereading I think I see what you were getting at -- it was just very puzzling on a first reading (and yes I understand that taken literally  it was inconsistent with what you've posted before which is why I found it puzzling)

[bdunbar79]

Well it's like me and basketball.  In HS I couldn't miss in practice from the 3 point arc.  So you could say I was a great 3-point shooter.  However, I never hit a 3 in a game because I never got open and wasn't even quick enough to get open.  So it was actually meaningless.

We here on the forum know that this is just a sensor score.  Howver, consumers don't know that and if they just read the scores they automagically assume it is a better camera.  That's the problem.  It isn't.  Sensor is just ONE measured aspect of a camera and as a scientist, when I read their crap, it comes off to me as rather unscientific.  Whether it is or isn't, it just comes off that way to me.

You can defend it all you want, but it looks like crap to me and I don't use their scores at all.  Each to their own.

[RLPhoto]

How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|

[thepancakeman]

Well it's like me and basketball.  In HS I couldn't miss in practice from the 3 point arc.  So you could say I was a great 3-point shooter.  However, I never hit a 3 in a game because I never got open and wasn't even quick enough to get open.  So it was actually meaningless.

You, too, huh?  Excellent analogy!

[bdunbar79]

Yeah then I had to "find another sport" a year or two later .

[tnargs]

260 posts, how about a summary? 

Well, it seems that when people legitimately criticize DxO Labs' application of their measurements, apologists rush in to defend their measurements.

That's called an air ball, guys.

And when people point out that these DxO scores (applications) are being misused by review sites etc in a way that gives the wrong impression, apologists rush in to say that's not really DxO Labs' fault.

You must be joking.

Let us say Mercedes Inc engaged you to independently measure the quality of their product vis a vis say, Ford Inc, which they had also measured and knew would favour them. They also told you to summarize your findings and publish them. You go ahead and do it but goodness gracious, what's this? Your website is technically correct and shows Mercedes' quality advantage at the deepest level, but your way of summarizing and organizing and communicating gives the general impression that Ford is the better overall vehicle!

Suddenly, due to your excellent technical reputation, the world takes notice of your findings. Reviewers start to refer to 'The Ford Advantage' on their websites. Fleet buyers start recommending to boardrooms that current Mercedes contracts for trucks and buses should not be renewed, partly due to a new quality value matrix by an industry technical expert that favours other marques. The boardroom executive gossip soon reaches the ears of Mercedes executives.

Suddenly your client is on the phone to you in an absolute Teutonic fury. Yes, they say, your base measurements are correct but what on earth have you done with it? Why the blazes that way? Don't you realize how everyone is interpreting it?

Here is your answer: (hold on, wait for it, it's a beauty): "It's not my fault if they are not well enough informed to  grasp the underlying deep measurements correctly."

pfft

Good luck getting your invoice processed.

[tron]

How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|

+1000000 Very true. Rumor has it that a camera does not become worse simply because newer models appear     

[neuroanatomist]

Rumor has it that a camera does not become worse simply because newer models appear

Is that a CR1 or CR2 rumor?   

[tron]

Rumor has it that a camera does not become worse simply because newer models appear

Is that a CR1 or CR2 rumor?   

Definitely a CR3 one 

[RLPhoto]

How come the 5D2 IQ was impeccable before the D800? I find it still impeccable today. :|

+1000000 Very true. Rumor has it that a camera does not become worse simply because newer models appear     

ITS OVER NINE THOUSAND!