DXO: Fair and Balanced

[3kramd5]

...
You are implying that the poster thinks the 600 is better than the 50 because it's bigger than the 50.

Well if he is going to do what he said then clearly he does think so

?

"Next time I am at a sporting event, I am going to sneak down to the sidelines with a Sony A7s II, the Canon 50mm STM, and a printout of the DXO scores. Those guys using the 600 II lens and 1dx will be full of regret when they see my lens is several points better than theirs and my camera body totally destroys them for sports shooting scores."

If he were drawing a conclusion that bigger is better, the analogy is backwards. Rather, he was demonstrating is that scores don't relate to applicability.

You, not the quoted poster, are introducing an irrelevant variable (size) which no one else is considering. You're making a gross assumption that he thinks the 1Dx + 600 II is a better combo for the sidelines of a sporting event than the A7S2 + 50 STM because the former combo is bigger, when most people would consider it better because of higher framerate, longer focal length, better autofocus, easier handling, etc.

 

[PhotographyFirst]

...
You are implying that the poster thinks the 600 is better than the 50 because it's bigger than the 50.

Well if he is going to do what he said then clearly he does think so

?

"Next time I am at a sporting event, I am going to sneak down to the sidelines with a Sony A7s II, the Canon 50mm STM, and a printout of the DXO scores. Those guys using the 600 II lens and 1dx will be full of regret when they see my lens is several points better than theirs and my camera body totally destroys them for sports shooting scores."

If he were drawing a conclusion that bigger is better, the analogy is backwards. Rather, he was demonstrating is that scores don't relate to applicability.

You, not the quoted poster, are introducing an irrelevant variable (size) which no one else is considering. You're making a gross assumption that he thinks the 1Dx + 600 II is a better combo for the sidelines of a sporting event than the A7S2 + 50 STM because the former combo is bigger, when most people would consider it better because of higher framerate, longer focal length, better autofocus, easier handling, etc.

It's best to just put the trolls on your ignore list. Every time you reply, he gets his e-jollies and thus wins every time no matter how wrong. The guy is arguing over a part of a post that was written to be humorous, let alone arguing something completely imaginary.

 

[Don Haines]

The scoring system is fine. There is nothing wrong with the scoring system.

My problem with DXO is the entire concept of reducing things to a single number is deeply flawed.

While I understand all reviews at DpReview etc. etc. etc. are imminently painful to you, single number scores - sometimes much less developed than DXO's - enjoy extremely wide spread popularity on the net, in newspapers, magazines etc.

Why single DXO out for this?

It is because it is too wide.

Just like rating the best vehicle based on the number of cupholders is ludicrous, rating the best lens on one parameter is silly. If one wanted to rate the best economy car based on cost/reliability/fuel economy then that makes sense.... but rating the best dumptruck on those same parameters means that a mine should get a Toyota Prius rather than a Caterpillar 795D to haul that 240 ton load up out of the mine......

 

[ahsanford]

The scoring system is fine. There is nothing wrong with the scoring system.

My problem with DXO is the entire concept of reducing things to a single number is deeply flawed.

While I understand all reviews at DpReview etc. etc. etc. are imminently painful to you, single number scores - sometimes much less developed than DXO's - enjoy extremely wide spread popularity on the net, in newspapers, magazines etc.

Why single DXO out for this?

It is because it is too wide.

Just like rating the best vehicle based on the number of cupholders is ludicrous, rating the best lens on one parameter is silly. If one wanted to rate the best economy car based on cost/reliability/fuel economy then that makes sense.... but rating the best dumptruck on those same parameters means that a mine should get a Toyota Prius rather than a Caterpillar 795D to haul that 240 ton load up out of the mine......

+1. Just kill the overall score metric and my respect for DXO goes from a 1 out 10 to perhaps a 4 out of 10. I'm not kidding.

Yes, their methods are nuts, but no one else on the planet is attempting a 'human genome project'-level of comprehensive testing that updates when sensor resolutions increase like they do. Though I always prefer dedicated reviews from folks I respect more than DXO, sometimes you want to know how a 28mm f/2.8 IS USM does on 50 MP compared to 22 MP, and DXO -- despite all it's flaws -- has that information.

- A

 

[Maiaibing]

I'm sorry, but where in their claim does it ever say "science" or "scientific"? Nowhere, because they are not allowed to.

Things starting to click now?

If you have a therapeutic cream, then they must do efficacy testing to make scientific claims, but that's more pharmaceutical.

Glad you spotted the difference yourself this time.

Do your scientific papers include the word "scientific" or "science"? I actually read such papers and cannot - off hand - remember ever having seen the words "scientific" or "science" mentioned (except in philosophical papers discussing what science is or is not). Not excluded - but seems very unusual to me.

But anyways, I guess since cosmetic companies mislead with garbage science, it's okay for DxOmark to do so.

This is the second time you try to draw me into a discussion of whether DXO is scientific. I never commented on this question - because I don't care. Yes, there's lots of garbage science around - also in the scientific world - be happy you can spot the difference. Too many cannot.

DXO has one lens measurement that I find very valuable - and I'm very happy to be able to get that info from them. YMMV.

 

[neuroanatomist]

Do your scientific papers include the word "scientific" or "science"? I actually read such papers and cannot - off hand - remember ever having seen the words "scientific" or "science" mentioned (except in philosophical papers discussing what science is or is not).

Some are published in this high impact journal.

 

[ahsanford]

Do your scientific papers include the word "scientific" or "science"? I actually read such papers and cannot - off hand - remember ever having seen the words "scientific" or "science" mentioned (except in philosophical papers discussing what science is or is not).

Some are published in this high impact journal.

+1. Scientists and engineers are probably the largest part of the enthusiast-class of CR forum dwellers.

- A

 

[Maiaibing]

Scientists and engineers are probably the largest part of the enthusiast-class of CR forum dwellers.

Do you have stats to back that claim? ;D

 

[bdunbar79]

I'm sorry, but where in their claim does it ever say "science" or "scientific"? Nowhere, because they are not allowed to.

Things starting to click now?

If you have a therapeutic cream, then they must do efficacy testing to make scientific claims, but that's more pharmaceutical.

Glad you spotted the difference yourself this time.

Do your scientific papers include the word "scientific" or "science"? I actually read such papers and cannot - off hand - remember ever having seen the words "scientific" or "science" mentioned (except in philosophical papers discussing what science is or is not). Not excluded - but seems very unusual to me.

But anyways, I guess since cosmetic companies mislead with garbage science, it's okay for DxOmark to do so.

This is the second time you try to draw me into a discussion of whether DXO is scientific. I never commented on this question - because I don't care. Yes, there's lots of garbage science around - also in the scientific world - be happy you can spot the difference. Too many cannot.

DXO has one lens measurement that I find very valuable - and I'm very happy to be able to get that info from them. YMMV.

Then what are you arguing? Cosmetic companies (your example) aren't claiming to be scientific organizations providing scientific data. DxOmark is. That's why they are getting singled out. Because, they're not.

 

[AlanF]

Do your scientific papers include the word "scientific" or "science"? I actually read such papers and cannot - off hand - remember ever having seen the words "scientific" or "science" mentioned (except in philosophical papers discussing what science is or is not). Not excluded - but seems very unusual to me.

A search of Web of Knowledge gives 597,267 papers published with the word "science" in their title.

 

[ahsanford]

Scientists and engineers are probably the largest part of the enthusiast-class of CR forum dwellers.

Do you have stats to back that claim? ;D

I honestly don't, but call it a hunch -- too many people on this forum speak in the language I'm accustomed to hearing at work. (Though I traded writing peer-reviewed publications for creating patents some time ago.)

- A

 

[Maiaibing]

I honestly don't, but call it a hunch -- too many people on this forum speak in the language I'm accustomed to hearing at work. (Though I traded writing peer-reviewed publications for creating patents some time ago.)

- A

Good enough for me!

 

[neuroanatomist]

Do your scientific papers include the word "scientific" or "science"? I actually read such papers and cannot - off hand - remember ever having seen the words "scientific" or "science" mentioned (except in philosophical papers discussing what science is or is not). Not excluded - but seems very unusual to me.

A search of Web of Knowledge gives 597,267 papers published with the word "science" in their title.

Surely you're not suggesting that ISI has better data than some random person on the Internet?!

 

[Larsskv]

The scoring system is fine. There is nothing wrong with the scoring system.

My problem with DXO is the entire concept of reducing things to a single number is deeply flawed.

While I understand all reviews at DpReview etc. etc. etc. are imminently painful to you, single number scores - sometimes much less developed than DXO's - enjoy extremely wide spread popularity on the net, in newspapers, magazines etc.

Why single DXO out for this?

It is because it is too wide.

Just like rating the best vehicle based on the number of cupholders is ludicrous, rating the best lens on one parameter is silly. If one wanted to rate the best economy car based on cost/reliability/fuel economy then that makes sense.... but rating the best dumptruck on those same parameters means that a mine should get a Toyota Prius rather than a Caterpillar 795D to haul that 240 ton load up out of the mine......

I really dislike analogies because to me they represent a failure to be able to explain the situation using the terms of concern.

It's simple:

What DxO measures and scores is the quality of the image that is created on the sensor by the lens.

Nothing else.

It's up to the photographer to take that information and then use it as part of the selection process for determining which tool to buy or use.

Otherwise, what does Canon supply to help you choose between their lenses? A mm range and f-stop? MTF graph?

Dilbert, please tell me what the DXO scores mean, what are they actually telling us about a lens? Because when buying that f/4 zoom lens, I'm obviously not going to shot it at 150 lux at base ISO and 1/60 shutter speed, but in sufficient light. Is the DXO scores relevant for a purchasing decision in such a situation? Why/why not?

 

[Maiaibing]

A search of Web of Knowledge gives 597,267 papers published with the word "science" in their title.

Yeah - you're right. So not only unusual but extremely rare. Thanks for the check-up.

 

[Don Haines]

The scoring system is fine. There is nothing wrong with the scoring system.

My problem with DXO is the entire concept of reducing things to a single number is deeply flawed.

While I understand all reviews at DpReview etc. etc. etc. are imminently painful to you, single number scores - sometimes much less developed than DXO's - enjoy extremely wide spread popularity on the net, in newspapers, magazines etc.

Why single DXO out for this?

It is because it is too wide.

Just like rating the best vehicle based on the number of cupholders is ludicrous, rating the best lens on one parameter is silly. If one wanted to rate the best economy car based on cost/reliability/fuel economy then that makes sense.... but rating the best dumptruck on those same parameters means that a mine should get a Toyota Prius rather than a Caterpillar 795D to haul that 240 ton load up out of the mine......

I really dislike analogies because to me they represent a failure to be able to explain the situation using the terms of concern.

It's simple:

What DxO measures and scores is the quality of the image that is created on the sensor by the lens.

Nothing else.

It's up to the photographer to take that information and then use it as part of the selection process for determining which tool to buy or use.

Otherwise, what does Canon supply to help you choose between their lenses? A mm range and f-stop? MTF graph?

Ok.... How about this......

It is because it is too wide.

Just like rating the best lens based on the T-stop is ludicrous, rating the best lens on one parameter is silly. If one wanted to rate the best economy lens based on lowest cost then that makes sense.... but rating the best birding lens on those same parameters means that a birder should get a Nifty-fifty rather than a Big White to haul that 240 meter away target up onto the sensor....

 

[Hector1970]

I've no problem with DXO.
There is probably no perfect measurement.
It tells you something about the camera / lens.
I've no idea if any individual sensor from Nikon is actually better than Canon.
I just think Canon produce on balance of everything a more usable and reliable camera.
I've used Nikon's and maybe they are great on sensors but I just prefer the layout of Canon, the glass is better etc....
I prefer myself more descriptive reviews of camera gear.
I like the Digital Picture reviews by Bryan Carnathan .
I haven't gone wrong with any of his recommendations so far.

 

[neuroanatomist]

Well the problem is that it isn't easy to draw any conclusions about which parameter is most important.

So DxO used dilbertland logic to decide the most important parameters for a Lens Score are lens transmission, sensor DR at base ISO, and sensor color depth at base ISO.

 

[AlanF]

Talking about Lens score, there is a site called lenscore.org (+senscore), which claims to be scientific. But, I don't understand their overall ratings either. For instance, with zoom lenses they give an overall score for the lens without, unless I have missed it, stating the focal length etc. Seems nonsenscore to me.

 

[3kramd5]

...
"Next time I am at a sporting event, I am going to sneak down to the sidelines with a Sony A7s II, the Canon 50mm STM, and a printout of the DXO scores. Those guys using the 600 II lens and 1dx will be full of regret when they see my lens is several points better than theirs and my camera body totally destroys them for sports shooting scores."

If he were drawing a conclusion that bigger is better, the analogy is backwards. Rather, he was demonstrating is that scores don't relate to applicability.

You, not the quoted poster, are introducing an irrelevant variable (size) which no one else is considering. You're making a gross assumption that he thinks the 1Dx + 600 II is a better combo for the sidelines of a sporting event than the A7S2 + 50 STM because the former combo is bigger, when most people would consider it better because of higher framerate, longer focal length, better autofocus, easier handling, etc.

Ok, going back ... why do I assume it is related to size?

Nobody ever uses the 200-400/f4 or 70-200 or 14mm or any other non-50mm lens to compare the 50mm with. So quite clearly a 600mm lens is being chosen for a specific reason whereas any other lens would do.

The analogy here is sports, and in these discussions that's the recurring topic. The 600mm is the longest autofocusing (600 X 2) lens they make, which is why most people would choose it rather than the larger, heavier, more expensive lens.

If people were discussing studio, or landscape, or street photography, it's unlikely the 600 would be invoked.