DXO vs Reality

[wickidwombat]

I'm just wondering, alot of people get REALLY worked up over these dxo tests, however theire numbers relating to various cameras (to me anyway) dont appear to reflect real world results take the medium format digital backs for example, these are simply amazing yet score lower than a sony or a nikon?

Personally i dont put any faith in this sort of analysis

Note: lucky i cant get smited to death by the DxO brigade

[peederj]

Better than Ken Rockwell at least! 

[V8Beast]

I received feedback from a client today who wasn't too thrilled with my work. He commented on how the images were poorly lit and composed, and lacked any emotion. Fortunately, I was able to manipulate the exif data to make it appear as if the images were shot with a D800. I sent him a link to its DxOMark test results, explaining how it has the best-performing sensor on earth. He then said, "you're right, these image right here are some medium format $**t. Now everyone's happy

[Mt Spokane Photography]

If Nikon were a big customer, and Canon was not, what would you do?
 
http://www.dxo.com/us/image_quality/customers2
 
DXO would like you to think they are independent and do not take advertising from camera makers, but ...

[Zhanger]

I'm a Nikon shooter currently with a D5100. DxOMark ranks the D5100/D7000 very well in terms of DR, but I can never really get the amount of dynamic range that I'm happy with in my photos. This is primarily why I'm really looking to go full frame.

Just goes to show you how useless numbers are.

[Mt Spokane Photography]

I'm a Nikon shooter currently with a D5100. DxOMark ranks the D5100/D7000 very well in terms of DR, but I can never really get the amount of dynamic range that I'm happy with in my photos. This is primarily why I'm really looking to go full frame.

Just goes to show you how useless numbers are.

The DR ratings are like gas milage ratings.  They are the best ideal case, but not what most users see.  The DR ratings for jpeg are closer to reality.  You can get more DR from Raw files, but the images more often than not look flat and awful.

[Zhanger]

I'm a Nikon shooter currently with a D5100. DxOMark ranks the D5100/D7000 very well in terms of DR, but I can never really get the amount of dynamic range that I'm happy with in my photos. This is primarily why I'm really looking to go full frame.

Just goes to show you how useless numbers are.

The DR ratings are like gas milage ratings.  They are the best ideal case, but not what most users see.  The DR ratings for jpeg are closer to reality.  You can get more DR from Raw files, but the images more often than not look flat and awful.

Which is why I really hate processing RAWs, especially on a D5100, which is not exactly a top-notch camera. I find myself just shooting straight JPEGs cause I'm usually pretty good with nailing the exposures the first time around.

Poking my head around the Canon side cause I have pretty much no commitment to Nikon as far as lenses are considered (hell I use a Canon EOS strap on my D5100...)

[JR]

I'm just wondering, alot of people get REALLY worked up over these dxo tests, however theire numbers relating to various cameras (to me anyway) dont appear to reflect real world results take the medium format digital backs for example, these are simply amazing yet score lower than a sony or a nikon?

Personally i dont put any faith in this sort of analysis

Note: lucky i cant get smited to death by the DxO brigade

I agree with you wicki.  the more I look at real life sample picture from the various camera like D800, D4 and 5DmkIII at DP Review for example the less I find these DxO test relevant. 

If any of you guy are audiophile and sound system lovers, you would know that a cheep sony amplifier often has much better spec sheet then a $10,000 amplifiers!  But guess which one sounds better though in real life!  (I need to remember this one next time I get paranoid about my camera spec!)

 

[marekjoz]

If Nikon were a big customer, and Canon was not, what would you do?
 
http://www.dxo.com/us/image_quality/customers2
 
DXO would like you to think they are independent and do not take advertising from camera makers, but ...

Ot the other hand maybe Canon is not present there because doesn't win?

[Mt Spokane Photography]

If Nikon were a big customer, and Canon was not, what would you do?
 
http://www.dxo.com/us/image_quality/customers2
 
DXO would like you to think they are independent and do not take advertising from camera makers, but ...

Ot the other hand maybe Canon is not present there because doesn't win?

 
Or doesn't want to "Buy" a high rating?

[birtembuk]

Definitely agree on DXO discrepancy between lab bench and reality. Wrote a note already on this but the thread was stopped. Imo, this is pure academic exercise. A bit like taking an engine out of a car and test it on a formatted test-bench by white-coat-dressed technicians producing lots of data that nobody gives a damn about. Obviously, a car is much more than an engine. But, the way the data and numbers are presented give the false impression of authenticity. People love that. It's irrational, but they love that. For me, I would not buy any gear just based on DXO reviews. Just take it as informative as any other info available at the time.

Just remember how they found the 70-200/2.8 II to be just OK on their test-bench while absolutely every other review site around the world has only praise for that lens. Rings a bell ? 

[rpt]

The thing with testing is that you can test a product to death and find a huge number of issues or come up with fancy charts. However, if your test scenarios do not echo what 80% of your customers would do, there is no point testing. All the time spent in testing, finding issues and fixing them is a waste.

[rpt]

If Nikon were a big customer, and Canon was not, what would you do?
 
http://www.dxo.com/us/image_quality/customers2
 
DXO would like you to think they are independent and do not take advertising from camera makers, but ...

Ot the other hand maybe Canon is not present there because doesn't win?

Or Canon has not given DxO permission to put their logo up. In my job, a few of our best customers have a policy of never allowing their logo on such pages. So our "Customer Segments" page does not show their logos.

[marekjoz]

I will be convinced when 24-70 II will be treated same way.
I know that while testing you can show results in different ways to proof your point the way you want. But they have their testing procedures which are known before each test is performed. If so many people don't believe those tests, then why someone didn't proof yet that they lie?

I'm an engineer and believe in numbers, anyway. I know there are things difficult to measure (like bokeh). But I also know how difficult it's to talk to audiophiles, who pay 1k$ for better USB wire saying it sounds better.

I'm not saying they play fair, but judging that tests of Canon's gear are bad because there's Nikon as customer... Well, I'm not sure....

[pj1974]

I've also found DxO's camera sensor tests to be quite meaningless, when relating to the real world. They don't weight the criteria used in their tests well, imho. Often sensors which are given 'high' DxO ratings, don't perform as well across a number of 'real life situations' as other sensors which are given lower DxO ratings.

There are many other websites that cite themselves as professional, systematic, etc - whereas there are SO many variables, and if they don't get 1 thing just 'spot on' - it can ruin the overall results.  I recall one site that tested scores of lenses, but many of them at their very minimum focusing distance (MFD) - where some lenses (even high quality ones) are not at their sharpest... and the results were very skewed.

Another site I came across a few weeks ago - the testers only tested the 'minimum focal length' (eg 70-300mm @ 70mm - and all at f8) - and only a certain 'part' of the overall image (centre pixels).  It was crazy how some average quality lenses were ranked the same as others which were much higher, just based on that.

I have a marketing management and accounting degree, so I understand about corporate logos, branding, sponsorship, 'marketing words', etc - very well.  So that could be an issue... but I doubt that DxO would test eg Nikon more favourably just because of that (they could open themselves up to legal action if that were the case....)

Having said that, I do like DxO's Optics Pro software a LOT.  So I'm a customer of DxO's.  I won't be so 'put off' by the annoyances I have with DxO's sensor test not to use their good stuff!

Cheers.

Paul