September 18, 2014, 12:01:54 PM

Author Topic: Why so much trust in DXO.  (Read 14982 times)

skitron

  • 6D
  • *****
  • Posts: 509
    • View Profile
Re: Why so much trust in DXO.
« Reply #45 on: April 20, 2012, 02:29:31 PM »
It looks to me like their results are fabricated more than they are measured. It does not help that their results do not compare well with other published test results or the reality found by many photographers. I also have severe doubts to their claims of being trustworthy.

Actually, I find their charts to be very informative and definitely give me a good sense of what to expect from the copy of lens or body I receive when I buy.

Now the distilled numbers (the so called DxOMarks) are a different story...they really have no practical value beyond generating a bunch of buzz and chatter. Mission accomplished in the case of D800 vs 5D3.

Well for lenses not.
Glad I did buy the 70-200 version II instead of the better tested 70-200 version I.

I've heard about that. I suppose it's part of the grain of salt we have to take knowing they are testing sample batches of exactly one copy. But I still find their graphs to be very informative, just not *definitive* is all. But between them and reading the better user reviews it's pretty easy for me to set the expectations levels about right for what I buy.
5D3, 70-200 f/2.8L IS II, 100L, 24-105L, Sigma 50/1.4 DG, Canon TC 1.4x III

canon rumors FORUM

Re: Why so much trust in DXO.
« Reply #45 on: April 20, 2012, 02:29:31 PM »

altenae

  • EOS M2
  • ****
  • Posts: 186
  • Enjoy Wildlife
    • View Profile
    • Wildlife-photos
Re: Why so much trust in DXO.
« Reply #46 on: April 20, 2012, 02:30:02 PM »
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??


skitron

  • 6D
  • *****
  • Posts: 509
    • View Profile
Re: Why so much trust in DXO.
« Reply #47 on: April 20, 2012, 02:35:37 PM »
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...
5D3, 70-200 f/2.8L IS II, 100L, 24-105L, Sigma 50/1.4 DG, Canon TC 1.4x III

altenae

  • EOS M2
  • ****
  • Posts: 186
  • Enjoy Wildlife
    • View Profile
    • Wildlife-photos
Re: Why so much trust in DXO.
« Reply #48 on: April 20, 2012, 03:12:42 PM »
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...

Well for me this list is not the way my lenses deliver in real world.
Would you buy your lens from this list ?
Then you would buy the 70-200 2.8 IS version over the version II !!!!!!!
Look at the 300mm F2.8 version II
I am lost in this lens score.



Razor2012

  • 5D Mark III
  • ******
  • Posts: 639
    • View Profile
Re: Why so much trust in DXO.
« Reply #49 on: April 20, 2012, 03:26:05 PM »
What's Nikon's 70-200 score?
5D MKIII w grip, 70-200 2.8L IS II, 24-70 2.8L II, 16-35 2.8L II, 100 2.8L IS macro, 600EX-RT

skitron

  • 6D
  • *****
  • Posts: 509
    • View Profile
Re: Why so much trust in DXO.
« Reply #50 on: April 20, 2012, 03:26:28 PM »
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...

Well for me this list is not the way my lenses deliver in real world.
Would you buy your lens from this list ?
Then you would buy the 70-200 2.8 IS version over the version II !!!!!!!
Look at the 300mm F2.8 version II
I am lost in this lens score.

""I don't bother with the silly "DxOMark" numbers though...""

I only bother to look at the field maps, they are very informative. The numbers are basically meaningless to me.

Think of it this way, the field maps and the graphs are raw data that you interpret. The so called "DxOMark" is their attempt to interpret the field maps and graphs for you.
« Last Edit: April 20, 2012, 03:29:17 PM by skitron »
5D3, 70-200 f/2.8L IS II, 100L, 24-105L, Sigma 50/1.4 DG, Canon TC 1.4x III

1982chris911

  • EOS M2
  • ****
  • Posts: 205
    • View Profile
    • View my Flickr Stream here
Re: Why so much trust in DXO.
« Reply #51 on: April 20, 2012, 05:11:01 PM »
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...

Well for me this list is not the way my lenses deliver in real world.
Would you buy your lens from this list ?
Then you would buy the 70-200 2.8 IS version over the version II !!!!!!!
Look at the 300mm F2.8 version II
I am lost in this lens score.

OK I am completely lost on how they score the lenses, because according to their score the Canon 300mm f/2.8L II USM is worse resolution wise than the Canon EF 75-300mm f/4-5.6 IS USM or the Canon EF 300mm f/4L IS USM ... Can anyone having both confirm that ???  ::)   
5D MKII, 5D MK III, 7D, Sigma 12-24 HSM2, Canon 17-40 F/4.0 L, Canon 24-70 F/2.8 L, Canon 70-200 F/2.8 IS II L , Ext x2 III + some other stuff

canon rumors FORUM

Re: Why so much trust in DXO.
« Reply #51 on: April 20, 2012, 05:11:01 PM »

dichiaras

  • Guest
Re: Why so much trust in DXO.
« Reply #52 on: April 20, 2012, 05:36:36 PM »
Understanding the lens scores is much more complicated than for the camera scores. Here's how to compare:

- pick two or three lenses WITH the camera you would use: cameras make a huge difference on the results (just look at a 50mm f/1.8 with a 7D and a 5DII)
-go to measurements, resolution, field map, and select one after another the whole range of focal lengths, for maximum aperture: this usually gives you the minimum sharpness (excluding very small apertures)
-repeat the above for fnumber=8 or 11: this usually gives you the maximum sharpness

Than you can look at transmission and the other variables with the same care.

The results are a matter of personal taste and  needs: with my Canon 10-22mm I usually shoot landscapes at small apertures, and it performs amazingly. If I look at the Nikon 10-24mm test, in general it might be better (with cameras of the same category, indeed it scores better than the Canon lens), but for my needs is much inferior.
« Last Edit: April 20, 2012, 05:38:47 PM by dichiaras »

cpsico

  • EOS M2
  • ****
  • Posts: 202
    • View Profile
Re: Why so much trust in DXO.
« Reply #53 on: April 20, 2012, 06:53:37 PM »
DXO is crap, the have the 24-70L rated 1point lower than the 24-105L which is utterly absurd.

Mike Ca

  • Guest
Re: Why so much trust in DXO.
« Reply #54 on: April 20, 2012, 08:07:24 PM »
I think the DxO measurements are accurate and give you good information about the aspects of the camera they are measuring. There is, of course, many other things to take into consideration than just the sensor when picking a camera.

What I have never been able to make sense of is the DxO overall or case usage scores. These numbers don't seem to make a lot of sense to me. I guess part of the problem is they are only rating the sensor, not the camera.

At ISO 100 the D800 has 2.25 stops better DR than the 5D III. If the 5D III had 6 stops of DR and the D800 had 8.25 stops of DR, this would be a huge deal, but the 5D III has 11 stops of DR and that is more DR than any monitor or print can show. The extra two stops of DR will not be seen unless the dynamic range is compressed using single frame HDR adjustment techniques. In the vast majority of photographs you will never see that extra DR. You can increase the DR of a camera like the 5D III using multi frame HDR, so that extra DR is not a critical factor for me.

Professional and good non-professional photographers have learned to work with light, either find good natural light, supplement natural light or create their own light. I recently heard a talk by an old news photographer. He said when he was sent to cover a parade or similar event, his basic rule was find the good light, then find something interesting to photograph in the good light. There are types of photography, like landscape, where that isn't always possible, and those are the photographers most interested in the high dynamic range in their cameras.

DxO is a company that sells software that among other things does single frame HDR adjustments.  They talk about an "exclusive HDR mono-image feature" in their software. They have a vested interest in encouraging camera manufactures to increase the DR of their cameras so that  this feature becomes more useful. Without HDR like adjustments in post processing, you will never see the difference in the dynamic range of the 5D III and the D800.

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: Why so much trust in DXO.
« Reply #55 on: April 20, 2012, 08:31:39 PM »
OK I am completely lost on how they score the lenses, because according to their score the Canon 300mm f/2.8L II USM is worse resolution wise than the Canon EF 75-300mm f/4-5.6 IS USM or the Canon EF 300mm f/4L IS USM ... Can anyone having both confirm that ???  ::)   

I think the main thing with their method for scoring that gives counterintuitive results is that they appear to take the best result across the whole aperture range. They also do some averaging across focal lengths (the 75-300 doesn't peak at the long end). Also, they tend to allow for some degradation across the frame so a lens that is very good in the center with bad corners might get off lightly.

I'll describe the field maps in a bit more depth, I think this will be consistent with what users of these lenses observe.

You see that the 300mm f/2.8 is better wide open than the 300mm f/4. If you stop the 300mm f/2.8 down, it's not even close -- the 300mm f/2.8 is already near its peak by f/4. The 300mm needs to be stopped down to f/8 to be in the same territory.

According to the field map, the 75-300mm at 300mm is pretty horrid across the aperture range though if you stop down to f/11, the middle of the frame isn't too bad. At 100mm or less,  corners sharpen up by f/8 and the center is sharp. At 135mm it needs to be stopped down to f/11 and longer than that the corners are bad.

kirispupis

  • Canon 70D
  • ****
  • Posts: 308
    • View Profile
Re: Why so much trust in DXO.
« Reply #56 on: April 20, 2012, 10:17:18 PM »
People trust DXO because they do not have the skills/time to actually learn photography.  Rather than think for themselves, they trust some company to compute a number and believe if they own the camera with the highest number, their photography will automatically improve.
5D3|TS-E 24 II|TS-E 17|TS-E 90|200-400/1.4x|MP-E 65|100/2.8 IS Macro|70-200/2.8 IS II||16-35/2.8 II|EOS M

1982chris911

  • EOS M2
  • ****
  • Posts: 205
    • View Profile
    • View my Flickr Stream here
Re: Why so much trust in DXO.
« Reply #57 on: April 21, 2012, 12:38:30 AM »
OK I am completely lost on how they score the lenses, because according to their score the Canon 300mm f/2.8L II USM is worse resolution wise than the Canon EF 75-300mm f/4-5.6 IS USM or the Canon EF 300mm f/4L IS USM ... Can anyone having both confirm that ???  ::)   

I think the main thing with their method for scoring that gives counterintuitive results is that they appear to take the best result across the whole aperture range. They also do some averaging across focal lengths (the 75-300 doesn't peak at the long end). Also, they tend to allow for some degradation across the frame so a lens that is very good in the center with bad corners might get off lightly.

I'll describe the field maps in a bit more depth, I think this will be consistent with what users of these lenses observe.

You see that the 300mm f/2.8 is better wide open than the 300mm f/4. If you stop the 300mm f/2.8 down, it's not even close -- the 300mm f/2.8 is already near its peak by f/4. The 300mm needs to be stopped down to f/8 to be in the same territory.

According to the field map, the 75-300mm at 300mm is pretty horrid across the aperture range though if you stop down to f/11, the middle of the frame isn't too bad. At 100mm or less,  corners sharpen up by f/8 and the center is sharp. At 135mm it needs to be stopped down to f/11 and longer than that the corners are bad.

Well what I mean is only the linepair value as they tested it on the 5d MK II. In this regard they tested the 300 f2.8 II as the worst !!! This is in complete opposition of every other test you find on the internet and I am sure that no one would spend about 8000 US$ for this lens if the other 300mm options would even come close ...   
« Last Edit: April 21, 2012, 06:25:14 AM by 1982chris911 »
5D MKII, 5D MK III, 7D, Sigma 12-24 HSM2, Canon 17-40 F/4.0 L, Canon 24-70 F/2.8 L, Canon 70-200 F/2.8 IS II L , Ext x2 III + some other stuff

canon rumors FORUM

Re: Why so much trust in DXO.
« Reply #57 on: April 21, 2012, 12:38:30 AM »

1982chris911

  • EOS M2
  • ****
  • Posts: 205
    • View Profile
    • View my Flickr Stream here
Re: Why so much trust in DXO.
« Reply #58 on: April 21, 2012, 06:32:11 AM »
You can see what I mean looking at these side by side test charts:

The First one is the 300 f2.8 II at f4 against the 70-300mm f4 - 5.6 IS at f8 and 300mm. You see that these lenses look like they are not even from same the same planet resolution wise (as expected)

The second one shows the both 70-200mm f2.8 IS versions (I and II) against each other and the newer lens again is much better (also as expected)

So how can DxO test these in in any way differently ???   

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=739&Camera=453&Sample=0&FLI=0&API=2&LensComp=358&CameraComp=453&SampleComp=0&FLIComp=4&APIComp=3

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=103&Camera=453&Sample=0&FLI=4&API=0&LensComp=687&CameraComp=453&SampleComp=0&FLIComp=3&APIComp=0
« Last Edit: April 21, 2012, 06:35:10 AM by 1982chris911 »
5D MKII, 5D MK III, 7D, Sigma 12-24 HSM2, Canon 17-40 F/4.0 L, Canon 24-70 F/2.8 L, Canon 70-200 F/2.8 IS II L , Ext x2 III + some other stuff

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: Why so much trust in DXO.
« Reply #59 on: April 21, 2012, 06:53:28 AM »
Well what I mean is only the linepair value as they tested it on the 5d MK II. In this regard they tested the 300 f2.8 II as the worst !!! This is in complete opposition of every other test you find on the internet and I am sure that no one would spend about 8000 US$ for this lens if the other 300mm options would even come close ...

Well, please take the time to read my comment which explains why the numbers are the way they are. There is clearly an issue with the way they aggregate the numbers (across the aperture and focal length range) but from the measurements themselves the 300mm f/2.8 has some clear advantages.

I think posters here are fishing for examples of cases where Dxomark's aggregation method is misleading (e.g. this sudden affinity for medium format is a bit hilarious) but they do make their measurements available.

In this case (the D800 or a number of other cameras), it's pretty clear where Canon is falling short on the test - dynamic range at low ISO.

canon rumors FORUM

Re: Why so much trust in DXO.
« Reply #59 on: April 21, 2012, 06:53:28 AM »