Why so much trust in DXO.

Status
Not open for further replies.
Sorry, but British and American weight standards are much like those in antiquity: Anyone can come up with their own weights and anything like a common measurement such as the yard is mere coincidence ::). That does not make much sense to me with current societies facing globalization.

I would be in favour of coming up with an international standard for power plugs. Let's use the ones from Jordania, being fully compliant with the simplified European plug design. ;)
 
Upvote 0
Canon-F1 said:
Neeneko said:
In the real world, selection bias and brand loyalty tend to be the dominant factors. The whole point of repeatable tests is to determine, using set criteria, how things actually preform. When just out and shooting people build a lot of bias into what they think of performance, often with 'this is newer/more expensive, so it must be better otherwise I would be a real fool to have paid so much more for it!' thrown in.

that does not explain how techradar and DXO come to different conclusions... given they both use the DXO testing method.

they don't. they use the same software. there is no way to know if there are any external factors influencing the test. If you take DXO as having test setup free of external interference then it implies that tests that don't match it aren't setup the same or even properly.

Neeneko said:
psolberg said:

Actually.. eyes suck. The vast majority of what you see is extrapolated, the actual resolution of a human eye is very low, but we have a lot of grey matter behind them that tries to combine the little information it is getting with a bank of world knowledge and internal models to product its best guess about what it is seeing. Human vision is very easy to trick and what you expect to see plays a dominant role in what you actually perceive.

are you trying to say something? sorry but I feel asleep half way.....

My conspiracy theory:
I wonder if DxO is purposely picking the 8MP downsize option to boost Nikon scores
If canon had made a 46MP sensor then the conspiracy was that they are helping canon. Canon's score is what canon gets because of the decisions they took long ago, not DXO's.
 
Upvote 0
Reality check, please! I have read here how DXO has set up their tests perfectly and free of external interference and such.
First of all there is no such thing as a perfect test. There is always a possibility for someone doing it better. Secondly I don't believe for one nanosecond that DXO is free of what has been described as Nikon fanboyism. Maybe this affects only one DXO employee determining test results, but that is where it counts, especially if DXO could not get rid of this employee over years. That whole thing smells rather fishy to me, like a fish that has slowly travelled from the Mediterranean sea to the vicinity of Paris, sitting there on a shelf for a while.

If Nikon's market performance would mirror DXO's test results, perhaps there would be something there, but this is obviously not the case. So why are people's buying decisions so different from DXO's test results? Why would photographers owning both Nikon and Canon gear still use Canon gear?

In my humble opinion Nikon's and Canon's products are comparable to some extend and market performance by both companies does support this view, even giving Canon a generous advantage, but according to DXO it appears like no one should buy Canon's products. That doesn't fit reality, so DXO's test results must be way off. It does not help that they try to cover up, by making this appear to be impartial and proper, when it appears to be not.
 
Upvote 0
altenae said:
skitron said:
AmbientLight said:
It looks to me like their results are fabricated more than they are measured. It does not help that their results do not compare well with other published test results or the reality found by many photographers. I also have severe doubts to their claims of being trustworthy.

Actually, I find their charts to be very informative and definitely give me a good sense of what to expect from the copy of lens or body I receive when I buy.

Now the distilled numbers (the so called DxOMarks) are a different story...they really have no practical value beyond generating a bunch of buzz and chatter. Mission accomplished in the case of D800 vs 5D3.

Well for lenses not.
Glad I did buy the 70-200 version II instead of the better tested 70-200 version I.

I've heard about that. I suppose it's part of the grain of salt we have to take knowing they are testing sample batches of exactly one copy. But I still find their graphs to be very informative, just not *definitive* is all. But between them and reading the better user reviews it's pretty easy for me to set the expectations levels about right for what I buy.
 
Upvote 0
skitron said:
altenae said:
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...

Well for me this list is not the way my lenses deliver in real world.
Would you buy your lens from this list ?
Then you would buy the 70-200 2.8 IS version over the version II !!!!!!!
Look at the 300mm F2.8 version II
I am lost in this lens score.
 

Attachments

  • DXOLens.jpg
    DXOLens.jpg
    429 KB · Views: 1,046
Upvote 0
altenae said:
skitron said:
altenae said:
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...

Well for me this list is not the way my lenses deliver in real world.
Would you buy your lens from this list ?
Then you would buy the 70-200 2.8 IS version over the version II !!!!!!!
Look at the 300mm F2.8 version II
I am lost in this lens score.

""I don't bother with the silly "DxOMark" numbers though...""

I only bother to look at the field maps, they are very informative. The numbers are basically meaningless to me.

Think of it this way, the field maps and the graphs are raw data that you interpret. The so called "DxOMark" is their attempt to interpret the field maps and graphs for you.
 
Upvote 0
altenae said:
skitron said:
altenae said:
And then the lens scores.
Do they reflect the real feeling when we use the lenses our self ??

For the dab of stuff I've bought the *plots* definitely do. I don't bother with the silly "DxOMark" numbers though...

Well for me this list is not the way my lenses deliver in real world.
Would you buy your lens from this list ?
Then you would buy the 70-200 2.8 IS version over the version II !!!!!!!
Look at the 300mm F2.8 version II
I am lost in this lens score.

OK I am completely lost on how they score the lenses, because according to their score the Canon 300mm f/2.8L II USM is worse resolution wise than the Canon EF 75-300mm f/4-5.6 IS USM or the Canon EF 300mm f/4L IS USM ... Can anyone having both confirm that ??? ::)
 
Upvote 0
D

dichiaras

Guest
Understanding the lens scores is much more complicated than for the camera scores. Here's how to compare:

- pick two or three lenses WITH the camera you would use: cameras make a huge difference on the results (just look at a 50mm f/1.8 with a 7D and a 5DII)
-go to measurements, resolution, field map, and select one after another the whole range of focal lengths, for maximum aperture: this usually gives you the minimum sharpness (excluding very small apertures)
-repeat the above for fnumber=8 or 11: this usually gives you the maximum sharpness

Than you can look at transmission and the other variables with the same care.

The results are a matter of personal taste and needs: with my Canon 10-22mm I usually shoot landscapes at small apertures, and it performs amazingly. If I look at the Nikon 10-24mm test, in general it might be better (with cameras of the same category, indeed it scores better than the Canon lens), but for my needs is much inferior.
 
Upvote 0
M

Mike Ca

Guest
I think the DxO measurements are accurate and give you good information about the aspects of the camera they are measuring. There is, of course, many other things to take into consideration than just the sensor when picking a camera.

What I have never been able to make sense of is the DxO overall or case usage scores. These numbers don't seem to make a lot of sense to me. I guess part of the problem is they are only rating the sensor, not the camera.

At ISO 100 the D800 has 2.25 stops better DR than the 5D III. If the 5D III had 6 stops of DR and the D800 had 8.25 stops of DR, this would be a huge deal, but the 5D III has 11 stops of DR and that is more DR than any monitor or print can show. The extra two stops of DR will not be seen unless the dynamic range is compressed using single frame HDR adjustment techniques. In the vast majority of photographs you will never see that extra DR. You can increase the DR of a camera like the 5D III using multi frame HDR, so that extra DR is not a critical factor for me.

Professional and good non-professional photographers have learned to work with light, either find good natural light, supplement natural light or create their own light. I recently heard a talk by an old news photographer. He said when he was sent to cover a parade or similar event, his basic rule was find the good light, then find something interesting to photograph in the good light. There are types of photography, like landscape, where that isn't always possible, and those are the photographers most interested in the high dynamic range in their cameras.

DxO is a company that sells software that among other things does single frame HDR adjustments. They talk about an "exclusive HDR mono-image feature" in their software. They have a vested interest in encouraging camera manufactures to increase the DR of their cameras so that this feature becomes more useful. Without HDR like adjustments in post processing, you will never see the difference in the dynamic range of the 5D III and the D800.
 
Upvote 0
1982chris911 said:
OK I am completely lost on how they score the lenses, because according to their score the Canon 300mm f/2.8L II USM is worse resolution wise than the Canon EF 75-300mm f/4-5.6 IS USM or the Canon EF 300mm f/4L IS USM ... Can anyone having both confirm that ??? ::)

I think the main thing with their method for scoring that gives counterintuitive results is that they appear to take the best result across the whole aperture range. They also do some averaging across focal lengths (the 75-300 doesn't peak at the long end). Also, they tend to allow for some degradation across the frame so a lens that is very good in the center with bad corners might get off lightly.

I'll describe the field maps in a bit more depth, I think this will be consistent with what users of these lenses observe.

You see that the 300mm f/2.8 is better wide open than the 300mm f/4. If you stop the 300mm f/2.8 down, it's not even close -- the 300mm f/2.8 is already near its peak by f/4. The 300mm needs to be stopped down to f/8 to be in the same territory.

According to the field map, the 75-300mm at 300mm is pretty horrid across the aperture range though if you stop down to f/11, the middle of the frame isn't too bad. At 100mm or less, corners sharpen up by f/8 and the center is sharp. At 135mm it needs to be stopped down to f/11 and longer than that the corners are bad.
 
Upvote 0
elflord said:
1982chris911 said:
OK I am completely lost on how they score the lenses, because according to their score the Canon 300mm f/2.8L II USM is worse resolution wise than the Canon EF 75-300mm f/4-5.6 IS USM or the Canon EF 300mm f/4L IS USM ... Can anyone having both confirm that ??? ::)

I think the main thing with their method for scoring that gives counterintuitive results is that they appear to take the best result across the whole aperture range. They also do some averaging across focal lengths (the 75-300 doesn't peak at the long end). Also, they tend to allow for some degradation across the frame so a lens that is very good in the center with bad corners might get off lightly.

I'll describe the field maps in a bit more depth, I think this will be consistent with what users of these lenses observe.

You see that the 300mm f/2.8 is better wide open than the 300mm f/4. If you stop the 300mm f/2.8 down, it's not even close -- the 300mm f/2.8 is already near its peak by f/4. The 300mm needs to be stopped down to f/8 to be in the same territory.

According to the field map, the 75-300mm at 300mm is pretty horrid across the aperture range though if you stop down to f/11, the middle of the frame isn't too bad. At 100mm or less, corners sharpen up by f/8 and the center is sharp. At 135mm it needs to be stopped down to f/11 and longer than that the corners are bad.

Well what I mean is only the linepair value as they tested it on the 5d MK II. In this regard they tested the 300 f2.8 II as the worst !!! This is in complete opposition of every other test you find on the internet and I am sure that no one would spend about 8000 US$ for this lens if the other 300mm options would even come close ...
 
Upvote 0
You can see what I mean looking at these side by side test charts:

The First one is the 300 f2.8 II at f4 against the 70-300mm f4 - 5.6 IS at f8 and 300mm. You see that these lenses look like they are not even from same the same planet resolution wise (as expected)

The second one shows the both 70-200mm f2.8 IS versions (I and II) against each other and the newer lens again is much better (also as expected)

So how can DxO test these in in any way differently ???

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=739&Camera=453&Sample=0&FLI=0&API=2&LensComp=358&CameraComp=453&SampleComp=0&FLIComp=4&APIComp=3

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=103&Camera=453&Sample=0&FLI=4&API=0&LensComp=687&CameraComp=453&SampleComp=0&FLIComp=3&APIComp=0
 
Upvote 0
1982chris911 said:
Well what I mean is only the linepair value as they tested it on the 5d MK II. In this regard they tested the 300 f2.8 II as the worst !!! This is in complete opposition of every other test you find on the internet and I am sure that no one would spend about 8000 US$ for this lens if the other 300mm options would even come close ...

Well, please take the time to read my comment which explains why the numbers are the way they are. There is clearly an issue with the way they aggregate the numbers (across the aperture and focal length range) but from the measurements themselves the 300mm f/2.8 has some clear advantages.

I think posters here are fishing for examples of cases where Dxomark's aggregation method is misleading (e.g. this sudden affinity for medium format is a bit hilarious) but they do make their measurements available.

In this case (the D800 or a number of other cameras), it's pretty clear where Canon is falling short on the test - dynamic range at low ISO.
 
Upvote 0
Status
Not open for further replies.