Why so much trust in DXO.

[altenae]

So ONE company in the world with there numbers are the only right numbers ?????

Other review sites show other numbers and yet the only true numbers are from DXO ?????

Why ?

If you want real trustable numbers you need more than ONE instance to come up with numbers.

And still there are only numbers

 

[elflord]

You can see what I mean looking at these side by side test charts:

The First one is the 300 f2.8 II at f4 against the 70-300mm f4 - 5.6 IS at f8 and 300mm. You see that these lenses look like they are not even from same the same planet resolution wise (as expected)

The second one shows the both 70-200mm f2.8 IS versions (I and II) against each other and the newer lens again is much better (also as expected)

So how can DxO test these in in any way differently ???

The website you link to doesn't attempt to summarize resolution in a single number. You're not comparing apples and apples You're comparing a test chart crop at a specific focal length/aperture with an aggregate summary statisti, so it's not terribly surprising that the two aren't at all equivalent.

As I wrote, you can look at the field maps in DXO (e.g. see their test results across the aperture range, focal length range and frame). Click on "Measurements", then "resolution" then "field map". Generally the field maps aren't too surprising. Where they are surprising, that's reason to step back and absorb what the results mean instead of hastily dismissing any test that doesn't produce the numbers we want.

Back to the 5DIII, Canon haven't been the leaders in sensor technology for some time and DR at low ISO has been wanting in most of their newer cameras. I really don't understand why everyone is getting so bent out of shape (and fishing for some really far fetched criticisms of DxO ) over these test results.

 

[elflord]

So ONE company in the world with there numbers are the only right numbers ?????

Other review sites show other numbers and yet the only true numbers are from DXO ?????

Why ?

If you want real trustable numbers you need more than ONE instance to come up with numbers.

And still there are only numbers

I don't see anyone claiming anything of the sort. I see a lot of very hasty attempts to dismiss DxOMark because they don't like the latest test scores.

 

[1982chris911]

You can see what I mean looking at these side by side test charts:

The First one is the 300 f2.8 II at f4 against the 70-300mm f4 - 5.6 IS at f8 and 300mm. You see that these lenses look like they are not even from same the same planet resolution wise (as expected)

The second one shows the both 70-200mm f2.8 IS versions (I and II) against each other and the newer lens again is much better (also as expected)

So how can DxO test these in in any way differently ???

The website you link to doesn't attempt to summarize resolution in a single number. You're not comparing apples and apples You're comparing a test chart crop at a specific focal length/aperture with an aggregate summary statisti, so it's not terribly surprising that the two aren't at all equivalent.

As I wrote, you can look at the field maps in DXO (e.g. see their test results across the aperture range, focal length range and frame). Click on "Measurements", then "resolution" then "field map". Generally the field maps aren't too surprising. Where they are surprising, that's reason to step back and absorb what the results mean instead of hastily dismissing any test that doesn't produce the numbers we want.

Back to the 5DIII, Canon haven't been the leaders in sensor technology for some time and DR at low ISO has been wanting in most of their newer cameras. I really don't understand why everyone is getting so bent out of shape (and fishing for some really far fetched criticisms of DxO ) over these test results.

What I see when I look at the testmaps of DxO is that the 300mm f2.8 II is generally worse (resolution wise) than the zoom 70-300 F4-5.6 IS.

http://www.dxomark.com/index.php/Lenses/Camera-Lens-Database/Canon/Canon-EF-300mm-F28L-IS-II-USM/%28camera%29/483[url]]]http://www.dxomark.com/index.php/Lenses/Camera-Lens-Database/Canon/Canon-EF-300mm-F28L-IS-II-USM/%28camera%29/483[url]


http://www.dxomark.com/index.php/Lenses/Camera-Lens-Database/Canon/EF70-300mm-f-4-5.6-IS-USM/%28camera%29/436[/url]

Excuse me but are they kidding ??? These result are a contradiction to about everything you see when you look at pictures taken with those two lenses, test charts, MTF charts etc ... IT IS JUST NOT POSSIBLE ... The 300mm f2.8 II is the sharpest and best lens resolution wise that was ever produced by Canon (by even quite a margin to the 400mm f2.8 II) ... So how can you come up with these numbers then ?

 

[altenae]

So ONE company in the world with there numbers are the only right numbers ?????

Other review sites show other numbers and yet the only true numbers are from DXO ?????

Why ?

If you want real trustable numbers you need more than ONE instance to come up with numbers.

And still there are only numbers

I don't see anyone claiming anything of the sort. I see a lot of very hasty attempts to dismiss DxOMark because they don't like the latest test scores.

This has nothing to do with the latest scores.
My point is that these are numbers published by ONE company only.

 

[psolberg]

So ONE company in the world with there numbers are the only right numbers ?????

Other review sites show other numbers and yet the only true numbers are from DXO ?????

Why ?

If you want real trustable numbers you need more than ONE instance to come up with numbers.

And still there are only numbers

I don't see anyone claiming anything of the sort. I see a lot of very hasty attempts to dismiss DxOMark because they don't like the latest test scores.

yes, which is silly because other reviews have reached the same conclusion: The 5DmkIII does not offer significant gains for the price, and the D800 easily edges it out in dynamic range and low ISO noise. The whole debate seems to have shifted now to lenses. who cares? The 5DmkIII score has NOTHING to do with lenses. While I agree that no one site should be taken as the final word, the feeling that the 5DmkIII isn't all that it was hyped to be has been echoed again and again. It is time to move on. seriously, it may not be the best camera, but it is a really good one. Isn't that enough?

 

[elflord]

What I see when I look at the testmaps of DxO is that the 300mm f2.8 II is generally worse (resolution wise) than the zoom 70-300 F4-5.6 IS.

In your earlier post, it was the 75-300mm (which clearly does worse). Now you're fishing for a different lens ? This is data dredging at its worst. You can convince yourself of anything if you selectively fish for data that confirms your hypothesis and ignore data that falsifies it.

In this case (with the other lens), it looks like the MTF is comparable to the 300mm, but it's "wide open" at f/5.6. The 300mm f/2.8 is already good by f/4. Who knows, they could have gotten a good copy.

Excuse me but are they kidding ???

Who are you kidding ? This faux outrage is the result of a data dredging exercise you undertook because you don't like the test scores that DxO gave to Canon's new flagship, but this lens talk is a big red herring. Complaining about DxO doesn't alter the fact that Canon seem to have lost the edge in sensor technology.

 

[elflord]

This has nothing to do with the latest scores.
My point is that these are numbers published by ONE company only.

So what ? no-one suggested otherwise.

 

[altenae]

This has nothing to do with the latest scores.
My point is that these are numbers published by ONE company only.

So what ? no-one suggested otherwise.

Nevermind.

 

[1982chris911]

What I see when I look at the testmaps of DxO is that the 300mm f2.8 II is generally worse (resolution wise) than the zoom 70-300 F4-5.6 IS.

In your earlier post, it was the 75-300mm (which clearly does worse). Now you're fishing for a different lens ? This is data dredging at its worst. You can convince yourself of anything if you selectively fish for data that confirms your hypothesis and ignore data that falsifies it.

In this case (with the other lens), it looks like the MTF is comparable to the 300mm, but it's "wide open" at f/5.6. The 300mm f/2.8 is already good by f/4. Who knows, they could have gotten a good copy.

Excuse me but are they kidding ???

Who are you kidding ? This faux outrage is the result of a data dredging exercise you undertook because you don't like the test scores that DxO gave to Canon's new flagship, but this lens talk is a big red herring. Complaining about DxO doesn't alter the fact that Canon seem to have lost the edge in sensor technology.

Which was a typo, I just entered the wrong lens name... and btw the newer 70-300 is rated higher so the typo was to my disadvantage ... the comparison is still the same faulty one (you can go through all resolution charts on the interactive comparison I linked) I assure you there is not one possible combination where the 70-300 IS is better than the 300 f2.8 II ...

However if you seriously suggest that the 300 f2.8 II compares to the 70-300 or 75-300 and DXO does that it is nothing but a plain joke. I personally have never seen any serious pro photographer at any sports event even in the brightest daylight at noon using one of the other two lenses (which according to DXO should deliver the same or better results) ...

Have you ever used both ? I guess you would also question those results if you had ...

 

[elflord]

In your earlier post, it was the 75-300mm (which clearly does worse). Now you're fishing for a different lens ? This is data dredging at its worst. You can convince yourself of anything if you selectively fish for data that confirms your hypothesis and ignore data that falsifies it.

In this case (with the other lens), it looks like the MTF is comparable to the 300mm, but it's "wide open" at f/5.6. The 300mm f/2.8 is already good by f/4. Who knows, they could have gotten a good copy.

Excuse me but are they kidding ???

Who are you kidding ? This faux outrage is the result of a data dredging exercise you undertook because you don't like the test scores that DxO gave to Canon's new flagship, but this lens talk is a big red herring. Complaining about DxO doesn't alter the fact that Canon seem to have lost the edge in sensor technology.

Which was a typo, I just entered the wrong lens name... and btw the newer 70-300 is rated higher so the typo was to my disadvantage ... the comparison is still the same faulty one
[/quote]

No, the 75-300mm (what you wrote originally) is as I described in my earlier post (poor unless stopped down considerably and pretty much weak in the edges across the aperture/FL range)

However if you seriously suggest that the 300 f2.8 II compares to the 70-300 or 75-300 and DXO does that

I haven't used either lens, so I wouldn't suggest any such thing. I have seen some cases where cheaper lenses do reasonably well in their MTF scores, though at much smaller apertures than good lenses. Generally, if I saw a test result that seemed a bit surprising, I'd check other test results.

it is nothing but a plain joke. I personally have never seen any serious pro photographer at any sports event even in the brightest daylight at noon using one of the other two lenses (which according to DXO should deliver the same or better results) ...

Actually, DxO include a "sports" use case score, and the score they assign does not suggest that the two lenses should deliver the same or better results. There's more to evaluating a lens than MTF at f/11.

But I really don't see the relevance of this. Again, I believe this is just a data dredging exercise that Canon fans have undertaken because they are unhappy with the 5DIII test scores. The complaint is misdirected -- they should be complaining to Canon.

 

[1982chris911]

But I really don't see the relevance of this. Again, I believe this is just a data dredging exercise that Canon fans have undertaken because they are unhappy with the 5DIII test scores. The complaint is misdirected -- they should be complaining to Canon.

As far as I can see this here is about DXO and how they do their tests (number ratings) and not limited to the academic poor sensor quality of the 5d MKIII... I guess there are 3-4 other threads in this forum here parallel to this one where people who never used the MK III on their own are doing this... Btw I own both the MKII and MK III so I can at least tell that the MKIII is much better IQ wise... and I have also tested most of lenses myself so I know that the 300mm f2.8 II is the about the highest resolving lens I have ever seen.

 

[elflord]

As far as I can see this here is about DXO and how they do their tests (number ratings) and not limited to the academic poor sensor quality of the 5d MKIII..

But why the sudden data dredging exercise against DxO ? Did the Canon fans suddenly discover an affinity for medium format ? Or is it because they don't like the new test result ?

If it's "academic", why are the fans so angry ?

I guess there are 3-4 other threads in this forum here parallel to this one where people who never used the MK III on their own are doing this...

I haven't used it because I'm very happy with the 5DII. It's not on the cutting edge of sensor tech but given the incremental progress in sensor performance it's pretty close and a damn good camera. I think there is a tendency in rumors forums to have unrealistic expectations of technological progress -- predictions of 1-2 stop improvements here which is clearly impossible given the trajectory of sensor technology. It's not clear to me what I stand accused of doing.

Generally, I prefer reviews were the tester tries to dig deeper and figure out what's going on with their results, a "review" that consists of some data but no explanation or insight would carry less weight to me, so I'd tend to give tdp's (and photozones and lenstips) reviews of the 70-300 more weight than DxO. However, I wouldn't go so far as to say that this "proves" that DxO are incompetent or untrustworthy.

What bothers me about the attacks on DxO is that the "critics" appear to be lashing out because they don't like the test results. Most of the criticism of DxO in these threads has been not only uninformed, it appears that most of the critics go out of their way to not understand the measurements.

 

[1982chris911]

As far as I can see this here is about DXO and how they do their tests (number ratings) and not limited to the academic poor sensor quality of the 5d MKIII..

But why the sudden data dredging exercise against DxO ? Did the Canon fans suddenly discover an affinity for medium format ? Or is it because they don't like the new test result ?

If it's "academic", why are the fans so angry ?

I guess there are 3-4 other threads in this forum here parallel to this one where people who never used the MK III on their own are doing this...

I haven't used it because I'm very happy with the 5DII. It's not on the cutting edge of sensor tech but given the incremental progress in sensor performance it's pretty close and a damn good camera. I think there is a tendency in rumors forums to have unrealistic expectations of technological progress -- predictions of 1-2 stop improvements here which is clearly impossible given the trajectory of sensor technology. It's not clear to me what I stand accused of doing.

Generally, I prefer reviews were the tester tries to dig deeper and figure out what's going on with their results, a "review" that consists of some data but no explanation or insight would carry less weight to me, so I'd tend to give tdp's (and photozones and lenstips) reviews of the 70-300 more weight than DxO. However, I wouldn't go so far as to say that this "proves" that DxO are incompetent or untrustworthy.

What bothers me about the attacks on DxO is that the "critics" appear to be lashing out because they don't like the test results. Most of the criticism of DxO in these threads has been not only uninformed, it appears that most of the critics go out of their way to not understand the measurements.

Well I just question the test results for the lenses as these are as I said in contradiction to everything experienced in the real world so I don't know how DXO can come up with such measurements ...

Regarding the disappointed fans:
I don't know why ppl are unhappy with the 5D MK III, most of these unhappy ones as far as I see either don't have the camera and are only referring to test results or are only looking at the D800 without much real world experience (btw I would not want to have one after using it for a short time) ...

About everyone who has the 5D MKIII now really loves it (minus maybe 1-2% which always have sth. to complain, but I think that is normal and in some cases these are of ppl with real technical problems on their copy) ... So I don't see many actual owners complaining about the camera and imo there is also not much to complain when I use it and compare it to the MKII it is better in about every regard ...

 

[kirispupis]

But I really don't see the relevance of this. Again, I believe this is just a data dredging exercise that Canon fans have undertaken because they are unhappy with the 5DIII test scores. The complaint is misdirected -- they should be complaining to Canon.

As far as I can see this here is about DXO and how they do their tests (number ratings) and not limited to the academic poor sensor quality of the 5d MKIII... I guess there are 3-4 other threads in this forum here parallel to this one where people who never used the MK III on their own are doing this... Btw I own both the MKII and MK III so I can at least tell that the MKIII is much better IQ wise... and I have also tested most of lenses myself so I know that the 300mm f2.8 II is the about the highest resolving lens I have ever seen.

I absolutely agree with you here on the 5D2 vs. 5D3. What I find funny/sad is how many people there are who prefer to trust a number vs. reviews from people who have actually used both cameras. DXO is a joke. They are like that computer in Hitchhiker's Galaxy that spits out the number 42.

 

[Loswr]

They are like that computer in Hitchhiker's Galaxy that spits out the number 42.

Wait, wait, Douglas Adams have the 5DIII a score of 42?!?

 

[kirispupis]

They are like that computer in Hitchhiker's Galaxy that spits out the number 42.

Wait, wait, Douglas Adams have the 5DIII a score of 42?!?

If Douglas Adams did, I would trust it more than I trust the DXO score.

 

[Neeneko]

About everyone who has the 5D MKIII now really loves it (minus maybe 1-2% which always have sth. to complain, but I think that is normal and in some cases these are of ppl with real technical problems on their copy) ... So I don't see many actual owners complaining about the camera and imo there is also not much to complain when I use it and compare it to the MKII it is better in about every regard ...

Well, there is a bit of a bias in there. In general the only people who are going to jump on a high cost new body like the 5D3 are people who are going to be pre-disposed to want it, either though a good matchup of use cases or rabbid fanboyism... and both of those groups are going to be pretty happy with it.

The people who are likely to be unhappy with a 5D3 or match up poorly with the use cases have generally not plunked down the kilobucks needed to buy from this first batch. Thus most of the negative views of the 5D3 are, of course, going to come from people who have not bought one. Would you plop down thousands of dollars on a brand new body that you were not excited about or had reservations?

 

[Loswr]

Thus most of the negative views of the 5D3 are, of course, going to come from people who have not bought one. Would you plop down thousands of dollars on a brand new body that you were not excited about or had reservations?

Meaning...it's being bashed by people who haven't used it. Give those 'opinions' the weight they deserve - less than a feather.

 

[Neeneko]

Meaning...it's being bashed by people who haven't used it. Give those 'opinions' the weight they deserve - less than a feather.

It also means that the 'opinions' of the people who like it are worth about as much.