Days since DXO sensor score of 5DS = 112. Number of lenses re-tested = 0.

[ahsanford]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

Why do people care about the number of re-tested lenses?

Because DXO states that identical non-exclusive lenses (Sigma Art, Zeiss Otus) are 'best' or 'disappointing' depending on how many pixels are sitting behind them.

I don't want to see EF mount glass heretofore ranked as average or poor suddenly leap to the top of their rankings now that 50 MP are sitting behind them... but I do want to see DXO try to explain that. :

Keep in mind that it's not fanboyism motivating my ire nearly so much as the gall they have to rank lenses across different sensors and mounts. That needs to end. They should report data and let people interpret it.

- A

 

[ahsanford]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

Why do people care about the number of re-tested lenses?

With the number of insults hurled in the direction of DxO it is surprising to see that people care about this.

It's because they marry the data to bodies. If they tested them on an optical bench re-testing would be irrelevant.

Exactly, but why would a sensor testing company run a test that sensors have nothing to do with?

That would be silly.

- A

 

[Don Haines]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

Why do people care about the number of re-tested lenses?

With the number of insults hurled in the direction of DxO it is surprising to see that people care about this.

It's because their lens testing is not lens testing....

 

[StudentOfLight]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

Yep. This comment came up earlier: if it's in the software, they've tested the lens on that body.

Do we actually know this for sure? I can't see any reason they couldn't interpolate corrections from one body to another.

Given how they come up with "print size," I wouldn't be surprised if they use a math model (perhaps using linear inter- or extrapolation). But that would take almost no time at all, so again: where's the beef?

Without measurements the strength of the AA filter effect is unknown.

e.g. The 5D-III and 1Ds-III have quite similar resolution but the 5D-III produces a sharper image due to the weaker AA filter. Similarly, comparing 24MP Nikon D3X and the D750, the D750 produces sharper images.

But if you characterize the effect of AA when testing a body and relate it via some coefficient in your math model, you can wave your hands and say "here's the number" when publishing lens data.

I just checked and there are a handful of lens/body combinations that are available in the DxO Optics Module database that are marked as no measurements available in the DxO lens score database. I think that it's very likely that the lens module are generated by interpolated data and honestly I can't see any good reason why that wouldn't be a perfectly reasonable approach.

I believe that resolution is subject to a quantization error. Interpolation is one thing, extrapolation is another. 50MP is higher than anything previously tested and Canon lenses have not been tested with a AA-free/AA-cancellation sensor so no hard data to base potential extrapolation on.

I think this is why a large number of lenses need to be re-tested. Some of the more exotic lenses might be difficult to get ahold of so might not necessarily be retested due to backorders or non-stocking in rental houses, but popular lenses like the 2.8 zooms and the classic primes should be re-tested and not necessarily left up to interpolation/extrapolation.

 

[ahsanford]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.


Good news, gang: DXO finally got to testing a brand new lens, the Sigma 24-35 f/2 Art!

http://www.dxomark.com/Reviews/Sigma-24-35mm-F2-DG-HSM-Art-Canon-mount-review-Redefining-expectations

I'm so glad they tested it on their highest resolution EF-mount rig, the 5D Mark III.

They are still withholding 5DS / 5DS R lens testing. Wow.

#dxo #fairandbalanced

- A

 

[zim]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

Good news, gang: DXO finally got to testing a brand new lens, the Sigma 24-35 f/2 Art!

http://www.dxomark.com/Reviews/Sigma-24-35mm-F2-DG-HSM-Art-Canon-mount-review-Redefining-expectations

I'm so glad they tested it on their highest resolution EF-mount rig, the 5D Mark III.

They are still withholding 5DS / 5DS R lens testing. Wow.

#dxo #fairandbalanced

- A

Maybe they are waiting for a new sensor 'fabrication' line to be installed

 

[StudentOfLight]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

Good news, gang: DXO finally got to testing a brand new lens, the Sigma 24-35 f/2 Art!

http://www.dxomark.com/Reviews/Sigma-24-35mm-F2-DG-HSM-Art-Canon-mount-review-Redefining-expectations

I'm so glad they tested it on their highest resolution EF-mount rig, the 5D Mark III.

They are still withholding 5DS / 5DS R lens testing. Wow.

#dxo #fairandbalanced

- A

Nikon D810:
a) Sensor test published: 2014/07/24
b) Lens re-testing published: 2014/10/29

(b)-(a) = 97 days

#dxo #fairlyclose #notmuchlonger

 

[StudentOfLight]

I'd be happy to wait a couple more weeks if they are also including the new Tamron lenses as well as the kickass new 35mm f/1.4 L II in the rating system. I think the new 35L II will dethrone the Otus lenses with its phenomenal wide open performance.

Hopefully the 35L II won't be to badly hamstrung by the fact that it closes down to f/22 vs the Otus lenses that only close down to f/16. No doubt there is a difference between averaging performance between "f/1.4-f/22" vs "f/1.4-f/16".

 

[ahsanford]

I'd be happy to wait a couple more weeks if they are also including the new Tamron lenses as well as the kickass new 35mm f/1.4 L II in the rating system. I think the new 35L II will dethrone the Otus lenses with its phenomenal wide open performance.

Hopefully the 35L II won't be to badly hamstrung by the fact that it closes down to f/22 vs the Otus lenses that only close down to f/16. No doubt there is a difference between averaging performance between "f/1.4-f/22" vs "f/1.4-f/16".

I don't care about dethroning 'bests' at a site whose methods I do not respect. I just need one staple Canon pro lens, say the 70-200 f/2.8L IS II, to be retested. We all know that is an excellent lens, but sure, there are better lenses out there.

The issue, of course, is having the gall to globally rank a lens, which is folly. DXO should report its data, throw a ninja smoke bomb and disappear to allow photographers to interpret the results. But ranking lenses so comically heavily weighted on sensor resolution is nuts.

I want DXO to have to explain why a lens that previously got a 23 now gets a 45 because there are now more pixels sitting behind it. That's really all I want. I'm not a kid who needs to see Canon ranked above everyone else -- that's nonsense. I just want to DXO have to live with their wretched lens rating system and try to explain it away.

- A

 

[3kramd5]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

They are still withholding 5DS / 5DS R lens testing. Wow.

#dxo #fairandbalanced

- A

Meanwhile, the anti-Sony bias continues as well (no love for the A7R2).

Or they just have a weird operation

 

[StudentOfLight]

I'd be happy to wait a couple more weeks if they are also including the new Tamron lenses as well as the kickass new 35mm f/1.4 L II in the rating system. I think the new 35L II will dethrone the Otus lenses with its phenomenal wide open performance.

Hopefully the 35L II won't be to badly hamstrung by the fact that it closes down to f/22 vs the Otus lenses that only close down to f/16. No doubt there is a difference between averaging performance between "f/1.4-f/22" vs "f/1.4-f/16".

I don't care about dethroning 'bests' at a site whose methods I do not respect. I just need one staple Canon pro lens, say the 70-200 f/2.8L IS II, to be retested. We all know that is an excellent lens, but sure, there are better lenses out there.

The issue, of course, is having the gall to globally rank a lens, which is folly. DXO should report its data, throw a ninja smoke bomb and disappear to allow photographers to interpret the results. But ranking lenses so comically heavily weighted on sensor resolution is nuts.

I want DXO to have to explain why a lens that previously got a 23 now gets a 45 because there are now more pixels sitting behind it. That's really all I want. I'm not a kid who needs to see Canon ranked above everyone else -- that's nonsense. I just want to DXO have to live with their wretched lens rating system and try to explain it away.

- A

It's not hard to explain, the answer lies in your own question. It gets a higher score because it can reproduce a test chart in higher quality with the given increased number of pixels.

 

[StudentOfLight]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

They are still withholding 5DS / 5DS R lens testing. Wow.

#dxo #fairandbalanced

- A

Meanwhile, the anti-Sony bias continues as well (no love for the A7R2).

Or they just have a weird operation

A7s:
Sensor Test published: 17-June-2014 (499 days ago)
http://www.dxomark.com/Reviews/Sony-Alpha-7S-sensor-review-New-low-light-champ

Lens tests published: Not yet
http://www.dxomark.com/Cameras/Sony/A7S---Lenses-tested

 

[ahsanford]

Re: Days since DXO sensor score of 5DS = 86. Number of lenses re-tested = 0.

They are still withholding 5DS / 5DS R lens testing. Wow.

#dxo #fairandbalanced

- A

Meanwhile, the anti-Sony bias continues as well (no love for the A7R2).

Or they just have a weird operation

A7s:
Sensor Test published: 17-June-2014 (499 days ago)
http://www.dxomark.com/Reviews/Sony-Alpha-7S-sensor-review-New-low-light-champ

Lens tests published: Not yet
http://www.dxomark.com/Cameras/Sony/A7S---Lenses-tested

Wow. I knew about the A7R II having a long queue time the 5DS is seeing, but had no idea first gen A7 hardware hadn't been tested yet. Perhaps they didn't want a 12 MP rig dragging down the standing of the Zeiss lenses?

- A

 

[Larsskv]

I'd be happy to wait a couple more weeks if they are also including the new Tamron lenses as well as the kickass new 35mm f/1.4 L II in the rating system. I think the new 35L II will dethrone the Otus lenses with its phenomenal wide open performance.

Hopefully the 35L II won't be to badly hamstrung by the fact that it closes down to f/22 vs the Otus lenses that only close down to f/16. No doubt there is a difference between averaging performance between "f/1.4-f/22" vs "f/1.4-f/16".

I don't care about dethroning 'bests' at a site whose methods I do not respect. I just need one staple Canon pro lens, say the 70-200 f/2.8L IS II, to be retested. We all know that is an excellent lens, but sure, there are better lenses out there.

The issue, of course, is having the gall to globally rank a lens, which is folly. DXO should report its data, throw a ninja smoke bomb and disappear to allow photographers to interpret the results. But ranking lenses so comically heavily weighted on sensor resolution is nuts.

I want DXO to have to explain why a lens that previously got a 23 now gets a 45 because there are now more pixels sitting behind it. That's really all I want. I'm not a kid who needs to see Canon ranked above everyone else -- that's nonsense. I just want to DXO have to live with their wretched lens rating system and try to explain it away.

- A

Lets face it. DXO will never publish anything that supports Canons superiority, not in lenses or otherwise. Dxo are anti-Canon. All their tests and the way they angle their articles support that. Hope I'm proven wrong. Soon.

 

[3kramd5]

Wow. I knew about the A7R II having a long queue time the 5DS is seeing, but had no idea first gen A7 hardware hadn't been tested yet. Perhaps they didn't want a 12 MP rig dragging down the standing of the Zeiss lenses?

- A

I presume it's merely not on their priorities list. Assigning motive or intent assumes FAR too much.

Lets face it. DXO will never publish anything that supports Canons superiority, not in lenses or otherwise. Dxo are anti-Canon. All their tests and the way they angle their articles support that. Hope I'm proven wrong. Soon.

From: Which lenses should you choose for your Canon EOS 5D Mark III? By Kevin Carter - Wednesday April 03 2013

When comparing the huge volume of data accumulated over measuring 147 lenses, one very surprising result was revealed. The average sharpness scores of the Canon EOS 5D Mark III matched the Nikon D800 and if the results were based solely on the mean average, the Canon actually out-performed the Nikon.

When using specific lenses (such as the new Sigma 35mm f/1.4 DG HSM A) the Nikon can out resolve the Canon. However, taken as a whole, the statistics reveal the EOS 5D MK III is capable of similar sharpness and of achieving a close DxOMark camera/lens score to the Nikon D800. Moreover, that’s despite the latter camera’s 60% extra pixel count.

 

[ahsanford]

And on the 144th day, DXO rested. EF lenses have been retested, but only on the 5DS R for some reason.

Details and some early thoughts here:
http://www.canonrumors.com/forum/index.php?topic=28477

- A

 

[3kramd5]

What's really interesting is lenses such as the Otus 55/1.4.

46/37 on 5DsR, 50/33 on D800.

This suggests that the scoring of lenses is not nearly as simple as it is for camera sensors.

Other than sharpness, vignetting and transmission are the differences. If the lenses are made the same then I wonder if this is to do with the filters that are stacked up in front of the sensor working better with Canon/Otus than Nikon/Otus (ref. MILC cameras, mount converters and IQ loss due to angle of light hitting sensor.)

DxO includes sensor dynamic range as a parameter in their lens scoring algorithm, and tests in a light-environment very friendly to the D800. Dollars to donuts that's the difference.

Refer to this writeup:

...while the Canon 500mm mounted on 5D Mark III is sharper optically than the Nikon model mounted on D800, at the light levels used for DxOMark score (1/60, 150 Lux), the excellent dynamic range of the Nikon D800 sensor helps it improve the DxO Mark Score and accounts for the level-pegging. In lower light levels, the Canon would have the advantage

Also, while the aggregate score is dependent on sensor performance, the sharpness score is not.

 

[ahsanford]

...
DxO includes sensor dynamic range as a parameter in their lens scoring algorithm, and tests in a light-environment very friendly to the D800. Dollars to donuts that's the difference.
...
Also, while the aggregate score is dependent on sensor performance, the sharpness score is not.

Which is why DxO have said that you should only compare lenses and their scores when tested on the same body.

Dilbert, DXO is not original with that statement. Everyone says that (and I believe everyone knows that).

But they still give an overall score and they still rank the lenses. That's ultimately what needs to stop. I'd hate them much less if they did.

- A