DXO tests the 50 f/1.8 STM

[ritholtz]

For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:


On the 7D2:

• EF 50 STM: 12 MP
• EF 50 1.4: 10 MP
• Sigma 50 Art: 13MP

On the 5DIII

• EF 50 STM: 16 MP
• EF 50 1.4: 17 MP
• Sigma 50 Art: 21MP

On the 5DsR

• EF 50 STM: 29 MP
• EF 50 1.4: 22 MP
• Sigma 50 Art: 40MP

For me this does not make any sense! Or does someone has a reasonable explanation???

Long answer:
The reason therefore is that DxO makes you believe they are testing lenses.
But they are always testing combinations of lens AND camera body/sensor.

So the lens of course is behaving different when you're using an APS-C sensor (smaller image circle)
or different FF sensors (different MP design).


Short answer:
Forget DxO scores. Go out and shoot photos 8)

Are these sharpness measurement in terms of Mpix good indicator of performance even for combination. Does it mean 5DSR with 50mm stm always produces sharper pictures compared to cheaper FF camera with expensive 50mm lens? In that case, every one will be spending 5DSR body and cheap glass than spending big on lenses right.

 

[ahsanford]

For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:


On the 7D2:

• EF 50 STM: 12 MP
• EF 50 1.4: 10 MP
• Sigma 50 Art: 13MP

On the 5DIII

EF 50 STM: 16 MPEF 50 1.4: 17 MPSigma 50 Art: 21MP

On the 5DsR

EF 50 STM: 29 MPEF 50 1.4: 22 MPSigma 50 Art: 40MP

For me this does not make any sense! Or does someone has a reasonable explanation???

If the same single lens is being used in all three tests, the resolution values should roughly scale with the sensor (the 7D2 brings a crop wrinkle to that, however).

So these inconsistent results as you go from crop 20 MP, FF 22 MP, to FF 50 MP could be due to:

• A different lens was used on one camera than on the other.
• The lenses weren't properly calibrated on each body
• The same lens was used, but it's been a few years since the 5D3 test and it took some wear and tear over that time
• [Insert myriad reasons here]

The point is: no lens testing site is perfect. But ALL of them would benefit from testing multiple copies of a lens like LR usually does. Why we hang our hat on a single lens' performance is nuts.

- A

 

[PhotographyFirst]

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD

Also strange about the 50 STM. Why is Transmission tested at 1.9 T-Stop, yet they measure 3 stops of vignetting?

I take it you know Tstop is the light transmission figure... not the focal ratio. it accounts for lossy optics, which suggest 0.1 stops of loss which is really very good if true.

The Vignetting is measured as 2.56 stops at photozone, so 3 stops isn't that far off.

Personally I'd drop the vignetting figure, especially for comarative purposes, and change to "speed in corners".. so a f1.2 lens with 2 stops of vignetting (f3.2) will then beat an F4 lens with 0.5 stops of vignetting (f4.5).. which in reality it would do, comfortably.

I know what I am saying.

Vignetting directly affects the transmission value. A lens that has heavy vignetting, should have a much higher t-stop value compared to aperture value. Unless of course, the center of the image area somehow is much brighter than the f1.8 aperture.

My beef with DXO is why they list the lens as having nearly perfect transmission when it has not even close to perfect vignetting control.

 

[old-pr-pix]

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD

Also strange about the 50 STM. Why is Transmission tested at 1.9 T-Stop, yet they measure 3 stops of vignetting?

I take it you know Tstop is the light transmission figure... not the focal ratio. it accounts for lossy optics, which suggest 0.1 stops of loss which is really very good if true.

...

I know what I am saying.

Vignetting directly affects the transmission value. A lens that has heavy vignetting, should have a much higher t-stop value compared to aperture value. Unless of course, the center of the image area somehow is much brighter than the f1.8 aperture.

My beef with DXO is why they list the lens as having nearly perfect transmission when it has not even close to perfect vignetting control.

I'm not going to defend DxO, but the calculation of T-stop value is based on the transmission of the lens on axis. I.E. through the center of the image area. Hence, vignetting will not impact the T-stop value unless it is so severe as to impinge on the central image area.

One big problem I have with any DxO measurement is that they never fully disclose methodology, accuracy or measurement error and they only test one copy. I suspect in many cases the differences between two manufacturer's similar items tested may well be less than the +/- error of the measurement. By algorithm, DxO then takes these statistically insignificant differences and magnifies them by various weighting factors resulting in what appears to be significant difference.

If we knew DxO's actual algorithm we could take some of the information that LensRentals finds in terms of copy-2-copy variation and plug it in to determine how wildly DxO's single number values would change.

 

[PhotographyFirst]

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD

Also strange about the 50 STM. Why is Transmission tested at 1.9 T-Stop, yet they measure 3 stops of vignetting?

I take it you know Tstop is the light transmission figure... not the focal ratio. it accounts for lossy optics, which suggest 0.1 stops of loss which is really very good if true.

...

I know what I am saying.

Vignetting directly affects the transmission value. A lens that has heavy vignetting, should have a much higher t-stop value compared to aperture value. Unless of course, the center of the image area somehow is much brighter than the f1.8 aperture.

My beef with DXO is why they list the lens as having nearly perfect transmission when it has not even close to perfect vignetting control.

I'm not going to defend DxO, but the calculation of T-stop value is based on the transmission of the lens on axis. I.E. through the center of the image area. Hence, vignetting will not impact the T-stop value unless it is so severe as to impinge on the central image area.

One big problem I have with any DxO measurement is that they never fully disclose methodology, accuracy or measurement error and they only test one copy. I suspect in many cases the differences between two manufacturer's similar items tested may well be less than the +/- error of the measurement. By algorithm, DxO then takes these statistically insignificant differences and magnifies them by various weighting factors resulting in what appears to be significant difference.

If we knew DxO's actual algorithm we could take some of the information that LensRentals finds in terms of copy-2-copy variation and plug it in to determine how wildly DxO's single number values would change.

Like I said, I don't know what I am saying.

Thanks for the info, I was not aware that the t-stop wasn't the average transmission of the whole frame.

 

[Sabaki]

I have my bodies and lenses. Last thing I think of when I'm out in the field is whether my kit is qualified according to DXO or not.

Canon is a superb brand, do we really need a dubious body to qualify that for us?

 

[Maximilian]

I have my bodies and lenses. Last thing I think of when I'm out in the field is whether my kit is qualified according to DXO or not.

Canon is a superb brand, do we really need a dubious body to qualify that for us?

As I said before:

Short answer:
Forget DxO scores. Go out and shoot photos 8)