DXO tests the 50 f/1.8 STM

[ahsanford]

FYI if you hadn't seen.

http://www.dxomark.com/Reviews/Canon-EF-50mm-F1.8-STM-lens-review-Thrifty-fifty/Measurements-Very-good-optical-performance

You'll notice the test, like with all DXO lens reviews, was conducted on every body that Canon sells ...Except for the 5DS and 5DS R. (Which we know they own -- they've already published those reviews.)

Someone at DXO is trying super hard to avoid having a good budget lens leap to the top of their absurd ranking system just because we now have a 50 MP sensor. Method #1 was to withhold all new lens testing until an a7R II review could be rushed into place, and now Method #2 appears to be avoiding lens testing at all on the 5DS rigs. Classy!

#DXO #fairandbalanced

- A

 

[Deleted member 375103]

#Thanks #4 #Posting #This.

As for the 5Ds/5Dr, I guess the DxO guys are a little frightened with all those mega pixels stuffed inside a Canon.

 

[ajfotofilmagem]

Someone at DXO is trying super hard to avoid having a good budget lens leap to the top of their absurd ranking system just because we now have a 50 MP sensor. Method #1 was to withhold all new lens testing until an a7R II review could be rushed into place, and now Method #2 appears to be avoiding lens testing at all on the 5DS rigs. Classy!

Maybe DXO read Canon Rumors and have decided that Canon owners are all a bunch of haters so why bother?

If members of CanonRumors deride DXO Mark is one more reason to prove us wrong. Why delay in showing how bad is the Canon lens in a body 50 megapixel?

Perhaps DXO is having a hard time trying to push dynamic range score on the lens ...
You know, right? There can only be one at the top of the mountain, and EXMOR have to make the performance of lenses be better.

 

[JoFT]

For me the results are not reasonable: If I look at my 50mm lenses on a 7D2 with similar pixel pitch to the 5Ds - the f1.8 is way better... I compared it with my EF 50mm f1.4: with a clear winner: the cheaper one..


If someone wants to have a look: http://bit.ly/1OCuFi5


unfortunately the admins do not allow me to embed the photos here ;-(

 

[PhotographyFirst]

It's pretty obvious DXO has an axe to grind with Canon. Anyone know why that is?

With their stupid scoring system, the crappy-plasty-fifty would shoot to the top of their MP scores on a 5Ds. I don't personally care about anything DXO does in terms of how I decide on gear, but there are many people who don't know any better and will take them at face value. They have a very legit and professional looking website, which is deceiving.

Read the title of this lens review for a good laugh, or cry. It get even better as you read the review.

http://www.dxomark.com/Reviews/Canon-EF-S-55-250mm-f-4-5.6-IS-STM-lens-review-Updated-EF-S-telephoto-zoom-no-improvement-on-predecessor

 

[beforeEos Camaras]

It's pretty obvious DXO has an axe to grind with Canon. Anyone know why that is?

With their stupid scoring system, the crappy-plasty-fifty would shoot to the top of their MP scores on a 5Ds. I don't personally care about anything DXO does in terms of how I decide on gear, but there are many people who don't know any better and will take them at face value. They have a very legit and professional looking website, which is deceiving.

Read the title of this lens review for a good laugh, or cry. It get even better as you read the review.

http://www.dxomark.com/Reviews/Canon-EF-S-55-250mm-f-4-5.6-IS-STM-lens-review-Updated-EF-S-telephoto-zoom-no-improvement-on-predecessor

I guess I have to give up in despair all my l lens are no good and my regular lens need to be in the dust bin how was I so naïve on buying canon Sarcasm

wow I have not used that website and thank god I did not, even kens site is more forgiving.

 

[3kramd5]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

 

[PhotographyFirst]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

I wonder how long it took them to test lenses on the D800 when it came out?

It sure didn't take them long to get the sensor score up on their website for the 5Ds. If they had any neutral bias, I can't imagine why they wouldn't be super eager to see how the Canon lenses perform on the 50MP sensors.

I guess DXO is balancing out the universe, since Bryan at TDP has a stiffy for Canon gear and often neglects the other brands in terms of priority. He doesn't bias his test data, but he sure biases his priorities. DXO seems to be the same but with a Nikon stiffy.

 

[chromophore]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

I wonder how long it took them to test lenses on the D800 when it came out?

It sure didn't take them long to get the sensor score up on their website for the 5Ds. If they had any neutral bias, I can't imagine why they wouldn't be super eager to see how the Canon lenses perform on the 50MP sensors.

I guess DXO is balancing out the universe, since Bryan at TDP has a stiffy for Canon gear and often neglects the other brands in terms of priority. He doesn't bias his test data, but he sure biases his priorities. DXO seems to be the same but with a Nikon stiffy.

Maybe you weren't around long enough to remember, but TDP originally was a Canon-only site. Bryan only had Canon gear to use and test. Later on, he expanded into Nikon products, in part because there was so little information on how the two brands would perform under comparatively similar testing conditions. This is why TDP continues to have a heavy emphasis on Canon products. It isn't that Bryan neglects the other brands--it is that he is predominately a Canon shooter and the site was originally intended as a resource for Canon photographers. It was not positioned as a general photography review site--unlike DXO, which purports to be an unbiased source of testing information across all brands, but is very clearly not.

 

[ahsanford]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

I wonder how long it took them to test lenses on the D800 when it came out?

It sure didn't take them long to get the sensor score up on their website for the 5Ds. If they had any neutral bias, I can't imagine why they wouldn't be super eager to see how the Canon lenses perform on the 50MP sensors.

I guess DXO is balancing out the universe, since Bryan at TDP has a stiffy for Canon gear and often neglects the other brands in terms of priority. He doesn't bias his test data, but he sure biases his priorities. DXO seems to be the same but with a Nikon stiffy.

Maybe you weren't around long enough to remember, but TDP originally was a Canon-only site. Bryan only had Canon gear to use and test. Later on, he expanded into Nikon products, in part because there was so little information on how the two brands would perform under comparatively similar testing conditions. This is why TDP continues to have a heavy emphasis on Canon products. It isn't that Bryan neglects the other brands--it is that he is predominately a Canon shooter and the site was originally intended as a resource for Canon photographers. It was not positioned as a general photography review site--unlike DXO, which purports to be an unbiased source of testing information across all brands, but is very clearly not.

Yep. TDP is a Canon resource first and foremost. Love the detail of his reviews and the content of the site.

I only wish Bryan Carnathan would curve his Canon grades to a 'B' and not the 'A' he always seems to give. He very very very rarely has a bone to pick with a Canon product, so I wish he would be a bit more critical. Stronger criticism of Canon products would would help us understand the purchasing tradeoffs better, and a rarer 'A' grade would really stand out as a sensational product.

But as it is right now, everything is great from Canon according to him.

Love his site, love that resource -- don't get me wrong.

- A

 

[chromophore]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

This is complete hearsay, but I seem to remember some time ago that they tried to get Canon to give them some money in exchange for their reviews--i.e., some kind of endorsement. Canon refused. DXO basically retaliated. But again, I emphasize that I have not substantiated this and I make no claims to its veracity.

What is undoubtedly true is that DXO's testing and analysis methods are not statistically sound. I should know: I hold degrees in mathematics and statistics. It's my job. Anyone following my posting history here should take note that I am not mathematically illiterate. I actually have not yet seen any source of information regarding photographic equipment testing that I would regard to be statistically reliable. LensRentals comes very, very close: I would take some issue with the sample sizes, and the way they present some of the charts; I would also recommend a blinded approach to measurement and analysis. DXO, however...picking random numbers out of a hat would be less biased than their results.

 

[ahsanford]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

This is complete hearsay, but I seem to remember some time ago that they tried to get Canon to give them some money in exchange for their reviews--i.e., some kind of endorsement. Canon refused. DXO basically retaliated. But again, I emphasize that I have not substantiated this and I make no claims to its veracity.

What is undoubtedly true is that DXO's testing and analysis methods are not statistically sound. I should know: I hold degrees in mathematics and statistics. It's my job. Anyone following my posting history here should take note that I am not mathematically illiterate. I actually have not yet seen any source of information regarding photographic equipment testing that I would regard to be statistically reliable. LensRentals comes very, very close: I would take some issue with the sample sizes, and the way they present some of the charts; I would also recommend a blinded approach to measurement and analysis. DXO, however...picking random numbers out of a hat would be less biased than their results.

The industry turns a blind eye to stats with testing. LR is relatively above-board there, but other sites which are considered more honest/transparent than DXO -- say, PhotoZone or LensTip -- just test one lens, and often only on one body. That's an inappropriate sample size for the metrics they are reporting on.

The problem with DXO is that, at any level of scrutiny, you can find all sorts of loose ends to their arguments. Their methods, transparency, rating system and interpretation of results are all highly questionable. This forum is littered with comical examples of their mis-deeds to the fact-craving photography community. In particular, I cannot stand their lens reviews -- on their sensor work, they are slightly less irritating and are (in many cases) the only show in town.

- A

 

[3kramd5]

It's pretty obvious DXO has an axe to grind with Canon.

I highly doubt that is the case.

I wonder how long it took them to test lenses on the D800 when it came out?

I don't know. I do know, however, that they haven't tested any lenses on the Sony A7R ii, and there is little indication DXO has an anti-sony bias.

 

[K]

I gave them the benefit of the doubt being a new camera body, and with so many lenses to test and so many samples to collect it isn't something that is done overnight.

However, it has been several months and they've had ample time to put out some numbers and results.

At this point, this is bias by omission.

Being that they are Sonikon Shills --

Yes, they are waiting for high MP Sony/Nikon body results so that their glass will then be ranked high enough to then be covered over by other factors. They have to get close enough to the margin where their other measures can then pick up the slack and rank either equal to or better than Canon. They do this because Canon has a slight edge on optical quality.

It's the same way they arbitrarily rank low ISO DR so heavily - this then picks up the slack for areas.

The more one digs into DXO, the more bias, inconsistency and flawed methodology one finds.


Folks, if you want to compare gear -- SEEING IS BELIEVING. Nothing, nothing, nothing beats analyzing RAW files of test shots on your own computer and and coming to your own conclusions.

For all the hype about competitors to Canon, when I load that RAW file and go 1:1 - I'm sorry, but they are NOT better than Canon. They are all very close and comparable. What I see on the screen does not reflect or equate to the hype and hysteria out there.

 

[jeffa4444]

DXO = questionable testing methods. They would not last a day in the cinematography world.

 

[zim]

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD

 

[PhotographyFirst]

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD

Also strange about the 50 STM. Why is Transmission tested at 1.9 T-Stop, yet they measure 3 stops of vignetting?

 

[rfdesigner]

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD

Also strange about the 50 STM. Why is Transmission tested at 1.9 T-Stop, yet they measure 3 stops of vignetting?

I take it you know Tstop is the light transmission figure... not the focal ratio. it accounts for lossy optics, which suggest 0.1 stops of loss which is really very good if true.

The Vignetting is measured as 2.56 stops at photozone, so 3 stops isn't that far off.

Personally I'd drop the vignetting figure, especially for comarative purposes, and change to "speed in corners".. so a f1.2 lens with 2 stops of vignetting (f3.2) will then beat an F4 lens with 0.5 stops of vignetting (f4.5).. which in reality it would do, comfortably.

 

[JoFT]

For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:


On the 7D2:

• EF 50 STM: 12 MP
• EF 50 1.4: 10 MP
• Sigma 50 Art: 13MP

On the 5DIII

EF 50 STM: 16 MPEF 50 1.4: 17 MPSigma 50 Art: 21MP

On the 5DsR

EF 50 STM: 29 MPEF 50 1.4: 22 MPSigma 50 Art: 40MP

For me this does not make any sense! Or does someone has a reasonable explanation???

 

[Maximilian]

For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:


On the 7D2:

• EF 50 STM: 12 MP
• EF 50 1.4: 10 MP
• Sigma 50 Art: 13MP

On the 5DIII

• EF 50 STM: 16 MP
• EF 50 1.4: 17 MP
• Sigma 50 Art: 21MP

On the 5DsR

• EF 50 STM: 29 MP
• EF 50 1.4: 22 MP
• Sigma 50 Art: 40MP

For me this does not make any sense! Or does someone has a reasonable explanation???

Long answer:
The reason therefore is that DxO makes you believe they are testing lenses.
But they are always testing combinations of lens AND camera body/sensor.

So the lens of course is behaving different when you're using an APS-C sensor (smaller image circle)
or different FF sensors (different MP design).


Short answer:
Forget DxO scores. Go out and shoot photos 8)