DXO Test on 16-35mm from sony

[xps]

DXO Tested the Sony 16-35 GM lens:
https://www.dxomark.com/sony-fe-16-35mm-f28-gm-lens-review-highest-rated-wide-angle-zoom/

How can you explain the big gap in sharpness compared to the 16-35 III from Canon?

 

[Larsskv]

My 16-35 LIII is way sharper than any of the four 24-70LII lenses I’ve had, yet DXO says the 24-70 scores 32 in sharpness on the 5Dsr, and the 16-35LIII scores 24.

My 16-35 LIII is as sharp as my 35LII, maybe sharper at f5.6. The DXO score does not correspond with my experience.

 

[Mt Spokane Photography]

The first law of lens testing says not to compare tests on one model with tests on a different model camera. That law does not apply to France, of course, they have their own set of rules.

There is also the small matter if using a undefined number to rate sharpness rather than the standard resolution used by the rest of the world. What is Perceptual Mpix? As far as I know, its a rating that is determined by a secret formula that no one else knows or can repeat.


So, wait for other testers to weigh in with their test methods that are repeatable and at least comparible across testers. Roger at Lens Rentals will eventually look at the results using just the lens with his tester, that takes the camera out of the equation, so its another data point.

I have no reason to believe that it is not a super lens, Canon does compromise between producability, repairability, and pricing and leans heavily toward profitability. I think that Sony makes a lens more expensive to produce, very difficult and expensive to repair, and has a fat profit as well, that's a common denominator.

Companies buying equipment often look at a thing called Life Cycle Costs. I suspect that the Sony cost over its life is going to be a huge difference. For Enthusiasts who do not use a product day in and day out, its likely a lesser cost, but, if they drop one, they will wish it were insured.

 

[AlanF]

Roger has done proper MTF tests. The Canon looks superior.
https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/

 

[raptor3x]

Roger has done proper MTF tests. The Canon looks superior.
https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/

Could be due to sample variation in DxO's copies or it could be coming from the Sony or Canon performing differently at short focal distances versus infinity.

 

[Mt Spokane Photography]

Roger has done proper MTF tests. The Canon looks superior.
https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/

Thanks for the link. Looking at his display of the wide variability of 35mm performance, its possible that a single Sony lens could be better than a single Canon lens.

Its the expected result of issuing a single test value that is done using a secret formula to give a nonsense value.

As other lens testers weigh in, we may see a lot of variability at the 35mm focal length, but they should all look good at 16mm.

 

[AlanF]

Yes. Drawing conclusions from testing a single copy of one lens vs a single copy of another is in the realm of fake news.

 

[BillB]

Roger has done proper MTF tests. The Canon looks superior.
https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/

Could be due to sample variation in DxO's copies or it could be coming from the Sony or Canon performing differently at short focal distances versus infinity.

Could also have to do with DXO's use of different camera bodies with different sensors for the two lenses, along with secret, and quite possibly unreliable, assessment methods.

 

[Jopa]

Roger has done proper MTF tests. The Canon looks superior.
https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/

Could be due to sample variation in DxO's copies or it could be coming from the Sony or Canon performing differently at short focal distances versus infinity.

Also could be due to a few dollars (euros, yens, russian roubles, etc...) coming from Sony

 

[SecureGSM]

What are they smoking over there at DXO Labs?

Canon 16-35 III is so sharp it is reduculous!
Ignore them. They are total nuts.
I ran FoCal calibration on my 16-35 III and was shocked how sharp the lens is.
The Mark II lens is quite soft at 35mm end but Mark III is a stellar performer.

Could also have to do with DXO's use of different camera bodies with different sensors for the two lenses, along with secret, and quite possibly unreliable, assessment methods.

 

[Talys]

What are they smoking over there at DXO Labs?

Canon 16-35 III is so sharp it is reduculous!
Ignore them. They are total nuts.
I ran FoCal calibration on my 16-35 III and was shocked how sharp the lens is.
The Mark II lens is quite soft at 35mm end but Mark III is a stellar performer.

Could also have to do with DXO's use of different camera bodies with different sensors for the two lenses, along with secret, and quite possibly unreliable, assessment methods.

One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency.

My favorite lenses aren't just really sharp when they're mounted on a tripod and I have the luxury of making sure that it's perfectly focused. My favorite lenses have highly reproducible autofocus, where I can rely on AF being in sharp focus every time. Or, in Reikan FoCal terms, the curve "fit". It doesn't really matter to me on a lens that I'm not manually focusing, if it's really sharp, but it only AFs to really sharp 1 time out of 4.

As an example, my Canon 50/1.8 and 50/1.4 have terrible AF consistency, and my Sigma 150-600 is slightly inconsistent. So I CAN get amazing sharp images out of them. But AF inconsistency makes AFMA much less useful -- sometimes -8 gives me a perfect image; sometimes +3 gives me a perfect image.

In comparison, my 24-70 or 70-200 or 100-400 all have dead-on consistency, so once tuned, I can be very confident about it. But this is never considered in DXO tests.

 

[ahsanford]

One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency.

+1. I've owned/used/rented three f/2.8L zooms and got nothing but flawless AF performance from them. I continue to marvel at how fast/accurate/consistent my 70-200 f/2.8L IS II is.

And Sony has the gall to court L-lens loving Canonites with FBW lenses. : I'm sure they are sharp for the money they are charging for them, but LensTip -- the only show in town for fairly consistent AF testing -- doesn't have this 16-35 G Master or many other recent offerings to report on AF consistency. I'm actually curious to see how they perform.

- A

 

[SecureGSM]

Are you saying there is a AF consistency problem with MILCs now?
There is none in my books
What about MF lenses? Zeiss Otus glass is second rate now? ;D

One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency.

+1. I've owned/used/rented three f/2.8L zooms and got nothing but flawless AF performance from them. I continue to marvel at how fast/accurate/consistent my 70-200 f/2.8L IS II is.

And Sony has the gall to court L-lens loving Canonites with FBW lenses. : I'm sure they are sharp for the money they are charging for them, but LensTip -- the only show in town for fairly consistent AF testing -- doesn't have this 16-35 G Master or many other recent offerings to report on AF consistency. I'm actually curious to see how they perform.

- A

 

[Jopa]

Are you saying there is a AF consistency problem with MILCs now?
There is none in my books
What about MF lenses? Zeiss Otus glass is second rate now? ;D

One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency.

+1. I've owned/used/rented three f/2.8L zooms and got nothing but flawless AF performance from them. I continue to marvel at how fast/accurate/consistent my 70-200 f/2.8L IS II is.

And Sony has the gall to court L-lens loving Canonites with FBW lenses. : I'm sure they are sharp for the money they are charging for them, but LensTip -- the only show in town for fairly consistent AF testing -- doesn't have this 16-35 G Master or many other recent offerings to report on AF consistency. I'm actually curious to see how they perform.

- A

It's possible (but very rare) with OSPDAF in the AF-S (one shot) mode. The camera showed a green OSPDAF square on my subject but the focus was somewhere else. I think a pure CDAF or DPAF should not have this problem.

Also it depends on the implementation. Sony is focusing at the chosen aperture, and it's good to prevent any focus shift, but if the lens doesn't have any - the result won't be as sharp as focusing wide open and then narrowing the aperture down. Diglloyd once complained about this issue. There is no perfect solution...

 

[ahsanford]

Are you saying there is a AF consistency problem with MILCs now?
There is none in my books
What about MF lenses? Zeiss Otus glass is second rate now? ;D

Have you seen the AF testing on the Otus? Wow. Worst AF LensTip ever tested.

- A

 

[Ryananthony]

I was suraised to realise that both the Sony 16-35/4 and 16-35/2.8 both extend going through the zoom range. I've never shot with an ultra wide that did that.

 

[ahsanford]

I was suraised to realise that both the Sony 16-35/4 and 16-35/2.8 both extend going through the zoom range. I've never shot with an ultra wide that did that.

The 16-35 f/4L IS and 16-35 f/2.8L III's front element moves with zooming, but not like a 24-whatever standard zoom does. It moves behind the front rim of the outer lens barrel. So it doesn't actually change physical length as you use it.

You can see what I mean here:
https://www.the-digital-picture.com/Reviews/Lens-Product-Images.aspx?Lens=1073&LensComp2=0&LensComp=949
(mouseover the non-hooded 'select view' options below the lens shots to see what I mean)

Can't speak for which 16-35 for Sony you are referring to (I get mixed up between A / FE Sony options and the Zeiss made FE options), but I believe one of them actually trombone extends like a 24-something zoom but in reverse -- 16mm is the longer configuration while 35 is the shorter configuration.

- A

 

[SecureGSM]

that was a joke of course.

Are you saying there is a AF consistency problem with MILCs now?
There is none in my books
What about MF lenses? Zeiss Otus glass is second rate now? ;D

Have you seen the AF testing on the Otus? Wow. Worst AF LensTip ever tested.

- A

 

[jeffa4444]

Working as I do in an exacting industry where lens results are projected onto movie theatre screens in everyday motion picture shoots the testing conducted by DXO is questionable.

All our lenses are measured against the theoretical specification we applied at the design stage where we manufacture lenses. Copy to copy variance will happen mainly due to mechanical set-up rather than optical but even optically you can have a variance albeit normally quite small. The testing is done on an MTF machine and on optical projectors which are calibrated and checked often in a controlled room (we also test f / T stops). The camera is separately checked for flange focal depth & sensor alignment and then finally the lens is tested on the camera both on test charts and projected images from a live feed. Of the tools I would say the optical projector is just as important if not more so than the MTF machine and simply testing a lens on a camera for a company like DXO is amateur.

Ive tested my copy of the Canon EF 16-35mm f4L IS USM on the projector and projected in our theatre on a 5DS, yes its one example but the results are extremely good and we test lenses from all the major brands and our results are at odds with DXO.

 

[3kramd5]

The “right” way to use DXO’s lens numbers is to see how various lenses perform on camera bodies you’re interested in, or vice versa. Comparing across mounts is pointless.