Your DXO chuckler for the day...

[ahsanford]

http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Milvus-F14-50mm-ZF2-Nikon-on-Nikon-D810-versus-Zeiss-Carl-Zeiss-Distagon-T-STAR-Otus-55mm-F14-ZF2-Nikon-on-Nikon-D810__1605_963_1242_963

On the same camera body, they demonstrated the same sharpness, transmission and vignetting, and had very close distortion and chromatic aberration values.

Yet one is considered 8 points better in their aggregate score.

#dxo #fairandbalanced

- A

 

[Larsskv]

http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Milvus-F14-50mm-ZF2-Nikon-on-Nikon-D810-versus-Zeiss-Carl-Zeiss-Distagon-T-STAR-Otus-55mm-F14-ZF2-Nikon-on-Nikon-D810__1605_963_1242_963

On the same camera body, they demonstrated the same sharpness, transmission and vignetting, and had very close distortion and chromatic aberration values.

Yet one is considered 8 points better in their aggregate score.

#dxo #fairandbalanced

- A

Thank you for pointing this out. DXO=X-files.

 

[Loswr]

After careful consideration, I've come to the conclusion that DxO's Lens Score would be much more reliable if the company was based in the US. The reason? OSHA. The US Occupational Safety and Health Administration has strict guidelines on the use of potentially hazardous equipment in the workplace. I suspect those guidelines would preclude use of the primary tool with which DxO determines their Lens Scores. I refer, of course, to the tool on their break room wall...

 

[Jack Douglas]

Maybe they need to reduce the "gain" on their scoring ratio. Why don't people just forget about DXO since they cause so much frustration? I look at the odd DXO thread for a laugh but they generally get super boring super fast.

Jack

 

[ahsanford]

After careful consideration, I've come to the conclusion that DxO's Lens Score would be much more reliable if the company was based in the US. The reason? OSHA. The US Occupational Safety and Health Administration has strict guidelines on the use of potentially hazardous equipment in the workplace. I suspect those guidelines would preclude use of the primary tool with which DxO determines their Lens Scores. I refer, of course, to the tool on their break room wall...

Silly Neuro, those guys can't even associate test results and accolades consistently, what makes you think they can aim?

Besides: we all know DXO's rating system requires the use lawn darts made of old 24-70 f/2.8L mk I lenses dropped off of their office roof. The mortality rate of small woodland creatures in Boulogne-Billancourt area is apocalyptic as a result.

- A

 

[Loswr]

Silly Neuro, those guys can't even associate test results and accolades consistently, what makes you think they can aim?

Why would they aim?!?

 

[IglooEater]

Silly Neuro, those guys can't even associate test results and accolades consistently, what makes you think they can aim?

Why would they aim?!?

I think the target would be a lot safer if they did aim.

 

[gregorywood]

Is there a definitive source for "objective" lens testing anywhere? I'm not suggesting in any way that DxO is that, but curious if there is any resource that does it with any integrity.

 

[PeterAlex7]

OOT, why they still didn't review 200 f2L?

 

[3kramd5]

Is there a definitive source for "objective" lens testing anywhere? I'm not suggesting in any way that DxO is that, but curious if there is any resource that does it with any integrity.

Sure, lens rentals is pretty good, for example.

But everyone will have certain test constraints (things like test distances, whether infinity focus is required, etc), and likely none can do an apples to apples comparison of anything other than bench-level optics (so for example variables such as how micro lenses affect system performance will be lost, though could maybe be characterized per camera) and maybe IS effectiveness, but for example AF performance is a crap shoot.

 

[ahsanford]

Is there a definitive source for "objective" lens testing anywhere? I'm not suggesting in any way that DxO is that, but curious if there is any resource that does it with any integrity.

Integrity / Objectivity? Yes. There are a few. LensRentals, LensTip, PhotoZone and TDP all do a decent job. None are perfect, however:

LensRentals is the best show in town for statistical power and de-coupling from any sort of camera/sensor, but it's a brute force optical assessment only. Variables like AF, handling, and real-world usage considerations are not there.

LensTip has the most comprehensive 'yeah they test that' sort of setup -- looking at all sorts of fun stuff like coma, AF speed, etc. alongside the commonly reported resolution / distortion / vignetting. But comparing lenses at that site is a manual and tedious process, and they only test one lens to my knowledge.

PhotoZone is like a condensed flavor of LensTip, but to their credit, they are making the painful climb up '50 Megapixel Mountain' where LT has not yet started that work. They also only test one lens unless there is a glaring issue.

I love TDP -- the IQ comparison & noise tools are terrific -- but on the question of objectivity, Mr. Carnathan (generally) gives rave reviews of Canon gear. He backs up what he says and he's a stellar usage/real-world/handling reviewer who's writing I respect and refer to often, but if everything gets an A grade, that A grade means less than it should.

- A

 

[ashmadux]

http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Milvus-F14-50mm-ZF2-Nikon-on-Nikon-D810-versus-Zeiss-Carl-Zeiss-Distagon-T-STAR-Otus-55mm-F14-ZF2-Nikon-on-Nikon-D810__1605_963_1242_963

On the same camera body, they demonstrated the same sharpness, transmission and vignetting, and had very close distortion and chromatic aberration values.

Yet one is considered 8 points better in their aggregate score.

#dxo #fairandbalanced

- A

Thank you for pointing this out. DXO=X-files.

X- Files, Bwahahah - nice one there.

 

[AlanF]

Is there a definitive source for "objective" lens testing anywhere? I'm not suggesting in any way that DxO is that, but curious if there is any resource that does it with any integrity.

Integrity / Objectivity? Yes. There are a few. LensRentals, LensTip, PhotoZone and TDP all do a decent job. None are perfect, however:

LensRentals is the best show in town for statistical power and de-coupling from any sort of camera/sensor, but it's a brute force optical assessment only. Variables like AF, handling, and real-world usage considerations are not there.

LensTip has the most comprehensive 'yeah they test that' sort of setup -- looking at all sorts of fun stuff like coma, AF speed, etc. alongside the commonly reported resolution / distortion / vignetting. But comparing lenses at that site is a manual and tedious process, and they only test one lens to my knowledge.

PhotoZone is like a condensed flavor of LensTip, but to their credit, they are making the painful climb up '50 Megapixel Mountain' where LT has not yet started that work. They also only test one lens unless there is a glaring issue.

I love TDP -- the IQ comparison & noise tools are terrific -- but on the question of objectivity, Mr. Carnathan (generally) gives rave reviews of Canon gear. He backs up what he says and he's a stellar usage/real-world/handling reviewer who's writing I respect and refer to often, but if everything gets an A grade, that A grade means less than it should.

- A

Lenstip and Photozone are both flawed in that they test only one copy of each lens, do it at a fixed and undefined distance, and don't state the line frequency of their Imatest results. ePhotozine, another good site, has the same problems. Cameralabs can be quite good as they do use different distances. TDP sometimes does more than one copy but comparisons can be misleading because they use different distances for different focal lengths of lenses. One experienced professional reviewer wrote me that he never takes any notice of 3rd party MTF measurements.

I have been thinking about the Panasonic-Leica 100-400mm. Lenstip panned it at 400mm a couple of days ago as having weak MTFs and poor IS. But, ePhotozine has much better MTF measurements, and Lenstip measured IS on an Olympus body whereas the Pana-Leica is designed to have its in-lens IS coupled with the 4 or 5 axis sensor stabilization of Panasonic bodies like the G85 and is very good then.

Distance of target on testing is very important. The Nikon 200-500mm is very sharp for close targets, like used for Imatest, but is soft for far distances. The Tamron and Sigma 150-600mm are sharper at long distances. How would you know this from Lenstip or Photozone pr ePhotozine? I know that because I Google and read every review I can before deciding to consider a lens myself - there are lots of very good, more specialised reviews. The only tests that matter are the ones you do yourself on your copy of the lens and your camera under the conditions that you work at, especially as there is so much copy variation on many lenses. So, I now order on-line, test myself and, if good, keep. If it's soft or decentered, back it goes.

 

[Maui5150]

http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Milvus-F14-50mm-ZF2-Nikon-on-Nikon-D810-versus-Zeiss-Carl-Zeiss-Distagon-T-STAR-Otus-55mm-F14-ZF2-Nikon-on-Nikon-D810__1605_963_1242_963

On the same camera body, they demonstrated the same sharpness, transmission and vignetting, and had very close distortion and chromatic aberration values.

Yet one is considered 8 points better in their aggregate score.

#dxo #fairandbalanced

- A

Silly you. Because the body is their fav Nikon 810, If you would have put the cheaper one on the Nikon and the Beast on a Canon, all of the sudden the cheaper one would be 8 points higher.

I have to laugh... Someone mentioned Integrity and DxO... This is like Honesty and Hillary

 

[Refurb7]

DxO has no credibility at all.

 

[rfdesigner]

I've raised this before but it's worth repeating.

The 300f2.8 is so good it gets you more perceptive megapixels than the camera starts with:

http://www.dxomark.com/Lenses/Canon/Canon-EF-300mm-F28L-IS-II-USM-mounted-on-Canon-EOS-6D__836

 

[slclick]

http://www.dxomark.com/Lenses/Compare/Side-by-side/Zeiss-Carl-Zeiss-Milvus-F14-50mm-ZF2-Nikon-on-Nikon-D810-versus-Zeiss-Carl-Zeiss-Distagon-T-STAR-Otus-55mm-F14-ZF2-Nikon-on-Nikon-D810__1605_963_1242_963

On the same camera body, they demonstrated the same sharpness, transmission and vignetting, and had very close distortion and chromatic aberration values.

Yet one is considered 8 points better in their aggregate score.

#dxo #fairandbalanced

- A

Silly you. Because the body is their fav Nikon 810, If you would have put the cheaper one on the Nikon and the Beast on a Canon, all of the sudden the cheaper one would be 8 points higher.

I have to laugh... Someone mentioned Integrity and DxO... This is like Honesty and Hillary

I'd like to build a wall around their website

 

[retroreflection]

We can have fun bashing so and so's integrity, but we should also bash ourselves for being cheap bastards. We expect fifty axis plots, easily searchable by any parameter we can imagine, statistically valid, beholden to no one, FOR FREE.
If we were willing to subscribe to a testing service, or pay per query to the service, we could expect rock solid integrity. Those actually willing to pay something would be shocked at how much these desires really cost. Instead, we rely on an advertisement model. That has consequences.
Thankfully, cameras are so capable that it doesn't matter for most of us. The fleet of YouTubers willing to post reviews will generally find the glaring issues. Beyond that, precise measurement of camera system capabilities is just a waste of time for ordinary folks.

 

[Mt Spokane Photography]

Is there a definitive source for "objective" lens testing anywhere? I'm not suggesting in any way that DxO is that, but curious if there is any resource that does it with any integrity.

There is no single set of test methods as is the case with much of the technology testing that I've done where a group of industry experts meets and develops a set of test methods that become officially recognized industry or government standards. DXO believes that they can do better by inventing their own methodology, and
has repeatedly offered their test methodology to ISO, and it has been rejected by the ISO committee members.

Certainly, Lens Rentals does not use standardized testing, but uses machinery developed for their own needs, which is for identifying faulty lenses so they can be repaired. Roger does not pretend to use a perfect or industry standard method.

The issue is that DXO scores are based on a secret scoring system which, often results in absurdities. A open methodology and grading system would help keep personal bias and mistakes out of the scores, but still would not address their inability to get the community of lens and camera manufacturers to buy in.

 

[Refurb7]

The issue is that DXO scores are based on a secret scoring system which, often results in absurdities.

A failure at the most basic level. Secret scoring. That is so ridiculous. It would be a funny joke if it weren't true.