September 17, 2014, 11:45:52 PM

Author Topic: Dxo tests canon/nikon/sony 500mm's  (Read 15852 times)

weixing

  • Canon 70D
  • ****
  • Posts: 261
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #45 on: July 15, 2013, 11:31:39 AM »
Quote
   DxO should come out a standard testing camera for testing lens... a mirrorless camera should be idea since it'll have the shortest flange focal distance and can use adapter for different vendor lens... then the only variables will be the lens and the result can be valid to compare between different lens.

that wouldnt really be more valid as you still have to mount those lenses on their brand camera for real world purposes. why would i care what the test results of a lens would be mounted to a camera i would never shoot. i would rather see test results from a lens/body combo that i could actually use.

not that i really care about DxO that is....
    Hmm... might not mean anything to Nikon DSLR user, but Canon user can mount Nikon lens to Canon DSLR...  ;)

    Have a nice day.

canon rumors FORUM

Re: Dxo tests canon/nikon/sony 500mm's
« Reply #45 on: July 15, 2013, 11:31:39 AM »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14390
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #46 on: July 15, 2013, 11:55:45 AM »
If transmission really was the most important factor then the Sony should  be out scoring the Nikon.  The Sony equals or beats the Nikon in all the listed categories and has  the lowest transmission of the three lenses.

Transmission on macro scale, not micro. In fact, probably not even measured transmission, but rather the specified max aperture. That's why the 50/1.8 lenses outscore the 500/4 lenses.

If you look at the test results they are measuring the combination of lens and body.  Hard to do otherwise.  Makes comparison between manufactures difficult and less than meaningful.

Yes, and sorry, but I think you are missing the point.  For every measurement, all generated with a body attached, the Canon lens comes out on top, in some cases by a significant margin.  Yet, the Score is a tie.  So...the score is fabricated, pulled from their nether orifices, etc.

If you go over to photozone.de's page, they'll quite clearly state that the measurements (or scores) from one camera system (make + sensor size) cannot be compared with another. Given this I can't see why the same wouldn't be true for DxO. What does that mean? That you cannot compare a score of 25 for the Nikon lens with a score of 25 for the Canon or 22 for the Sony lens.

To be able to compare each lens properly would require each lens being mounted on the same camera.

How do they calculate their lens score? Would love to know but I'm pretty sure that it is corporate secret. For the Nikon one to be so high must mean that the readings are somehow weight on the sensor (e.g pixel size.)

The point is that they measure several parameters of optical image quality from the lens, such as sharpness, transmission, distortion, vignetting, and CA.  They could generate a Lens Score based on those parameters, but they don't. Had they done so, the Canon 500/4 II would have soundly trounced the Nikon 500/4. 

For those who argue that it's reasonable that DxO consider the camera in the 'Lens Score', note that the sensor is already factored into the measurements themselves.  P-Mpix measures sharpness of camera + lens, pixel size affects CA, etc.  Even their transmission measurement changes with different cameras.  So by factoring in the camera directly in the measurements (reasonable) then factoring it in again in the overall score where it's given an undisclosed (but evidently very significant) weighting, means their 'Lens Score' is as much if not more a camera score than a lens score.

One more point - considering just their P-Mpix measure of sharpness, by their definition the Nikon 500/4 results in a loss of more that 55% of the resolution of which the D800 sensor is capable, whereas the Canon 500/4 II only decreases the 5DIII's potential resolution by less than 14%.  Of course, the Canon lens also outresolves the Nikon by an absolute assessment, even taking the higher resolution D800 sensor into account.  But they get the same 'Lens Score'. Right.

They should rename their Overall Scores to a Camera Basic Score and a Lens Basic Score, so we could abbreviate them for what they really are: BS.  Actually, that's probably giving them too much credit, because real Bovine Scat makes good fertilizer, whereas DxOMark's BS has no real-world utility.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

msm

  • PowerShot G1 X II
  • ***
  • Posts: 68
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #47 on: July 15, 2013, 01:25:54 PM »
whereas DxOMark's BS has no real-world utility.

It does precisely what it is supposed to do, generate a lot of views for the DXO site where they advertise and sell their software. This is evident from this and other threads. It costs them very little since they need to test the gear to make their software anyways.

Woody

  • 6D
  • *****
  • Posts: 572
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #48 on: July 15, 2013, 08:25:53 PM »
For those who argue that it's reasonable that DxO consider the camera in the 'Lens Score', note that the sensor is already factored into the measurements themselves.  P-Mpix measures sharpness of camera + lens, pixel size affects CA, etc.  Even their transmission measurement changes with different cameras...

Actually, that's probably giving them too much credit, because real Bovine Scat makes good fertilizer, whereas DxOMark's BS has no real-world utility.

Good point.

dilbert

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 2916
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #49 on: July 15, 2013, 10:47:57 PM »
...
They should rename their Overall Scores to a Camera Basic Score and a Lens Basic Score, so we could abbreviate them for what they really are: BS.  Actually, that's probably giving them too much credit, because real Bovine Scat makes good fertilizer, whereas DxOMark's BS has no real-world utility.

Your assertions of DxO publishing BS are just as BS as their results because you don't know how they calculate their "lens score" and thus you have to make up reasons as to why it is so. Yes, you're compounding the BS by stating that they're publishing BS. Just don't buy into it.

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14390
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #50 on: July 15, 2013, 11:20:06 PM »
...
They should rename their Overall Scores to a Camera Basic Score and a Lens Basic Score, so we could abbreviate them for what they really are: BS.  Actually, that's probably giving them too much credit, because real Bovine Scat makes good fertilizer, whereas DxOMark's BS has no real-world utility.

Your assertions of DxO publishing BS are just as BS as their results because you don't know how they calculate their "lens score" and thus you have to make up reasons as to why it is so. Yes, you're compounding the BS by stating that they're publishing BS. Just don't buy into it.

Except I'm not scoring lenses without disclosing my methods. In my scientific publishing, I must fully disclose my methods such that another scientist can duplicate my experiment.  DxOMark is clearly not held to the standards of peer-reviewed scientific publications. (Which is reasonable, as they're not publishing in scientific journals, but only to their own website - so, they can make up 'data' or 'scores' if they choose, change those data post hoc without explanation, whatever they want.)

We don't know what's in their black box? They don't say. Why not? If someone is hiding something, it's usually for a good reason.  Could be as simple as maintaining a competitive advantage (against who, I have no idea).

Clearly, I don't buy into it. Sadly, I suspect many people do...
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

Woody

  • 6D
  • *****
  • Posts: 572
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #51 on: July 16, 2013, 08:38:26 AM »
Your assertions of DxO publishing BS are just as BS as their results because you don't know how they calculate their "lens score" and thus you have to make up reasons as to why it is so.

Actually, based on their description, we have some idea of how the lens score is computed. For example, we know they take sensor score into account. However, this method of computation makes no sense.

canon rumors FORUM

Re: Dxo tests canon/nikon/sony 500mm's
« Reply #51 on: July 16, 2013, 08:38:26 AM »

Pi

  • 1D Mark IV
  • ******
  • Posts: 937
    • View Profile
    • Math and Photography
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #52 on: July 16, 2013, 11:56:34 AM »
why would i care what the test results of a lens would be mounted to a camera i would never shoot. i would rather see test results from a lens/body combo that i could actually use.

Because lenses, especially one like this, will probably be used on new bodies for the next 10 years or so.  It will far outlast the current "best" body of a brand, possibly by several generations. [...]

So my answer is yes: to the greatest degree feasible, a "lens test" should isolate the lens, even mounting the competitors on the same body if possible.  (of course that's difficult, but that's what I'd like to see)

But there is still a problem: if you have a pure lens test, are you sure you know how to compute how it will perform on a future, say, 50mp body?
« Last Edit: July 16, 2013, 11:58:29 AM by Pi »

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4497
  • POTATO
    • View Profile
    • Nature Photography
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #53 on: July 16, 2013, 12:32:57 PM »
why would i care what the test results of a lens would be mounted to a camera i would never shoot. i would rather see test results from a lens/body combo that i could actually use.

Because lenses, especially one like this, will probably be used on new bodies for the next 10 years or so.  It will far outlast the current "best" body of a brand, possibly by several generations. [...]

So my answer is yes: to the greatest degree feasible, a "lens test" should isolate the lens, even mounting the competitors on the same body if possible.  (of course that's difficult, but that's what I'd like to see)

But there is still a problem: if you have a pure lens test, are you sure you know how to compute how it will perform on a future, say, 50mp body?

MTF charts are, for all intents and purposes, "pure lens tests". They already give us a way to compare lenses across the board, brands be damned. Simple fact of the matter is a better lens will perform better on ALL sensors, 20mp, 30mp, or 50mp. The problem with DXO's tests is they quite simply don't give you a reasonable camera-agnostic basis from which to compare lenses. The Nikon 500/4 performs "on par" (toung in cheek) with the Canon 500/4 solely because of the higher resolution sensor. That sort of tells you that the Canon lens is particularly good, because it is performing so well on a worse sensor...but you don't really have any exact way of comparing. You only get a "feeling" that it performs so well.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

Orangutan

  • 5D Mark III
  • ******
  • Posts: 688
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #54 on: July 16, 2013, 01:03:27 PM »
But there is still a problem: if you have a pure lens test, are you sure you know how to compute how it will perform on a future, say, 50mp body?

You don't; however, you do know that a better lens will perform no worse, and likely better, on any future camera than would a lens that performed worse in a "pure" lens test.  Though not perfect, it's superior to the tainted lens+body test.


MTF charts are, for all intents and purposes, "pure lens tests". They already give us a way to compare lenses across the board, brands be damned.

If memory serves me well (no guarantee here) most manufacturers use computed (theoretical) MTF charts that are derived from the design of the optics.  A few (?Zeiss?) actually test production copies.   I hope someone will correct me if I'm mistaken.  Yes, it would be great to see MTF charts using data from real, production copies.

TexasBadger

  • Rebel T5i
  • ****
  • Posts: 135
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #55 on: July 16, 2013, 01:20:38 PM »
Since you can mount the Nikon lens on a Canon body with an adapter, why not compare apples to apples?
5DC, 5D3, Elan7, G12, 28 1.8, 50 1.8 II, 85 1.8 USM, 135 2.0 L, 24-70 2.8 L, 70-200 2.8 L, 560 EX, 580 EX II (2) --- all Canon.

sandymandy

  • 6D
  • *****
  • Posts: 599
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #56 on: July 16, 2013, 01:54:25 PM »
i didnt read it...let me guess: nikon lenses the best as always? except in sales...

applecider

  • Rebel SL1
  • ***
  • Posts: 91
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #57 on: July 16, 2013, 02:19:55 PM »
This test should be added to a permanent faq so that any time a dxo discussion come up we all refer to this chart to show that the tests are meaningless only done to make nikon look good.
AE-1, T90,EOS Elan 7e, EOS-M, 7D, 5D3, 1dx, ef40 2.8, ef 85 1.2ii, ef100 is 2.8 macro, ef300 2.8ii, ef400 2.8ii, ef600 is ii usm, ef8-15 f4, ef16-35 ii, ef24-70 ii, ef 24-105, ef 70-200, 2xext ii, 2xext iii, 1.4x ext iii.

canon rumors FORUM

Re: Dxo tests canon/nikon/sony 500mm's
« Reply #57 on: July 16, 2013, 02:19:55 PM »

Orangutan

  • 5D Mark III
  • ******
  • Posts: 688
    • View Profile
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #58 on: July 16, 2013, 02:21:35 PM »
i didnt read it...let me guess: nikon lenses the best as always? except in sales...

Nope.

This comparison of DXO's results is what makes me EXTREMELY SUSPICIOUS of them. The Nikkor 500mm scores 25, when every single trait that factors into that score is worse than the Canon 500mm...which also scores 25. That is just plain wrong. The Canon has zero distortion, higher sharpness, less vignetting, less CA, and the same transmission...on a LOWER RESOLUTION BODY! It should have a higher score than the Nikon.

Pi

  • 1D Mark IV
  • ******
  • Posts: 937
    • View Profile
    • Math and Photography
Re: Dxo tests canon/nikon/sony 500mm's
« Reply #59 on: July 16, 2013, 05:42:45 PM »
But there is still a problem: if you have a pure lens test, are you sure you know how to compute how it will perform on a future, say, 50mp body?

You don't; however, you do know that a better lens will perform no worse, and likely better, on any future camera than would a lens that performed worse in a "pure" lens test.  Though not perfect, it's superior to the tainted lens+body test.

But a better lens on, say, the 5D3, will preform better on the 5D4, you know that for sure. Again, where is the problem exactly?

canon rumors FORUM

Re: Dxo tests canon/nikon/sony 500mm's
« Reply #59 on: July 16, 2013, 05:42:45 PM »