December 03, 2016, 05:15:11 AM

Author Topic: DxOMark Sensor Performance: Nikon vs. Canon  (Read 126949 times)

Zlatko

  • Guest
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #90 on: September 20, 2012, 02:32:51 PM »
I have the D800 and the 5D3, and I would agree that the Canon does better at high ISO.  For a predominantly landscpe shooter like myself, though the amount of banding at lower ISO in combination with not great DR renders the camera a handicap to me.
I do weddings and portraits with the 5D3 and have never seen any banding issue.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #90 on: September 20, 2012, 02:32:51 PM »

sarangiman

  • Canon 6D
  • *****
  • Posts: 375
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #91 on: September 20, 2012, 03:47:21 PM »
I have said this many times. You CAN NOT compare dynamic range on ISO100, and assume everybody works in ISO100, and that this is the correct ISO to measure by, like DxO does. I RARELY go below ISO 1600 for my kind of work.

Just because this is brought up rather often, allow me to point out:

  • At ISO 100: D800 has >2.5EV greater DR than 5DIII.
  • At ISO 12,800, D800 has 0.09EV worse DR than 5DIII, probably below the margin of error in measurement.
  • At ISO 25,600, D800 has 0.26EV worse DR than 5DIII, probably still within the margin of error in measurement.

Full measurements here: http://bit.ly/OEUgZY

So now matter how you look at it, D800 is rarely going to perform worse in DR than the 5DIII, if at all, but can perform much much better than the 5DIII.

When camera/software goes ISO-less, we'll see cameras maintaining base ISO DR even at high ISOs. Imagine >13EV DR at ISO 6400!
« Last Edit: September 20, 2012, 03:51:24 PM by sarangiman »

sarangiman

  • Canon 6D
  • *****
  • Posts: 375
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #92 on: September 20, 2012, 03:52:15 PM »
I do weddings and portraits with the 5D3 and have never seen any banding issue.

I see banding in my 5DIII images just from having Lightroom automatically correct the vignetting for my 24/1.4 & 35/1.4 lenses. I just try to ignore it.  :'(

neuroanatomist

  • CR GEEK
  • ************
  • Posts: 19977
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #93 on: September 20, 2012, 05:03:39 PM »
I do weddings and portraits with the 5D3 and have never seen any banding issue.

I see banding in my 5DIII images just from having Lightroom automatically correct the vignetting for my 24/1.4 & 35/1.4 lenses. I just try to ignore it.  :'(

Have you tried DPP instead of LR?
EOS 1D X, EOS M2, lots of lenses
______________________________
Flickr | TDP Profile/Gear List

mystic_theory

  • Guest
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #94 on: September 20, 2012, 05:18:17 PM »
I have said this many times. You CAN NOT compare dynamic range on ISO100, and assume everybody works in ISO100, and that this is the correct ISO to measure by, like DxO does. I RARELY go below ISO 1600 for my kind of work.

Just because this is brought up rather often, allow me to point out:

  • At ISO 100: D800 has >2.5EV greater DR than 5DIII.
  • At ISO 12,800, D800 has 0.09EV worse DR than 5DIII, probably below the margin of error in measurement.
  • At ISO 25,600, D800 has 0.26EV worse DR than 5DIII, probably still within the margin of error in measurement.

Full measurements here: http://bit.ly/OEUgZY

So now matter how you look at it, D800 is rarely going to perform worse in DR than the 5DIII, if at all, but can perform much much better than the 5DIII.
+1

I get tired of reading a pile of fuming BS to cover up the 5DIII shortcomings compared to the D800.

DB

  • Guest
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #95 on: September 20, 2012, 05:23:52 PM »

Define capable, and in your definition please address their evaluation of the performance of the 70-200mm f/2.8L IS II, which they score lower than the MkI version of that lens.   :o

DxO have actually responded to this anomaly on their website when questioned by Canon 70-200mm f2.8 IS II users as to why the apparently superior mark II scores so much lower than its predecessor and their reply was that the older lens has better resolving power (@ 62 line pairs per millimeter vs 52 lp/mm for the new lens). But their website also states that there is enormous variation in resolution throughout the focal range, plus their Resolution Index scores are based on averages of multiple shots taken typically between f5 and f8 throughout the zoom range.

Clearly then, one can challenge the validity & reliability of this 'Resolution' test by conducting a similar test (they  outline in graphic detail their testing procedure) using say multiple samples (half a dozen) of each of the 2 lenses in question (to also test if copy-to-copy variation exists).

Secondly, irrespective of the first answer, what the results suggest given that the Mark II surpasses the Mark I in each of the other sub-categories (especially CA), is that their arbitrary weighting (Black Box is an apt description as they do not divulge their weighting methodology) system is heavily skewed towards 'Resolution' when computing the Overall Score.

The very fact that they place more emphasis on one single category (more so than all other sub-categories combined) is evidence that their overall score is biased & subjective. According to DxO the most impressive Canon lens in the entire EF range is the 85mm f/1.8.

What they should do is post the summary stats of all the resolution test results in a table (freq dist along with standard deviation figures) - that way we can see if it is case of some outlier or skew (to wide or tele) that is causing them to conclude that the mark I is better.

No academic journal would publish a DxO report without both Data (in an Appendix) and Methods (with a clearly defined algorithm stating the parameters for weighting each category). In the world of peer review, Black-Box methodology would simply have REJECTED stamped on it and returned.

neuroanatomist

  • CR GEEK
  • ************
  • Posts: 19977
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #96 on: September 20, 2012, 05:31:58 PM »
I get tired of reading a pile of fuming BS to cover up the 5DIII shortcomings compared to the D800.

Honestly, I think the bottom line is that both produce excellent images - if testing to the nth degree is needed to see differences, you have to question the significance of those differences.

Are there shortcomings?  Yes.  IMO, the biggest shortcoming of the 5DIII relative to the D800 is that the former doesn't work with the excellent 14-24/2.8 and an actually available 200-400/4.  Likewise, the biggest shortcoming of the D800 relative to the 5DIII is that it doesn't work the MP-E 65mm, TS-E 17mm, and Canon's slightly better superteles.  Of course, if you're willing to give up AF, Nikon lenses can be mounted on Canon bodies, whereas the converse isn't true...

No academic journal would publish a DxO report without both Data (in an Appendix) and Methods (with a clearly defined algorithm stating the parameters for weighting each category). In the world of peer review, Black-Box methodology would simply have REJECTED stamped on it and returned.

+1, and that's my beef with DxOMark.
EOS 1D X, EOS M2, lots of lenses
______________________________
Flickr | TDP Profile/Gear List

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #96 on: September 20, 2012, 05:31:58 PM »

bdunbar79

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 3111
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #97 on: September 20, 2012, 05:39:54 PM »
Show up to a fast-action soccer game with a D800 and then see which one, the 5D Mark III, or the D800, really has the shortcomings.  Kick off is at 7pm under the lights.
2 x 1DX
B1G, MAC, GLIAC

Zlatko

  • Guest
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #98 on: September 20, 2012, 05:49:10 PM »
According to DxO the most impressive Canon lens in the entire EF range is the 85mm f/1.8.
... which is freaking bizarre.  The 85 f/1.8 is very fine lens, but there is NO WAY that it is the most impressive lens in the entire EF range.  That tells me a lot about DxO.  When everyday experience informs you better than the DxO score, then the DxO score lacks credibility.

Fishnose

  • Rebel T6i
  • ****
  • Posts: 104
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #99 on: September 20, 2012, 05:55:36 PM »

Define capable, and in your definition please address their evaluation of the performance of the 70-200mm f/2.8L IS II, which they score lower than the MkI version of that lens.   :o

DxO have actually responded to this anomaly on their website when questioned by Canon 70-200mm f2.8 IS II users as to why the apparently superior mark II scores so much lower than its predecessor and their reply was that the older lens has better resolving power (@ 62 line pairs per millimeter vs 52 lp/mm for the new lens). But their website also states that there is enormous variation in resolution throughout the focal range, plus their Resolution Index scores are based on averages of multiple shots taken typically between f5 and f8 throughout the zoom range.

Clearly then, one can challenge the validity & reliability of this 'Resolution' test by conducting a similar test (they  outline in graphic detail their testing procedure) using say multiple samples (half a dozen) of each of the 2 lenses in question (to also test if copy-to-copy variation exists).

Secondly, irrespective of the first answer, what the results suggest given that the Mark II surpasses the Mark I in each of the other sub-categories (especially CA), is that their arbitrary weighting (Black Box is an apt description as they do not divulge their weighting methodology) system is heavily skewed towards 'Resolution' when computing the Overall Score.

The very fact that they place more emphasis on one single category (more so than all other sub-categories combined) is evidence that their overall score is biased & subjective. According to DxO the most impressive Canon lens in the entire EF range is the 85mm f/1.8.

What they should do is post the summary stats of all the resolution test results in a table (freq dist along with standard deviation figures) - that way we can see if it is case of some outlier or skew (to wide or tele) that is causing them to conclude that the mark I is better.

No academic journal would publish a DxO report without both Data (in an Appendix) and Methods (with a clearly defined algorithm stating the parameters for weighting each category). In the world of peer review, Black-Box methodology would simply have REJECTED stamped on it and returned.

One very good reason that resolution should be weighted quite heavily is that resolution is pretty much impossible to 'fix in the mix'. Okay, a bit of sharpening, but that's not optimal as a solution for lower resolution.

Other issues such as CA, vignetting and distortion, on the other hand, can be fixed very well. And that is precisely the business DxO are in.

neuroanatomist

  • CR GEEK
  • ************
  • Posts: 19977
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #100 on: September 20, 2012, 06:02:35 PM »
Other issues such as CA, vignetting and distortion, on the other hand, can be fixed very well. And that is precisely the business DxO are in.

Sure they can. But not for free...and the price you pay for many of those fixes is decreased resolution.
EOS 1D X, EOS M2, lots of lenses
______________________________
Flickr | TDP Profile/Gear List

sarangiman

  • Canon 6D
  • *****
  • Posts: 375
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #101 on: September 20, 2012, 06:56:43 PM »
Have you tried DPP instead of LR?

Hi neuroanatomist,

Yes, I tried DPP, since threads quickly popped up (I think on dpreview) claiming that DPP fixed the issue. It most certainly did not, IMHO. To me, the default noise reduction settings is what helped cover up the banding. But once you turned all that off, the pattern noise was still there.

Furthermore, DPP allows less ability to lift shadows than Lightroom, making comparisons of more drastic edits impossible.

sarangiman

  • Canon 6D
  • *****
  • Posts: 375
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #102 on: September 20, 2012, 06:59:29 PM »
No academic journal would publish a DxO report without both Data (in an Appendix) and Methods (with a clearly defined algorithm stating the parameters for weighting each category). In the world of peer review, Black-Box methodology would simply have REJECTED stamped on it and returned.

Yes, it'd be wonderful if they published their full methodology & made their RAW files available. As you mention, that's what'd have to happen in a peer-reviewed journal. I think they might gain more credibility if they did these things.

As for the lens tests -- how many copies of the 70-200 f/2.8L II did they test? The II is most certainly sharper than I wide open on the few copies I've handled. Copy variability can definitely skew results, as has been mentioned before. Not so much the case with sensors, which is why I trust their sensor data.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #102 on: September 20, 2012, 06:59:29 PM »

mystic_theory

  • Guest
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #103 on: September 20, 2012, 07:05:00 PM »

Define capable, and in your definition please address their evaluation of the performance of the 70-200mm f/2.8L IS II, which they score lower than the MkI version of that lens.   :o

No academic journal would publish a DxO report without both Data (in an Appendix) and Methods (with a clearly defined algorithm stating the parameters for weighting each category). In the world of peer review, Black-Box methodology would simply have REJECTED stamped on it and returned.

For a living I don't take pics, but I do publish papers in theoretical physics. Now, compared to the complete lack of rigor in photography testing (at least from my very limited experience of forum, blogs, and other online sources reader), the DxO tests seem outstanding, especially when compared to those of the various Ken Rockwell and company (including DPreview, which now roots for Canon like a cheerleader in the interpretation of their results). Are the DxO tests rigorous enough to be published in a serious scientific journal? Very likely, no. But for what they are meant to do (publicize a software), they are outstanding.

DBCdp

  • Canon AE-1
  • ***
  • Posts: 64
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #104 on: September 20, 2012, 07:45:48 PM »
My 3.2MP Pentax Optio S took wonderful pictures. My Canon 20D was soft over the entire 20K+ range I took with it. The 5D Classic I had was Superb. As is my 5DMkII. But I prefer the images from my old 1DsMkII. What does all the rhetoric actually mean without the burden of proof? Where are the PICTURES?!? 8 pages of hyperexented talk about this or that with zero pictures to back or explain any of the viewpoints!

I like the way a Canon body fits my hand. I like the build of a Pro series 1D. I do not like the ergonomics of Nikon. Either one takes pictures. Neither one composes them. Not a single camera out there sets up the lighting for a perfect shot. Who's perfect shot? Well that would have to be the client. Not the guy who buys the camera, but the multiple clients that pay for it! My client last night was tickled to death with the shots of her business taken with Canon's 5DMkII. That's what counts. Does she know about DxO? Does it matter? Does she or any of my other clients know wether Canon, Nikon, Pentax, Sony or Olympus would give them the look they want? Or do they simply trust me to deliver? So in the end isn't it always a subjective issue based on the need and the delivery and NOT anything at all to do with test results in the lab?

I look at DxO. I look at SLR Gear. I also read DP Review's outlook as well as delve through the pages here. And the only thing I can say with absolute certainty is that I'm not getting my Client's wedding album built while doing any of this.

Show Me The Pictures!  8)

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #104 on: September 20, 2012, 07:45:48 PM »