October 24, 2014, 05:23:21 PM

Author Topic: DxOMark Sensor Performance: Nikon vs. Canon  (Read 82101 times)

rpt

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 2219
  • Could not wait for 7D2 so I got the 5D3
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #435 on: April 21, 2013, 08:18:27 AM »
As my posts were not constructive, I have removed them.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #435 on: April 21, 2013, 08:18:27 AM »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14749
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #436 on: April 21, 2013, 08:48:06 AM »
Frankly, IMO, I find far too much of a potentially good thread is often taken up by off-topic and unconstructive posts with an inflammatory tone, making it tedious for those who want to follow it ...

Many potentially useful threads have been derailed by the very thing the that instigates the 'humor' or outright attacks that lead to thread moderation/locking.  I'd say the inflammatory posts are those posts that are made knowing exactly what outcome will result.  Before ankorwatt/Mikael had his Mikeal account and all his posts deleted, we'd seen that awning comparison in at least 10 threads, and countless QPcards on barbecues, sheds, and now pianos.

Go to an astronomy forum, and in any thread which mentions the solar system, post a modern diagram of ours and state that Copernicus was right and Ptolemy was wrong.  That's just stating a fact, but after a while its going to start pissing people off...and that's not even considering the fact that there aren't many devotees of the Ptolemaic model. 

Maybe there are people on Nikon forums who repeatedly post the same example images to show how certain of Canon's lenses are better than the Nikon equivalent.  If so, that's just as inflammatory.  Should Mikael be insulted for posting the 'truth' that the D800 has a wider DR?  No.  But repeated posting of the same contrived examples is the epitome of tedious, and downright annoying.  Since he's repeatedly posting those examples in response to those he's argued with on many previous occasions, he knows exactly the response it will provoke.  That he does it anyway, again and again, and indeed seems to have created new accounts just to do so, says a lot about his character. 
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

TrumpetPower!

  • 1D Mark IV
  • ******
  • Posts: 951
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #437 on: April 21, 2013, 09:34:06 AM »
with what should I answer for example trumpet power when he declare that 5dmk3 and 1dx has the same exposure latitude that d800 ? = large DR?

Mikael, I have never, ever claimed any such thing.

My point is, and always has been, that, though the D800 offers marginally more dynamic range, the dynamic range of the 5DIII and 1DX is already most substantial. When you can take a 5DIII image that's been so grossly underexposed that it's mostly black on the preview window and easily rescue it into a clean file suitable for extreme enlargements, it's meaningless to bitch about how shitty its dynamic range is -- which is basically all you do here. It's as meaningless as bitching that minivan x is crippled because it "only" has a top speed of 95 mph, and we know this to be true because minivan y has a top speed of a whopping 105 mph.

So who cares that you can rescue even more grossly underexposed images with a D800?

If you can't get clean images in bright light -- and all you do here is post very noisy images taken in very bright light -- then the problem isn't even close to being with the equipment. The problem is the gross incompetence of the person operating the equipment.

Lots of people here have posted examples of very clean images taken with the 5DIII and 1DX in extreme situations. And those same people have tried to explain to you the basic, introductory-level photographic techniques they used to achieve those results. Yet you continue to insist that those people haven't a clue...while you go on to post yet more really bad snapshots plagued with all sorts of technical problems, all the while insisting that you know more about technique than those who don't have your problems.

Dude, the problem isn't your gear. Your gear is just fine. The problem is with the person who's operating your gear.

Cheers,

b&

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14749
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #438 on: April 21, 2013, 09:37:16 AM »
Neuro
with what should I answer for example trumpet power when he declare that 5dmk3 and 1dx has the same exposure latitude that d800 ? = large DR?  well,  I show it  with a comparison from 1dx and d800  ,

Do you think your comparison convinced him?  Since he's seen it at least a dozen times and still responded the way he did, it's pretty obvious that showing him the same comparison a 13th, 20th, or 200th time won't convince him, either.  Did you notice that I've given up responding to your comparison posts?  It became pointless.

There's a sign over the bar in The Squealing Pig, a pub near where I work, taken from a Robert Heinlein quote: "Never try to teach a pig to sing; it wastes your time and it annoys the pig.

So, I'd suggest that you simply don't answer at all.  I'd suggest the same to TrumpetPower.  The issues are important to some people here, but the repeated arguing is really damn pointless.  Neither of you will convince the other, just accept it and move on. 

</rant>
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

Aglet

  • 1D X
  • *******
  • Posts: 1037
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #439 on: April 21, 2013, 03:09:03 PM »
...
..
So, I'd suggest that you simply don't answer at all.  I'd suggest the same to TrumpetPower.  The issues are important to some people here, but the repeated arguing is really damn pointless.  Neither of you will convince the other, just accept it and move on.  </rant>

That is good advice all around.
I think that too often people are compelled to post something in response to a post, even if it's not contributing to the actual technical aspect.

If your momma never told you, "If you don't have something nice to say, don't say anything at all." ..
Well, here it is.

Amending it for an internet based discussion forum would likely go something more like,
"If you can't contribute something useful to the topic, please reconsider posting anything at all."

I have to commend all involved that, at least from what I've seen recently, the level of rancour amongst differing viewpoints has taken a more civil route, even on this topic which has been beaten to death since I joined up a year ago to discuss this very issue.

So thank-you, various posters, for doing your best to remain civil.  It improves what is already one of the better places for photographic discourse.

Now for the tech contribution to the topic: I think we'll find that, as Canon moves into DiGiC 5 products, we'll see somewhat less of the general noise and banding issues which plagued a few of the DiGiC 4 equipped models and may have been less of a problem with earlier DiGiC 2 & 3 bodies.

A recent chat with a Canon rep, busy as ever manning a booth, solo in front of a large crowd of curious onlookers, who said that there are many improvements in DiGiC 5 for high AND low ISO noise issues... A point he was eager to make when I told him that I'd dumped all my DiGiC 4 bodies due to poor noise performance for my uses.
He then handed me a 6D and asked me to play with it a while and let him know what i thought of it compared to the 5D2 I had... I know he already knew the answer to that. :)

Gotta say, other than the reduced overall noise and noticeably reduced FPN on the 6D, the thing I really like about it is the smoother quieter shutter.  The 5D2 had such a crude THWACK! along with significant body shake induced by that big mirror being let loose and poorly damped.  The 6D feels more like a nicely balanced match-rifle than the shotgun-like 5D2.  it's even nicer, i think, than the quick CRUNCH sound the D800's shutter makes.
I haven't had a chance to really shoot with a 6D, likely won't unless I buy one.  Likely won't do that either until I find a compelling enough reason to go back to Canon for FF.  And about the only reason that will be is if my hankering for a tilt-shift 24mm is not satisfied by the upcoming Samyang for F-mount.
Plunking a 6D on the back of Canon's fabulous 24 and 17mm TS lenses will likely be the most practical solution for some of what I want those lenses for.  This might have been speeded up, somewhat, if the darn resale value for D800s was better. All those factory refurbs have dropped my resale value to the point where I now will likely have to keep using both of them for at least a year to get more value from them.

bdunbar79

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 2600
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #440 on: April 21, 2013, 11:30:01 PM »
See.  Nobody cares.
2 x 1DX
Big Ten, GLIAC, NCAC

RLPhoto

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3507
  • Gear doesn't matter, Just a Matter of Convenience.
    • View Profile
    • My Portfolio
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #441 on: April 22, 2013, 12:25:55 AM »
See.  Nobody cares.

Yup, especially when no photography is in the posts.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #441 on: April 22, 2013, 12:25:55 AM »

sarangiman

  • Canon 6D
  • *****
  • Posts: 374
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #442 on: May 09, 2014, 08:08:27 PM »

I see the difference in noise and it sure is there! But, this example is flawed in my opinion since the exposure in these photo's was just wrong. I have seen that current Sony sensors offer better shadow recovery, no doubt about that. But in this particular situation it was not necessary to have such an underexposed image and the same photograph could be achieved by both Canon and Nikon camera when properly exposed.


But in another scenario, this exposure could have been needed to avoid clipped highlights. So what you're really complaining about is that I didn't - in my infinite free time during which I happen to have both a D800 & 5DIII on my hands - find a scene with enough DR. That's a fair point, sure, but also - as you can hopefully see - a bit unfair. Because the end result would be the same.

And, frankly, that's what I'd say to everyone here who, in all the comments that followed my original post, indicated that my shot was underexposed. Meanwhile, just a few minutes before this scene, the sun was high enough in the sky that the sky would have blown. So are you complaining that I didn't shoot just a few minutes earlier when there was more scene DR? Would that have changed the end result? Are you arguing that there do not exist scenes that require more DR than what Canon sensors can provide?

Furthermore, according to my histogram when I was shooting, my channels were clipped in the sky! Therefore, the camera led me to believe that I did accurately 'expose to the right and for highlights'. So this is the result an informed photographer would have gotten anyway. Now, if you're going to complain that I didn't have a computer with me to actually check the RAW values as I was shooting, then...

There are many valid suggestions you could make. *Perfectly* nailing exposure within 1/3 EV to get your highlights just short of clipping is not one of them (though it is a noble effort). Not the least because most cameras don't accurately tell you when you're just short of clipping in RAW - in my experience, they tell you you're clipped well before you're actually clipped.

Now, you could have told me to bracket, or bracket for HDR. Valid points - but sometimes that's not an option for certain types of photography. My point is: the conclusion doesn't change. D800/Sony sensor performance gives you much more DR and much more processing latitude. That's useful to some people in certain scenarios.

Also from http://www.luminous-landscape.com/essays/on_safari.shtml#suckout :

Quote
The bottom line is that the vast majority of the tonal information that a sensor can capture lies on the far right side.  In fact, usually 75% of the total tonal information a sensor can capture lies in the small right hand area of the two top F/stops just below pure white.  And yet, almost all cameras leave the factory calibrated to center the histogram instead of moving it as far to the right as possible.  To make matters worse, the screens in the backs of cameras are also calibrated to show a good exposure with the histogram centered. This is nonsense!

So what are the consequences of Tonal Suckout?  As the name implies, it is as if someone grabbed your image and sucked the tonality out of it,  If there are millions of shades of green in a landscape, you may end up with a few dozen.  If there is a terrific richness of tonalities in a face, you will end up with a small sub-set that makes the face look bland.

Maybe useful to keep in mind the next time you want to get an underexposed image!

Just FYI, that's actually a fallacy. Read Emil Martinec's extremely informative treatise on noise: http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#bitdepth

Essentially, any extra 'tones' will oversample shot (statistical) noise. Consider a signal of 200 photons generated ~100 e- of signal (QE = 0.5). At ISO 100, say we bin 4 electrons 1 DN (digital number, or increment, in the digital file). So that signal is recorded as 25 in the RAW file - pretty darn low (dark). Now, b/c of shot noise, that signal already has a stdev of 10 e- (sqrt of 100), or 2.5 DN. So there you go, you have enough bit depth to still sample the noise in the signal. So it's not limited bit depth/tonality that's the problem - it's the amount of light itself. You expose to the right (ETTR) to minimize statistical/shot noise.

Or to avoid running into the noise floor of Canon sensors... make sense?

sarangiman

  • Canon 6D
  • *****
  • Posts: 374
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #443 on: May 09, 2014, 09:09:37 PM »

You are still missing the point of argument here, though. Yes, it does work well to underexpose those cameras by four stops, then lift the shadows in post. But doing so is an unrealistic test from a real-world standpoint. All it tells you is that IF someone were to accidentally expose their scene incorrectly by a HUGE amount (some 16x incorrectly), then they would have a greater ability to recover. Purposely underexposing by four stops for the purpose of comparing cameras is also an unrealistic real-world comparison.


Tell that to Sony, who in their A7S is introducing S-Log2 gamma curve application that raises shadows 3 stops and rolls off amplification in highlights. In other words, ISO 800 in shadows, but ISO 100 in highlights. All software-based, since actual ISO amplification off the sensor remains at ISO 100 levels.

If that's a reasonable push for videographers, why not for still photographers? Why is +4EV suddenly entirely outlandish, when a camera manufacturer is now allowing you a +3EV push as an option in-camera?

The levels to which my post was misconstrued was amazing, which is why I chose not to come back to this thread (until now). I mean, I even admitted I still shoot Canon and work around its limitations... but boy have I been so much happier with the Sony A7R sensor (barring shutter shock problems) over the past 6 months. On a boat ride in Cambodia at sunset I set my exposure for the sky and shot away without a care for my exposure - raising shadows 5 EV in post to balance out the shadows of detail & people's faces in the boat (all while holding back the exposure in the sky).

The point stands: +4 EV shifts are not unreasonable. It's the difference btwn ISO 100 and ISO 1600. If such pushes were unreasonable, why would Magic Lantern have implemented a dual ISO feature that people use to shoot ISO 100 and ISO 1600 simultaneously?

Oh, incidentally, Magic Lantern's dual ISO feature wouldn't even be needed on a Sony sensor - because you could perform the ISO 1600 push digitally! Which was the whole point of my original post.

I do still shoot Canon professionally for weddings/engagements, but that doesn't mean I don't wish it had the Sony A7R sensor. It's common knowledge now that some aspects of Canon's sensor tech are severely outdated. ChipWorks stated:

Quote
Of the Canon DSLRs analyzed, the imaging chip has remained analog, with Analog Devices’ analog front end (AFE) chips handling A/D conversion en route to the Digic-branded ISPs. Perhaps the column-parallel ADCs favored by others can’t be implemented using 0.5 µm design rules, but more likely Canon is satisfied with its system design and performance.

http://www.chipworks.com/en/technical-competitive-analysis/resources/blog/full-frame-dslr-cameras-canon-stays-the-course/?lang=en&Itemid=815

There are even other, far reaching implications of low base-ISO DR. It limits your ability to retain DR while shooting low light scenes (since you have to raise the ISO amplification in camera, as opposed to selectively raising it in post-processing from an underexposed, lower ISO file).

As for people who don't run into these problems much - YMMV based on the DR of the scenes you shoot. Posting an example of a scene you think has high DR, but where you didn't see offensive noise in shadows, is not a way to prove a camera has just as much DR as another. It's just a way of showing that it had enough DR for that particular scene. Which is why we have controlled tests, like the ones DXO perform.

By the way, thanks to those that stepped in to point out why the misinterpretations of my presentation were flawed. The truth is, low read noise sensor technology will enable new features for photographers only dreamt of in the past. To sit here complaining that some tests push (pun intended) technology too much to show unflattering limitations is, in my opinion, incredibly short-sighted. Which is especially apparent when you begin to see manufacturers and software developers doing +3 and +4 EV pushes of shadows in firmware/software (Sony, Magic Lantern, respectively). That's pretty telling.

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4482
  • EOL
    • View Profile
    • Nature Photography
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #444 on: May 09, 2014, 11:27:37 PM »
Hey Sarangiman, welcome back. Way to dig up an OOOLD thread, man! :P This was years ago...

Before I dive in...which "original post" of yours are you referring to? This is such a long thread, and it went through so many phases...

The point stands: +4 EV shifts are not unreasonable. It's the difference btwn ISO 100 and ISO 1600. If such pushes were unreasonable, why would Magic Lantern have implemented a dual ISO feature that people use to shoot ISO 100 and ISO 1600 simultaneously?

Well, first...I wasn't so much arguing against the use of 4 EV shifts "period". It was more an argument against Mikael's persistent use of particularly extreme "examples" to push is consistently extreme ideas about how uncompetitively bad Canon cameras and sensors are and (at least in his mind) would always be. Now that Mikael is gone, I think we can have a far more reasonable discussion on this subject. I fully agree that more DR is not a bad thing, and in many circumstances (sunsets being an ideal one) potentially extremely useful.

I'd offer that more accurate terminology could be used here, though. The difference between ISO 100 and ISO 1600 is four stops of LOST dynamic range. When you set ISO 1600, you lose four stops of DR, or lose 16x the fineness of levels between your black point and white point, and end up with 16x more noise. I understand what your getting at, though, your thinking about what ML is doing with Dual ISO, however I still wouldn't say that's the difference between ISO 100 and ISO 1600. To be most accurate, it would really be the theoretical difference between ISO 100 and ISO 6. :P (If you catch my drift).

Sony Exmor sensors still use analog amplification, BTW. When you set a higher ISO, the same fundamental rules apply to Exmor pixels as apply to Canon pixels. You have some charge level in your pixels, you amplify them BEFORE readout, then read (which at that point introduces the read noise). If you don't amplify before reading, even with the D800, noise levels when lifting an ISO 100 shot by four stops are higher than if you just shot at ISO 1600...the difference just isn't as extreme as when you do the same thing with the 5D III.


As for people who don't run into these problems much - YMMV based on the DR of the scenes you shoot. Posting an example of a scene you think has high DR, but where you didn't see offensive noise in shadows, is not a way to prove a camera has just as much DR as another. It's just a way of showing that it had enough DR for that particular scene.

Totally agree. Anecdotal claims are no substitute for proper controlled testing.

Which is why we have controlled tests, like the ones DXO perform.

TL;DR <RANT> :P

There ARE problems with DXO's results, though. Their direct measures are fine, they are pretty much the only thing of value that DXO produces. The problem with DXO is that not all their "measures" are actually measured, many are derived and extrapolated from actual measures, and they are NOT entirely clear about their exact methodology or formulas. They HAVE been caught on a couple occasions changing their results after the fact when flaws in their methodology were found, without any explanation (at all, not even a blog post or article) of what was changed, how, or why, leaving those who identified or knew about the flaws left to wonder what kind of biased re-weighting or whatever was performed in the black box behind the scenes to "fix" the problems (assuming anything was actually fixed, and that data wasn't simply massaged). DXO has shown considerable bias with their lens reviews, on a very consietent basis, producing very arbitrary results and scores that show certain lenses to be equivalent when they are nothing of the sort, and conversely show other lenses to be inversely related to their actual status (i.e. the EF 50mm f/1.4 lens outscores the EF 600mm f/4 L II lens for the sole reason that it has a wider maximum aperture...results that COMPLETELY undermine the 600mm's vastly superior traits...that clearly indicates poorly balanced lens score weighting that heavily leans towards T-stops as being the most fundamentally important trait as far as DXO is concerned.)

The biggest single issue I think most of the people who complain about them (which does indeed include myself) have with DXO is their often outright and obvious bias, which is exhibited in their scores. If DXO would completely eliminate, drop entirely, the whole notion of linearly scoring the complexity of cameras and camera traits via a single scalar number, I think that would go a very LONG way to improving their reputation among those who hold them in fault. I think that scoring should be eliminated entirely, as I think it is a generally fallacious approach to take when the goal is to produce "scientific" results. When you throw in the fact that many of DXO's scores are weighted, and that some of those same scores are based on "measures" that are not actually measured, they are extrapolated via simple and pure mathematical formulas (and therefor do not actually match reality, when compared with other REAL measures from other testers), DXO is quite simply not as scientific as they claim to be.

That's my core problem with DXO. I like some of their literal measures, ones taken directly from original data. An example would be their Screen DR measure...it's based directly on samples taken from RAW image files. That's a true, honest measure. Print DR, on the other hand, is a purely mathematical EXTRAPOLATION, derived from the Screen DR measure. They call Print DR a "measure", but it is nothing of the sort. Their simplistic and pure mathematic formulas are absolutely no substitute for actually measuring the change in noise levels after actually downsampling a full-sized RAW image to their standardized 8x12" "print" output. The only reason the D800 could possibly score some 14.4 stops of Print DR when downsampled is because the Print DR "measure" is a mathematically computed number, a computation that assumes 100% perfectly ideal circumstances. Reality cannot be ideal, so their formulas are missing aspects of real-world downsampling. If they did actually measure downsampled images, they would very likely produce a less significant difference between the 13.2 stops Screen DR and whatever Print DR ended up being if properly measured, and the differences would shrink as pixel size shrunk (fill factor in FSI sensor designs shrinks with an increase in pixel density, as a greater and greater percentage of sensor die area becomes dedicated to readout wiring and the like...even with reductions in fabrication process, fill factor still shrinks, and when you throw in extra logic onto the sensor die, which is usually the case with smaller processes, fill factor shrinks further).

Sorry to dive into the diatribe on DXO again, but the reasons why there are complaints about them are well-founded, and I think it's important to be clear that they aren't just "because DXO is biased against Canon". I don't really care what the brands are, that honestly doesn't matter to me. There are definable problems with the way DXO does things, because the way they do things allows bias, and bias is quite obvious in many of their scores. The day Canon comes out with a 16-bit ADC and scores 17.5 stops of Print DR on DXO, I guarantee you I'll be making the EXACT same argument AGAINST their Canon scores...because it will be the exact same deal as the D800 and it's 14.4 stops of DR with a 14-bit ADC.

DXO's scores suck. The way they score is very unscientific, regardless of whether it is self-consistent. They should just stick to the raw measurements, they should actually literally measure downsampled versions of each image instead of just running a simple mathematical extrapolation, publish clear and concise facts about ALL of their testing methodologies for EVERYTHING (i.e. be completely open, get rid of the black box), and ditch scoring entirely.

They should rebuild their comparison engine to allow people to pick cameras, and let an openly defined algorithm compute comparative differences and highlight them in a meaningful manner. That would eliminate any option for bias, eliminate this whole inane notion of reducing complex products to single numbers, and potentially educate DXO viewers about the cameras they are comparing, without any DXO opinions sitting in the middle. Just share the raw facts.

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14749
    • View Profile
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #445 on: May 10, 2014, 09:24:57 AM »
Speaking of TL;DR, let me explain...no, there is too much, let me sum up:

DxOMark Scores = Biased Scores = BS.

'Nuff said.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4482
  • EOL
    • View Profile
    • Nature Photography
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #446 on: May 10, 2014, 11:42:51 AM »
Speaking of TL;DR, let me explain...no, there is too much, let me sum up:

DxOMark Scores = Biased Scores = BS.

'Nuff said.

 ;D ;D ;D SUPERB summary! My thoughts in a nutshell.  :P

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #446 on: May 10, 2014, 11:42:51 AM »