December 05, 2016, 07:56:55 PM

Author Topic: DxOMark Sensor Performance: Nikon vs. Canon  (Read 127048 times)

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #315 on: October 19, 2012, 12:47:48 AM »
jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. Hello DxO!

Can you imagine a food nutrition analysis website published by scientists, where they publish highest scores for palm oil and birthday cake, and lowest scores for raw fish and broccoli. After delving several levels further into their website you find other surprising and almost bizarre scores and numerics. Then after delving another 40 pages into explanatory papers and definitions, if you are sufficiently scientific in mindset, you will discover that there are no flaws in their logic or measurement methods and their system is completely consistent within itself. How useful is this website? How misleading is it?

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.

Sure he knows about many various things and he also takes the time to very carefully write out long posts filled with technical term while being careful to check his spelling and grammar and thus comes off as more authoritative than most of the people finding fault with some of his key points who quickly dash out sloppy posts riddled with spelling errors or poor grammar or less perfectly written posts because English is not their first language so everyone who doesn't know any better decides that jrista must be the one who knows what he is talking about (and the fact that he is also bashing down a site that scored the brand that most readers here use and spent a ton of money on for some scores also makes for a sneaky, and likely not at all insignificant, further bias in his favor in such a regard in minds of many readers). But for all of his knowledge he is making a number of major conceptual errors, conceptual understanding is different than spouting off this and that from various books and technical papers.

What you write may be somewhat true when it comes to their overall sensor scores and overall sub-scores to some extent but it doesn't apply to their normalization stuff or their plots.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #315 on: October 19, 2012, 12:47:48 AM »

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #316 on: October 19, 2012, 12:52:41 AM »

Still assuming that one is perpetually comparing camera sensors. Statistical knowledge about a sensor is not necessarily solely useful for the purpose of comparing them. If I buy a 40mp camera, I have no intention of comparing it to anything else. I have the intention of using it...as a 40mp camera. As such, I want the full stinkin fourty megapixels, not something less. I also want to know what my camera is capable of natively, unscaled, untainted. I want to know what my 40mp camera can do itself, not how well it might do thing X in comparison to cameras A, B, and C! In other words, I want to know its intrinsic capabilities, not its relative capabilities.

Sure you might want to know how it does at 40MP and what the sensor straight out delivers and then you have the screen plot and it's certainly valid to have some interest in that but most people also do want to know how it does relative to other options and to see whether they are maybe also giving up other things compared to other options at different MP counts or not, most people when they want to compare two cameras to each other want to use the print plots. I am gaining the ability to get a ton more res under good lighting but also giving up the ability to ever do as well for SNR or am I actually not giving up anything there compared to my old camera? etc.

« Last Edit: October 19, 2012, 03:02:36 AM by LetTheRightLensIn »

MarkII

  • Canon AE-1
  • ***
  • Posts: 71
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #317 on: October 19, 2012, 01:21:35 AM »
Can you point me to some reading material?
Try http://en.wikipedia.org/wiki/Entropy_(information_theory

(The sections on Entropy as Information Content and Data Compression are the most relevant).

tnargs

  • Rebel T6i
  • ****
  • Posts: 138
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #318 on: October 19, 2012, 01:45:14 AM »
jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. ....

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.....
.... it's been my entire point all along, as my complaint is with the fact that DXO sells "Print DR" as something it is not, by its very name, even. ...

You have a strange way of putting people down just before proceeding to agree with them ... but without ever admitting you agree with them.

jrista

  • Canon EF 600mm f/4L IS II
  • **********
  • Posts: 5334
  • EOL
    • Nature Photography
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #319 on: October 19, 2012, 01:49:45 AM »
jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. Hello DxO!

Can you imagine a food nutrition analysis website published by scientists, where they publish highest scores for palm oil and birthday cake, and lowest scores for raw fish and broccoli. After delving several levels further into their website you find other surprising and almost bizarre scores and numerics. Then after delving another 40 pages into explanatory papers and definitions, if you are sufficiently scientific in mindset, you will discover that there are no flaws in their logic or measurement methods and their system is completely consistent within itself. How useful is this website? How misleading is it?

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.

Sure he knows about many various things and he also takes the time to very carefully write out long posts filled with technical term while being careful to check his spelling and grammar and thus comes off as more authoritative than most of the people finding fault with some of his key points who quickly dash out sloppy posts riddled with spelling errors or poor grammar or less perfectly written posts because English is not their first language so everyone who doesn't know any better decides that jrista must be the one who knows what he is talking about (and the fact that he is also bashing down a site that scored the brand that most readers here use and spent a ton of money on for some scores also makes for a sneaky, and likely not at all insignificant, further bias in his favor in such a regard in minds of many readers). But for all of his knowledge he is making a number of major conceptual errors, conceptual understanding is different than spouting off this and that from various books and technical papers.

What you write may be somewhat true when it comes to their overall sensor scores and overall sub-scores to some extent but it doesn't apply to their normalization stuff or their plots.

Way to gloss over an insult man! You have some serious skill! Why not tell me what you REALLY think, huh? ;) BTW, resorting to personal attack as a means of argument demonstrates weakness in your own opinion. If you truly think people simply listen to me because I'm a dumb guy who just sounds smart, well, might not want to weaken your own position like that. I might dumly conjure up something intelligent in retaliation. Just a thought.

I have no problem with the concepts. The concepts are not the issue. The problem is that we are on entirely different pages about what the issue is. Part of that is a misunderstanding of what DXO DR really refers to, what it is defined as...by MANY people...which in and of itself is WHY we are on different pages (because it is the ambiguity and ease by which readers are mislead by DXO because of the way they describe their results, the terms they choose to use to describe certain scores) that is the problem, and what I dislike. You think I don't understand the need to normalize to properly compare IQ. I DO understand that...if you search these forums, I've argued the normalization point myself many times (and many times well before the D800 came out.)

I've been very clear about exactly what my complaint with DXO is, and very clear about the context within which I voice my complaint. Veiled insults not withstanding, I think it is you who misunderstands the argument we nay-sayers are trying to make. Either that, or you are conveniently ignoring the context, and attempting to use the notion that individuals such as myself "simply don't understand" as a means of trying to strengthen your argument.

Well, ironically, you don't need to strengthen your argument. In the context within which you are arguing, the context of non-isolated results used purely for the purpose of comparisons, DXO's definition of DR is fine and dandy, if confusing based on the way they name their results and scores. Only the people who dig into the mathematical descriptions of their results (which is only the few hypergeeky of the millions who view DXO results and base their purchasing decisions on them) will have the option of gleaning a full understanding. I do not disagree that one needs to normalize image size to determine the difference in how noise levels of a sensor might affect perceived IQ (which, btw, is a subjective or observer-based comparison...SQF would be a better, and standardized, way to provide that information). So, in terms of arguing a narrow point in a narrow context, you guys win. You won a long time ago. People just don't fully understand it because DXO DR is not really how people think about DR in a real-world context...(thus the reason the debate exists in the first place.)

Most people don't understand dynamic range as the ratio between white and black points, or noise in terms of decibels. Neither is DXO's definition of dynamic range the sole valid, concrete definition of dynamic range in the context of a digital image produced by a digital sensor. There are a variety of ways to compute DR, including some offered by standards bodies like ISO (who's ISO 15739-2003 standard aims to provide a standardized way of computing digital image DR in a more "photographic" manner. It clearly defines the types of noise it accounts for, including random or "temporal" noise (photon shot noise) that changes from shot to shot and fixed forms of noise (electronic or read noise) that remain the same from shot to shot.) There are more useful ways to describe dynamic range, and differing contexts within which definitions of dynamic range has meaning (and potentially very different meanings.)

I don't believe there is anything wrong with stating my disdain for the way DXO uses their Print DR score. I don't believe there is anything "dangerous" about me wanting a frequently-quoted resource like DXO to produce more meaningful, more useful results, even if it simply means renaming some of the terms they use to describe their results, for the benefit of potential buyers who will base radical decisions on DXO's scores (such as the dumping of an expensive Canon kit in favor of another expensive Nikon replacement kit, when for their needs they neither need the dynamic range of the D800, nor would benefit from the other traits of the D800 camera body...such as huge image size or slower frame rate when they are an action photographer who needs smaller image size and faster frame rate. Or even for the photographer who does need more DR, but might incorrectly believe the D800 offers an extra 1.2 stops of DR at native, unscaled RAW size than it actually does.)

And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.
« Last Edit: October 19, 2012, 01:55:55 AM by jrista »

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #320 on: October 19, 2012, 03:03:47 AM »
jrista, I think you have just shown that DxO should be completely ignored by every intelligent photographer.

Photographic hobbyists and professionals need test measurements that are highly relevant to their realistic needs in typical photographic situations.

What they DON'T need is a scientific measurement and subsequent aggregation that is completely inconsistent with their needs as a photographer. Complete with company-specific definitions and randomly chosen normalisation points, that one has to read all the fine print to get a grasp of how on earth they came up with that number, score, or ranking. ....

See this is precisely why jrista's 'explanations' are so damaging. He just totally left you less informed than you had been to start with judging by what you wrote above.....
.... it's been my entire point all along, as my complaint is with the fact that DXO sells "Print DR" as something it is not, by its very name, even. ...

You have a strange way of putting people down just before proceeding to agree with them ... but without ever admitting you agree with them.

Nah I messed up the message and forgot to chop off the last parts and it made it look like his words were my words, my bad, totally messed the post up. I fixed it now.

LetTheRightLensIn

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 4753
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #321 on: October 19, 2012, 03:06:18 AM »
And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

You seem to be slowly shifting your position, because you definitely didn't say that you thought it was valid or the best way to go about when comparing between different sensors before.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #321 on: October 19, 2012, 03:06:18 AM »

AvTvM

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2305
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #322 on: October 19, 2012, 05:04:22 AM »
aside of the debate re. comparing pcitures downsized to 8MP and all aspects connected to this ... one more thign that really irks me about DXO

they do NOT state anywhere on their webseite, what lens/es they use in their "sensor" tests, which in reality are not "sensor tests" but rather comparisons of downsized RAW images obtained by taking test pictures using a lens, a camera with sensor, hardware and firmware .. so comparisons of "lens-sensor-hardware-firmware"-combinations. They also state this here:   http://www.dxomark.com/index.php/Publications/DxOMark-Insights/DxOMark-Score

I doubt they are targeting a laser beam at a naked sensor and use their own electronics - hardware and standardized firmware - to measure "pure sensor performance".

There must be a reason, why they are so clandestine about wht lesnes they use for their testeing. Until they fuly dislose all relevant information regarding their test-setup, i completely disregard their results. For all we know, they might be testing Canon cameras with the 50/1.8 and other cameras with a Zeiss 100 Macro.   

Razor2012

  • 5D Mark III
  • ******
  • Posts: 618
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #323 on: October 19, 2012, 10:24:06 AM »
Hard facts dynamic range

Do you have a link to those figures?
5D MKIII w grip, 70-200 2.8L IS II, 24-70 2.8L II, 16-35 2.8L II, 100 2.8L IS macro, 600EX-RT

jrista

  • Canon EF 600mm f/4L IS II
  • **********
  • Posts: 5334
  • EOL
    • Nature Photography
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #324 on: October 19, 2012, 10:44:08 AM »
And for the record, my argument against DXO's "Print DR", even if Print DR is "valid within it's own context", is consistent across brands. From a pure statistical comparison standpoint, the use of the Print DR number is useful as a scalar, unitless score for comparing subjective image quality across sensors. However I do not believe, for any camera, Nikon, Canon, etc., that it actually provides any amount of meaningful information that would tell a potential buyer who HAS made a decision about what camera to purchase how much exposure latitude they might actually have when post-processing their RAW images (and it is exposure latitude...the ability to "recover", particularly shadows, that people think about when they read "dynamic range" in the context of digital cameras). Rename it to "Subjective IQ Score" for ALL cameras, eliminate the associations with "print" and "landscape" (which instantly puts it into context the majority of readers will see and incorrectly interpret), and my complaint will instantly disappear.

You seem to be slowly shifting your position, because you definitely didn't say that you thought it was valid or the best way to go about when comparing between different sensors before.

"It", being normaizing scale between images, to compare "the amount of noise" in a sensor, is something I do not have a problem with. I don't normally equate the amount of noise in an image with dynamic range, not the kind of dynamic range I think about when I hear the term. Based on elflord's comments so far, it seems based on DXO's definition of Print DR that normalizing images for comparison is just that, but nothing really any more than that. I do have a problem with referring to the difference between two normalized images a change in dynamic range, and I've already explained in extensive detail why. I won't be explaining it again.

MarkII

  • Canon AE-1
  • ***
  • Posts: 71
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #325 on: October 19, 2012, 11:29:27 AM »
I doubt they are targeting a laser beam at a naked sensor and use their own electronics - hardware and standardized firmware - to measure "pure sensor performance".
Actually, it is quite possible that they perform the public sensor tests without any lens mounted at all. All you need is to be able to do is to illuminate the sensor in a consistent and calibrated manner, and the less that you put in the way the better.

rpt

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2626
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #326 on: October 19, 2012, 12:24:58 PM »
OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here ;) } or you just need a good copy...

Razor2012

  • 5D Mark III
  • ******
  • Posts: 618
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #327 on: October 19, 2012, 01:20:15 PM »
OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here ;) } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?
5D MKIII w grip, 70-200 2.8L IS II, 24-70 2.8L II, 16-35 2.8L II, 100 2.8L IS macro, 600EX-RT

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #327 on: October 19, 2012, 01:20:15 PM »

tron

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2845
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #328 on: October 19, 2012, 01:27:07 PM »
OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here ;) } or you just need a good copy...
In his pic Mikael lifted the shadows at a specific place ALOT! You will have to present a similar case to prove your point and not compare apples to oranges.

rpt

  • Canon EF 300mm f/2.8L IS II
  • ********
  • Posts: 2626
Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #329 on: October 19, 2012, 01:27:35 PM »
OK, so I took a few shots with my 5D3 (with the 24-105) in a darkish room. The mode was Av at f8 with ISO set at 12k8 and 25k6 and the noise level in the raw pics are nowhere near Mikael's picture noise. Mikael, may be you need to replace your copy of the 5D3. Either it does not like you one bit {notice no fractional reference here ;) } or you just need a good copy...

Or maybe, what's with the hard-on for Canon?
Oh dear! Are you saying that you also get the same noise levels as Mikael when you take pictures? Is your camera still under warranty? I hope so.

canon rumors FORUM

Re: DxOMark Sensor Performance: Nikon vs. Canon
« Reply #329 on: October 19, 2012, 01:27:35 PM »