October 01, 2014, 10:58:38 PM

Author Topic: Canon EOS-1D X DXOMark Sensor Scores  (Read 32868 times)

dtaylor

  • 5D Mark III
  • ******
  • Posts: 737
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #90 on: October 15, 2012, 06:22:31 PM »
What should be remembered is that doubling the resolution adds:

    3dB to the normalized SNR
    0.5 bit to the normalized DR

So according to DxO if we drum scan 4x5 Velvia, a 6 stop film, and down sample to 8 MP, we will end up with 9 stops of usable photographic detail?  ::)

Go ahead and try it  ;D

canon rumors FORUM

Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #90 on: October 15, 2012, 06:22:31 PM »

RLPhoto

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3468
  • Gear doesn't matter, Just a Matter of Convenience.
    • View Profile
    • My Portfolio
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #91 on: October 15, 2012, 06:24:39 PM »
Canon 61 pt AF > Nikon 51 pt AF. done.

Nonsense. Point count != superior AF. In my experience the 7D's AF is superior to the D7000's, practically on par with the earlier 45 point 1D bodies and several Nikon 51 point bodies, and it only has 19 points.

I don't honestly know who has the best AF right now because I don't spend sufficient time with the top tier bodies. But AF is an extremely complicated thing to objectively test, and subjective opinions are open to bias and error. AF performance can also be better on body A for situation 1, but better on body B for situation 2, etc, etc. Not to mention that lenses are at least as important as anything in the body, and it's a mistake to assume similar lenses from two manufacturers have similar AF.

Even with a case as one sided on paper as the 6D and D600 you can't conclusively say one body is always better than another. There's little point in discussing the top tier bodies unless you happen to own and shoot both regularly under a range of challenging conditions.

You must have never used the sheer awesomeness of the canon 61-Point AF system And yes, I've used nikons excellent 51-point system as well. So yes, I can say the Canon AF system is better than nikons right now. Plus, I never said that point count makes a better AF system, because then S0&y would be the best then.  ::)

My Xsi was good
My 7D is Better
but my 5D3 is Great.
« Last Edit: October 15, 2012, 06:26:54 PM by RLPhoto »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14537
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #92 on: October 15, 2012, 06:48:13 PM »
This is the belief of a math geek who spends too much time with graphs and not enough time with real photographs. Down sampling throws away noise and detail. Dynamic range is the range of useful photographic detail. It is not the simple difference between two numbers on a graph.

Try this: shoot 4x5 Velvia. Drum scan it. Down sample to 10 MP. Did you all of a sudden gain 3-4 stops of shadow detail? No? Hmmm...

+1

Side note: arguments like this are why I hate DR by software analysis. It has zero bearing on the real world. It just leads to paragraphs and paragraphs of irrelevant and pointless theorizing. Shoot a Stouffer transmission step wedge and look at it with your own two eyes. That tells you what you can expect in the real world.

FWIW, one of the things that I do at some point after getting a new camera body is put a Stouffer T4110 on a light box, and use that to determine the 'typical' DR I get across the available ISO settings, and how far I can push that DR using more intensive post-processing of the RAW file.  To me, that's a relevant empirical measure that actually helps me determine exposure when taking real pictures in the real world.  My DR estimates are always less what DxO determines manipulates and calls a Landscape Score.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #93 on: October 15, 2012, 06:49:54 PM »
You keep repeating it, but it's still wrong. It's wrong because the low end of the dynamic range is determined by SNR, and SNR changes when you downsample.

This is the belief of a math geek who spends too much time with graphs and not enough time with real photographs. Down sampling throws away noise and detail. Dynamic range is the range of useful photographic detail. It is not the simple difference between two numbers on a graph.

Of course it throws away noise AND detail, the fact that there is a noise-detail tradeoff is the point !

The point is that you normalize all cameras to the same resolution so that all cameras are at the same level of detail. The test is, how much dynamic range does each camera have for some given resolution.

I understand that there's a whole other can of worms regarding at what point "useful" dynamic range starts -- is it SNR = 0db, 10db, or maybe even 20db ? But I don't think that's what you are referring to.

It doesn't make any sense to compare data points which represent different tradeoffs/choices on the noise-detail continuum. Now, to your example --

Quote
Try this: shoot 4x5 Velvia. Drum scan it. Down sample to 10 MP. Did you all of a sudden gain 3-4 stops of shadow detail? No? Hmmm...

Here's a different test for you -- sample it at 10MP, create another image at 40MP. Then compute the "blackpoint" (SNR = 0db) of the two images.

Now I take it that we agree that the images both have the same "true"  dynamic range, but I put it to you that the measured blackpoint on the 40MP will be at a higher luminance level, and therefore the measured per-pixel dynamic range will be lower on the 40MP image.

Without normalization, your measured dynamic range has more to do with how the data was sampled and less with this useful photographic detail that you're so fond of.

Quote
Shoot a Stouffer transmission step wedge and look at it with your own two eyes. That tells you what you can expect in the real world.

For what it's worth, I am in agreement with you here -- if a test site were to include such shots, and allow the user to judge, it would be quite valuable. Of course it wouldn't resolve the debate about whether to "normalize" to a common resolution (or for that matter, what to do about different ISOs).
« Last Edit: October 15, 2012, 07:14:40 PM by elflord »

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #94 on: October 15, 2012, 06:59:00 PM »
FWIW, one of the things that I do at some point after getting a new camera body is put a Stouffer T4110 on a light box, and use that to determine the 'typical' DR I get across the available ISO settings, and how far I can push that DR using more intensive post-processing of the RAW file.  To me, that's a relevant empirical measure that actually helps me determine exposure when taking real pictures in the real world.  My DR estimates are always less what DxO determines manipulates and calls a Landscape Score.

Yes, probably because DxO use 0db as their baseline. A 0db SNR isn't really "usable" even by the most liberal of standards. In comparison,  even at the very permissive ISO settings on most modern digital cameras, most have > 10db SNR at 18% gray, max ISO.

So assuming you start from the screen score, you could lose a stop or two depending on how stringent your idea of "usable" dynamic range is.

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4734
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #95 on: October 15, 2012, 07:21:30 PM »
You keep repeating it, but it's still wrong. It's wrong because the low end of the dynamic range is determined by SNR, and SNR changes when you downsample.

This is the belief of a math geek who spends too much time with graphs and not enough time with real photographs. Down sampling throws away noise and detail. Dynamic range is the range of useful photographic detail. It is not the simple difference between two numbers on a graph.

Try this: shoot 4x5 Velvia. Drum scan it. Down sample to 10 MP. Did you all of a sudden gain 3-4 stops of shadow detail? No? Hmmm...

Side note: arguments like this are why I hate DR by software analysis. It has zero bearing on the real world. It just leads to paragraphs and paragraphs of irrelevant and pointless theorizing. Shoot a Stouffer transmission step wedge and look at it with your own two eyes. That tells you what you can expect in the real world.

Thanks for the backup, dtaylor. :) I think this nails it on the head (eloquently so...I am not so eloquent):

Quote
Dynamic range is the range of useful photographic detail. It is not the simple difference between two numbers on a graph.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4734
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #96 on: October 15, 2012, 07:31:55 PM »
Concerning all the discussions about whether or not 8Mp is a reasonable number for the 'print' scores:

DxO say "...high-resolution sensors will gain more SNR, DR, TR and CS when reduced to a lower reference resolution. For DxOMark Sensor Overall Score and Metrics, we chose a reference resolution equal to 8 Megapixels, which is a bit less than a 12" x 8" print with a 300dpi printer. However, any other resolution can be chosen, as doing so only shifts the normalized values by a constant (because the reference resolution appears only as a logarithm in the formulas above).
What should be remembered is that doubling the resolution adds:

    3dB to the normalized SNR
    0.5 bit to the normalized DR
    0.5 bit to the normalized TR
    1.5 bit to the normalized CS"

I'm guessing that at the time they chose 8Mp there were probably no cameras capable of anything over say 25Mp other than a couple of MF models. And many sensors were 8 or 12Mp. That's just a few years ago.

This is exactly what is wrong with DXO's mentality, here, though:

Quote
DxO say "...high-resolution sensors will gain more SNR, DR, TR and CS when reduced to a lower reference resolution.

The sensor gains NOTHING...its a fixed construct with fixed attributes. The only thing that changes when scaling an image down is the image. The only thing that indicates is the capabilities of the software doing the downscaling...it tells you little of the sensor (if it really tells you anything...I think the point is debatable.) Whatever you do with an image in post, that has no direct bearing on what the sensor is actually capable of. The only things that tell you about the sensor are measurements. DXO does provide that...in their "Screen" statistics.

The only way to discuss sensors objectively is to reference actual measurements. From the outcry of the D800 fanatic, one would think it actually had a really crappy sensor by the way they demand images be normalized in size to allow the only form of objective comparison a study of a sensor could possibly endure. Ironically, the D800, D600, and D7000 all still outperform Canon sensors when you only look at the objective sensor measurements, so the inane debate about Print DR and its subjectivity is really just that...inane.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

canon rumors FORUM

Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #96 on: October 15, 2012, 07:31:55 PM »

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4734
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #97 on: October 15, 2012, 07:35:04 PM »
Canon 61 pt AF > Nikon 51 pt AF. done.

Nonsense. Point count != superior AF. In my experience the 7D's AF is superior to the D7000's, practically on par with the earlier 45 point 1D bodies and several Nikon 51 point bodies, and it only has 19 points.

I don't honestly know who has the best AF right now because I don't spend sufficient time with the top tier bodies. But AF is an extremely complicated thing to objectively test, and subjective opinions are open to bias and error. AF performance can also be better on body A for situation 1, but better on body B for situation 2, etc, etc. Not to mention that lenses are at least as important as anything in the body, and it's a mistake to assume similar lenses from two manufacturers have similar AF.

Even with a case as one sided on paper as the 6D and D600 you can't conclusively say one body is always better than another. There's little point in discussing the top tier bodies unless you happen to own and shoot both regularly under a range of challenging conditions.

You must have never used the sheer awesomeness of the canon 61-Point AF system And yes, I've used nikons excellent 51-point system as well. So yes, I can say the Canon AF system is better than nikons right now. Plus, I never said that point count makes a better AF system, because then S0&y would be the best then.  ::)

My Xsi was good
My 7D is Better
but my 5D3 is Great.

While I think that in the aggregate the anecdotal evaluations of Canon's 61pt AF system would generally concur, that it is currently the best available as it produces a higher rate of keepers, and really good keepers at that...I have to agree with dtaylor that it is all still anecdotal. It can be tough to objectively test an AF system, so we can't really claim that any test results to date are anything other than subjective. Thats not terrible though...statistical analysis of subjective tests can still produce useful results when aggregated, and I think those results speak resoundingly in favor of the 1D X's AF system.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #98 on: October 15, 2012, 08:47:40 PM »
The sensor gains NOTHING...its a fixed construct with fixed attributes. The only thing that changes when scaling an image down is the image. The only thing that indicates is the capabilities of the software doing the downscaling...it tells you little of the sensor

Again, the sensor does not consist of a single pixel. DxO's normalization method does not use any fancy downsampling algorithms, it's based on simple averaging.

Quote
Whatever you do with an image in post, that has no direct bearing on what the sensor is actually capable of.

It has nothing to do with post processing, it is about resampling so that the resolutions are the same. Simple averaging. Not applying noise removal algorithms, etc.

Quote
The only things that tell you about the sensor are measurements. DXO does provide that...in their "Screen" statistics.

The problem is that the screen rating, using their definitions, using a 36 megapixel camera is not comparable to the same rating with a 10mpx camera. If you want to ask some question like "how do these two cameras compare if I use the same noise/resolution tradeoff for both cameras, then you need the print rating.
Quote
The only way to discuss sensors objectively is to reference actual measurements.

What is your definition of a "measurement" ? The screen score is hardly a single measurement. It's the number of stops between the measured saturation point (possibly with some extrapolation), and the (probably interpolated or extrapolated) SNR=0db point on the noise / luminosity curve. What difference does it make if I throw in an extra term to my formula which normalizes to 8 megapixels ? It does change the interpretation of the number, but maybe that's the number some people are interested in. It's the number I'm more interested in, and I do understand the underlying math reasonably well. Screent rating (not measurement) is only valid if you view 100% crops all the time.

Quote
From the outcry of the D800 fanatic, one would think it actually had a really crappy sensor by the way they demand images be normalized in size to allow the only form of objective comparison a study of a sensor could possibly endure. Ironically, the D800, D600, and D7000 all still outperform Canon sensors when you only look at the objective sensor measurements, so the inane debate about Print DR and its subjectivity is really just that...inane.

There's nothing "subjective" about the print DR measurement. The relative difference between print DR scores does not depend on whether they normalize to 8mpx or 12mpx or 6mpx. The choice of what resolution to normalize to is arbitrary but it doesn't change the rankings between sensors.

If there is anything "subjective" in the dynamic range score, it is the choice to use 0db as the baseline instead of some other number. The choice of of baseline could in theory change the relative performance of two cameras (though in practice I think this would be unlikely)
« Last Edit: October 15, 2012, 09:00:16 PM by elflord »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 14537
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #99 on: October 15, 2012, 09:17:07 PM »
While I think that in the aggregate the anecdotal evaluations of Canon's 61pt AF system would generally concur, that it is currently the best available as it produces a higher rate of keepers, and really good keepers at that...

I agree that it's anecdotal.  I'll add my anecdote.  I've got very little experience with Nikon bodies, so I make no attempt at comparison.  I can tell you that I love and hate the 1D X's AF system.  I love it, because it locks on fast, stays locked on, and delivers sharp shots consistently.  I hate it, because it locks on fast, stays locked on, and delivers sharp shots consistently, and that fact combined with the 12 fps frame rate means a harder time triaging my images.   ;) 

Seriously, though - in a burst from the 7D, I could always count on throwing away a few frames as OOF without even looking at them more magnified than reduced to fit my monitor (which is at least a 17" MBP at 1920x1200, and at best a 27" Apple Thunderbolt Display at 2560x1440).  With the 1D X, they are almost always all in focus.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

jrista

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4734
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #100 on: October 16, 2012, 01:21:33 AM »
I hate it, because it locks on fast, stays locked on, and delivers sharp shots consistently, and that fact combined with the 12 fps frame rate means a harder time triaging my images.   ;)

*sigh*...what I would give to have your problems. ;P

Seriously, though - in a burst from the 7D, I could always count on throwing away a few frames as OOF without even looking at them more magnified than reduced to fit my monitor (which is at least a 17" MBP at 1920x1200, and at best a 27" Apple Thunderbolt Display at 2560x1440).  With the 1D X, they are almost always all in focus.

Aye. I have the exact same problem with the 7D. Granted, the 7D is a hell of a lot better and more capable than any other AF system I've ever used, but when it misses, it really misses. That tends to be really bad for BIF, as that can mean a lot of misses in sequence. I'd love to get my hands on a 1D X though...the 61pt AF, RGB metering, and dedicated AF/metering processor...that would just be amazing.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

LetTheRightLensIn

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3813
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #101 on: October 16, 2012, 01:36:37 AM »
You keep repeating it, but it's still wrong. It's wrong because the low end of the dynamic range is determined by SNR, and SNR changes when you downsample.

This is the belief of a math geek who spends too much time with graphs and not enough time with real photographs. Down sampling throws away noise and detail. Dynamic range is the range of useful photographic detail. It is not the simple difference between two numbers on a graph.

Try this: shoot 4x5 Velvia. Drum scan it. Down sample to 10 MP. Did you all of a sudden gain 3-4 stops of shadow detail? No? Hmmm...

Side note: arguments like this are why I hate DR by software analysis. It has zero bearing on the real world. It just leads to paragraphs and paragraphs of irrelevant and pointless theorizing. Shoot a Stouffer transmission step wedge and look at it with your own two eyes. That tells you what you can expect in the real world.

Maybe you should learn to be more of a geek before you start calling names while at the same time getting everything you are talking about wrong....

LetTheRightLensIn

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3813
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #102 on: October 16, 2012, 01:44:07 AM »
The only way to discuss sensors objectively is to reference actual measurements. From the outcry of the D800 fanatic, one would think it actually had a really crappy sensor by the way they demand images be normalized in size to allow the only form of objective comparison a study of a sensor could possibly endure. Ironically, the D800, D600, and D7000 all still outperform Canon sensors when you only look at the objective sensor measurements, so the inane debate about Print DR and its subjectivity is really just that...inane.

So you would compare noise energy at two different frequencies as if they were the same frequency???

You can go on and on page after post about this but you are just not getting this right at all.

How about this, since you hate downsampling so much and are not getting it, try thinking about this, how about you keep the D800 at 36MP but filter out all noise at higher frequency than 22MP and then compare then? Now that would make it fair right (strictly speaking if you went to complex NR you could make the higher MP actually do better, but then it gets hard to judge since judgement on what sorts of adaptive NR is good, etc.)? Think about it that way then.

canon rumors FORUM

Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #102 on: October 16, 2012, 01:44:07 AM »

dtaylor

  • 5D Mark III
  • ******
  • Posts: 737
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #103 on: October 16, 2012, 09:27:10 AM »
You must have never used the sheer awesomeness of the canon 61-Point AF system And yes, I've used nikons excellent 51-point system as well. So yes, I can say the Canon AF system is better than nikons right now. Plus, I never said that point count makes a better AF system, because then S0&y would be the best then.  ::)

My apologies for assuming that your judgement was based merely on AF point count. I read that into your statement and it was not warranted.

dtaylor

  • 5D Mark III
  • ******
  • Posts: 737
    • View Profile
Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #104 on: October 16, 2012, 09:48:26 AM »
Quote
Try this: shoot 4x5 Velvia. Drum scan it. Down sample to 10 MP. Did you all of a sudden gain 3-4 stops of shadow detail? No? Hmmm...

Here's a different test for you -- sample it at 10MP, create another image at 40MP. Then compute the "blackpoint" (SNR = 0db) of the two images.

Now I take it that we agree that the images both have the same "true"  dynamic range, but I put it to you that the measured blackpoint on the 40MP will be at a higher luminance level, and therefore the measured per-pixel dynamic range will be lower on the 40MP image.

The second test is irrelevant because measuring the black point is not the same as measuring the floor of useful photographic detail. Assuming that it is leads to flawed extrapolations, such as DxO's 'normalized' DR graphs, or 9-stop scanned LF Velvia.

canon rumors FORUM

Re: Canon EOS-1D X DXOMark Sensor Scores
« Reply #104 on: October 16, 2012, 09:48:26 AM »