DxOMark vs. Reality

[Canihaspicture]

Then we go into the "shadows" and there  I select an area and let photoshop with auto contrast calculate pixel values​​, and if there are any banding, noise.This is the difference in the lower level = DR
blah blah

So you're saying if as a photographer I constantly screw up exposure I should get a Nikon? With the 5dMk III if I come anywhere near correct exposure within a few stops (which is a huge range) then I get a great photo with little noise if any. With the Nikon even if I absolutely nail exposure I'll get noise at higher ISOs without changing shadows AT ALL...

Correctly expose your shots in camera and get back to us about noise.

[MarkII]

Are you simple, dishonest or both? The >14-stop-DR fallacy has been discussed multiple times, and has been quite convincingly proven to be just that, a fallacy. You get better SNR from downsampling, but DR cannot magically expand outside the source data boundaries. Get a clue already, please.

Insulting the original poster while being wrong yourself is not particularly smart.

Of course you can achieve more than the per-pixel DR by downsampling. Suppose two cameras have identical idealised pixel-level DR, but one has double the resolution. With even lighting, the light-per-area is the same on each sensor, but one sensor has double the number of pixels and hence double the headroom. Of course, the higher resolution sensor has more shadow noise, but down-sampling reduces the noise and hence the result is greater DR - more than can be achieved by a single pixel.

In the real world, the light is probably not equally distributed between adjacent pixels, meaning that one may clip before the other (you want something useful from the resolution increase, after-all), and things like system noise also complicate things. However, none of this prevents a down-sampled image having more per-pixel DR than the original sensor pixels, particularly when the downsampling factor is large (4.5x for the D800).

The DXOMark tests are lab-tests, and probably use uniform images for DR (which will tend to maximise the DR benefit from downsampling a D800) and use a lower-noise floor limit that may or may properly account from the pattern noise on many Canon sensors. The DXOMark tests appear to be well performed and accurate - though you need to understand what they are measuring.

If you still do not accept that it is possible to increase DR by down-sampling, I would take a look at the many applications in which oversampling is used to improve performance. A good example is the 1-bit DAC in some CD players. The DAC is produces a single-bit signal that is hugely oversampled (in the time domain). This is then down-sampled by a filter, yielding a result equivalent to a 16 bit linear DAC (or more).

So not only is it possible to increase DR beyond the native capture range, it is a widely used technique in signal processing.

Of course, the real problem here is that people only want the numbers to show that their purchase is somehow the best and opinion is as valid as understanding. I have seen similar discussions at school in the 1980's - my home computer is better than yours. It is all a bit depressing really.

[thepancakeman]

All these posts that say of course you can have greater range on outputs than the sensor is capable of:

On a scale of 1-10, I give you an 11!

<please note the obvious logic flaw and sarcasm>

"But this one goes to 11!"

Or maybe this is a case of "lies, damn lies, and statistics"?  I dunno, it just seems be to be missing the forest for the trees.

[ankorwatt]

now this was more a answer to the Neuro who finds it difficult to understand that DXO measurements also can be seen / reflected in an image.

Sorry, but what the hell gives you the idea that I don't understand that what DxO measures can be made to show up in images.  Please try to comprehend the following:

• There are differences between Canon and Nikon sensors
• Some aspects of Nikon sensor performance are superior to Canon sensor performance
• Some aspects of Canon sensor performance are superior to Nikon sensor performance
• DxOMark attempts to quantify those differences and defines their own arbitraty priorities to reduce those measurements to 'Scores'
• DxOMark Scores are relevant only in specific situations, and those situations may or may not have any relevance for the user of a camera
• DxOMark Scores attempt to represent a complex system with a single numeric comparison, an effort which is basically doomed to failure from the start, and is ultimately meaningless

So, if you routinely find yourself in situations where you must push shadow exposures 4 stops with less noise impact, you should probably be shooting Nikon (or asking how you manage to get yourself into those situations in the first place..).  If you want to handhold your 600mm lens, you should probably be shooting Canon (or use a 300mm lens with an Oly OM-D, because DxOMark says it's better than a 7D anyway, right?  ) .  Personally, I don't care what gear you use.

give one example there Canons sensor is better, and  do you   understand the benefit and  the choiche of underexposure and moving middle grey down 1-4 levels and bring in highligts far above 3.5 stops .

[ankorwatt]

Then we go into the "shadows" and there  I select an area and let photoshop with auto contrast calculate pixel values​​, and if there are any banding, noise.This is the difference in the lower level = DR
blah blah

So you're saying if as a photographer I constantly screw up exposure I should get a Nikon? With the 5dMk III if I come anywhere near correct exposure within a few stops (which is a huge range) then I get a great photo with little noise if any. With the Nikon even if I absolutely nail exposure I'll get noise at higher ISOs without changing shadows AT ALL...

Correctly expose your shots in camera and get back to us about noise.

IF you have any understanding how a digital camera works-there is no correctly exposure.
Do you mean that I should give Canon more light? longer exposure time? clipping  the high lights?
Then we are not comparing the cameras under the same parameters.

[PackLight]

now this was more a answer to the Neuro who finds it difficult to understand that DXO measurements also can be seen / reflected in an image.

Sorry, but what the hell gives you the idea that I don't understand that what DxO measures can be made to show up in images.  Please try to comprehend the following:

• There are differences between Canon and Nikon sensors
• Some aspects of Nikon sensor performance are superior to Canon sensor performance
• Some aspects of Canon sensor performance are superior to Nikon sensor performance
• DxOMark attempts to quantify those differences and defines their own arbitraty priorities to reduce those measurements to 'Scores'
• DxOMark Scores are relevant only in specific situations, and those situations may or may not have any relevance for the user of a camera
• DxOMark Scores attempt to represent a complex system with a single numeric comparison, an effort which is basically doomed to failure from the start, and is ultimately meaningless

So, if you routinely find yourself in situations where you must push shadow exposures 4 stops with less noise impact, you should probably be shooting Nikon (or asking how you manage to get yourself into those situations in the first place..).  If you want to handhold your 600mm lens, you should probably be shooting Canon (or use a 300mm lens with an Oly OM-D, because DxOMark says it's better than a 7D anyway, right?  ) .  Personally, I don't care what gear you use.

give one example there Canons sensor is better

Here you go;
 
The Canon sensor can fit in the D body with its excelent auto focus and keeper rate,
where as the D800 fits inside a body that has a marginal AF keeper rate.

The Canon sensor can use both the excelent line up of Canon lenses, and with adapter the Nikons as well,
Where the Nikons can not use the Canon lenses.

But, I guess you were talking naked sensors?

[ankorwatt]

now this was more a answer to the Neuro who finds it difficult to understand that DXO measurements also can be seen / reflected in an image.

Sorry, but what the hell gives you the idea that I don't understand that what DxO measures can be made to show up in images.  Please try to comprehend the following:

• There are differences between Canon and Nikon sensors
• Some aspects of Nikon sensor performance are superior to Canon sensor performance
• Some aspects of Canon sensor performance are superior to Nikon sensor performance
• DxOMark attempts to quantify those differences and defines their own arbitraty priorities to reduce those measurements to 'Scores'
• DxOMark Scores are relevant only in specific situations, and those situations may or may not have any relevance for the user of a camera
• DxOMark Scores attempt to represent a complex system with a single numeric comparison, an effort which is basically doomed to failure from the start, and is ultimately meaningless

So, if you routinely find yourself in situations where you must push shadow exposures 4 stops with less noise impact, you should probably be shooting Nikon (or asking how you manage to get yourself into those situations in the first place..).  If you want to handhold your 600mm lens, you should probably be shooting Canon (or use a 300mm lens with an Oly OM-D, because DxOMark says it's better than a 7D anyway, right?  ) .  Personally, I don't care what gear you use.

give one example there Canons sensor is better

Here you go;
 
The Canon sensor can fit in the D body with its excelent auto focus and keeper rate,
where as the D800 fits inside a body that has a marginal AF keeper rate.

The Canon sensor can use both the excelent line up of Canon lenses, and with adapter the Nikons as well,
Where the Nikons can not use the Canon lenses.

But, I guess you were talking naked sensors?

I am not talking lenses , buttons, meny,ergonomi or AF, I am talking about sensors
 and do you  understand the benefit and  the choiche of underexposure and moving middle grey down 1-4 levels and bring in highligts far above 3.5 stops in a motive who has a large DR  and there the highlight are far above 3,5 stops?? .

[ankorwatt]

Then we go into the "shadows" and there  I select an area and let photoshop with auto contrast calculate pixel values​​, and if there are any banding, noise.This is the difference in the lower level = DR
blah blah

So you're saying if as a photographer I constantly screw up exposure I should get a Nikon? With the 5dMk III if I come anywhere near correct exposure within a few stops (which is a huge range) then I get a great photo with little noise if any. With the Nikon even if I absolutely nail exposure I'll get noise at higher ISOs without changing shadows AT ALL...

Correctly expose your shots in camera and get back to us about noise.

whao , i did not se this  "So you're saying if as a photographer I constantly screw up exposure I should get a Nikon"

Yes  exactly but expose after high lights and use raw. you have 14 stops to play with (not 11 with banding, pattern noise)

[elflord]

What precisely are his criticisms ? What exactly are his credentials as far as engineering and benchmarking are concerned ?

what are DXO´s some would ask.   

Well, for a start, they run what is by any reasonable measure the leading sensor benchmark site.

They also implemented raw processing software.

How many industry leading benchmarks and raw converters do their critics on camera rumors sites publish ?

[elflord]

All these posts that say of course you can have greater range on outputs than the sensor is capable of:

On a scale of 1-10, I give you an 11!

<please note the obvious logic flaw and sarcasm>

"But this one goes to 11!"

Or maybe this is a case of "lies, damn lies, and statistics"?  I dunno, it just seems be to be missing the forest for the trees.

In fact quite the opposite, it is you who are missing the forest for the trees, or in this case, not comprehending that "the sensor" does not consist of a single pixel.

[ankorwatt]

What precisely are his criticisms ? What exactly are his credentials as far as engineering and benchmarking are concerned ?

what are DXO´s some would ask.   

Well, for a start, they run what is by any reasonable measure the leading sensor benchmark site.

They also implemented raw processing software.

How many industry leading benchmarks and raw converters do their critics on camera rumors sites publish ?

Many seems to have problems to understand exposure, the subject's DR and the cameras DR when two cameras are compared with the same parameters,   and  also the benefit of a large DR . Throwing sh... on DXO because of DXOs Canon sensor ratings only shows lack of knowledge.

[elflord]

Are you simple, dishonest or both? The >14-stop-DR fallacy has been discussed multiple times, and has been quite convincingly proven to be just that, a fallacy. You get better SNR from downsampling, but DR cannot magically expand outside the source data boundaries. Get a clue already, please.

I understand that there are people in this forum who repeat the same thing over and over -- that doesn't mean that they have "proven" anything, or even that they are correct.

There are at least two fairly obvious reasons why the DxO mark number is not flawed.

First, the choice of normalization baseline is arbitrary. When comparing (for example) an 21mpx and a 36mpx camera, you could normalize to 21mpx instead and get equivalent (but lower) . You won't hit the "quantization ceiling" if you do this.

Second, even if quantization error is greater than actual sensor error at the black point,the shadow noise level a few EV above the blackpoint will exceed the quantization error. So the bump from 14EV to 14.4EV will manifest as cleaner shadows and  an increase in usable dynamic range.

[elflord]

Oly OM-D, because DxOMark says it's better than a 7D anyway, right?  ) . 

OM-D sensor could well be better on some measurements (it's a Sony sensor), but if ergonomics with long lenses, and AF performance are a factor, it's a slamdunk for the 7D. Also, since you mentioned shooting with a 300mm lens -- on Canon, you could shoot with the 300mm f/4, the 300mm f/2.8, the 70-300L, the Sigma 120mm-300, or the Canon 100mm-400mm to name a few. I'm not sure what lens you'd use on the olympus.

Which gets back to other points made in this thread -- the system consists of much more than a sensor.

[Meh]

Oh boy, what a thread.

[PackLight]

What precisely are his criticisms ? What exactly are his credentials as far as engineering and benchmarking are concerned ?

what are DXO´s some would ask.   

Well, for a start, they run what is by any reasonable measure the leading sensor benchmark site.

They also implemented raw processing software.

How many industry leading benchmarks and raw converters do their critics on camera rumors sites publish ?

Many seems to have problems to understand exposure, the subject's DR and the cameras DR when two cameras are compared with the same parameters,   and  also the benefit of a large DR . Throwing sh... on DXO because of DXOs Canon sensor ratings only shows lack of knowledge.

Sure it is important to know if certain cameras have a higher DR, better color or whatever, but none of them are the point in what is wrong with DxO's testing. It is how they arrive at the final number that is flawed and how they present this number to the public.

It has been a while since I consulted DxO and the reviews they put out. The reason is that while they do some reasonable tests that can be informative they make assumptions that the components they feel are important can give a camera an overall rating.  The assume Color Depth is Portrait, DR is landscape and Sports is low ISO. All are important components, and testing individually is good.

If your new to the DSLR world and are trying to learn you flip to the page that shows the D800, it shows a rating of 95. 
You have to take the time to read the fine print to find out that the rating number, as you put it, is just sh.... that DxO is throwing out. There is no standard for combining these three test to arrive at a number, other than the standard DxO has come up with.