Mother of God - D800 scores 95 DxOMark

[Astro]

Imho the groundbreaking fact about the D800 is that they have 36mp and *still* get better noise and dr results which contradicts everything ever said about the disadvantages of high mp sensors. If they'd put in a 22mp sensor too, the big leap forward would be much more obvious because dr/noise would make the Canon sensors look like a past generation.

well no.. not if you believe those who say pixelsize has no influence on signal/noise ratio or DR.
which the D800 sensor seems to prove... in some way.

they had to make an actual D800 sensor with bigger pixel to see what is correct.

 

[Canon-F1]

which contradicts everything ever said about the disadvantages of high mp sensors. If they'd put in a 22mp sensor too, the big leap forward would be much more obvious because dr/noise would make the Canon sensors look like a past generation.

logic dictates it is either one or the other... but not both.

if pixelsize has no influence then a 22mp sensor will not be better.

 

[Canon-F1]

sony/nikon seem to backup those who say SNR is independent from pixelsize.

canon on the other side says 22mp are the optimum for SNR and MP with current technology.

 

[V8Beast]

I congratulate Nikon on producing a stellar camera in the D800. Now can someone please post sample images taken with both a D800 and a 5DIII, where the D800 made them a better photographer? I have a feeling that I'll be waiting for a long time.

Even when you do pixel peep, people reach different conclusions based on their biases. If this is someone's idea of enjoying photography, I find it quite pathetic.

To my eyes, however, the real-world benefits of its awesomeness are tough to spot in most of the images taken with it so far.

And yet you'd already decided that the D800 was the way to go, purely on the basis of spec sheets.

Thanks for digging that up for me, although you could have figured out that I ordered up a D800 (and a 5DIII for that matter) right here on Canon Rumors. I'm sorry you find how I choose to spend my own money so offensive.

It is interesting to read back to see how much my opinion has changed in the last couple of months. When I posted that, the rumor mill was swirling with a 5DIII with only a slightly improved AF and a 3-4 FPS burst rate. It didn't look like Canon would address my two biggest gripes with the 5DII, but they delivered big time. I never dreamed that Canon would put its flagship 61-point AF system and a 6 FPS burst rate in the 5DIII, but they did.

For me, the D800 is a camera whose specs are very impressive, but my enthusiasm has fizzled now that sample images from the wild are coming in. The 5DIII, on the other hand, is a camera whose rumored specs seemed very underwhelming at a time, but that changed once Canon seriously upped the AF and FPS ante, and it has impressed me more by the day as real world sample images start rolling in.

Hopefully that's OK with Mr. KeithR, but if not, I don't care. How someone chooses to spend their own money is none of my business.

 

[japhoto]

I'm still standing by a few points that I've said earlier in other threads:

1.) The D800 seems to be a great "all-rounder" camera, but I must admit that 5DIII with the improved AF-system comes much closer to being one too.

2.) I still think that D800 owners will have to adapt a medium format "workflow" to get the best results from it. Hand-holding is going to be tricky one here, at least to get sharp images at 1:1.

No argument. As I have gone to smaller pixels, I've had to be more careful about my technique to get good images. I won't be surprised to see a lot complaints about it from users who haven't dealt with it before.

Hand holding won't a problem. It's 36mp but it still has bigger pixels than the 7D.

I know it has (slightly) bigger pixels than the 7D, but since I'm shooting with the 7D now I can say that it requires those same techniques to get the most out of it. And yes, hand-holding to get sharp images at 1:1 is a PITA when the light isn't exceptionally good.

And adding a third point to my original list:

3.) Since the pixel pitch is similar to the 7D, it also means that the DLA is at around f/8 which is something people don't seem to think about with this camera.

 

[Canon-F1]

3.) Since the pixel pitch is similar to the 7D, it also means that the DLA is at around f/8 which is something people don't seem to think about with this camera.

what means you will have all the benefits below f8 and no real negative effects above f8.
it´s not as if the 36mp sensor will fall under the resolution of, lets say, a 21mp sensor at a given aperture.

it´s just that the sweetspot is at a more open aperture.

 

[dtaylor]

That's true of the aggregate use case scores. But we are discussing the graphs which do indicate unambiguously that the D800 sensor has better dynamic range at low ISO.

I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

I was able to confirm DPReview's results. DxO wasn't even close to reality.

DxO does not know how to properly measure dynamic range. If you want accurate DR values, please view test results at DPReview.

We won't know exactly how good the 5DIII is until we see the scores for that, but it's pretty clear that the D800 has a better performing sensor than the 5D2

Nothing of the sort is clear for dynamic range. Resolution, yes. You can discern that just from the MP counts, and initial studio test images from DPR and IR confirm it. But we won't know how they compare in terms of noise or DR until a reputable testing site publishes their review.

(but hardly surprising -- previous Nikon models were already performing better. Even the Sony APS-C cameras were doing better for low ISO dynamic range)

What Sony APS-C camera would that be? It looks to me like they are no better than Canon's 18 MP bodies when DR optimization is on or when RAW is used under best possible settings. That would leave Sony's APS-C sensors a solid 1 stop behind the Canon 5D2. I doubt the 5D3 has less DR.

 

[Canon-F1]

I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

well it´s open to discussion if THAT is the right way to measure DR.

but there is enough pro and contra talk about it on the web already.


http://www.luminous-landscape.com/essays/dxomark_sensor_for_benchmarking_cameras.shtml

 

[dtaylor]

I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

well it´s open to discussion if THAT is the right way to measure DR.

That's a short discussion. If I carefully spot meter a scene to know its full brightness range, shoot it, then come home and see how much my camera captured, will the result match the step wedge test or the DxO test? The answer is the step wedge test.

Step wedge test results matched up nicely with real world results and the zone system back in the days of film as well.

 

[Canon-F1]

well you must understand the DXO scores and how to interpret them.. that is for sure.
unfortunately 90% who look at the scores have no clue at all at what they are looking. :

that is why i have posted the link.

but i guess the 90% are to lazy to read it.....

Dynamic Range corresponds to the ratio between the highest brightness a camera can capture [..] and the lowest brightness [..] when noise is [as strong as the actual signal].

Photographers run out of Dynamic Range once in a while: usually in terms of "burnt" or "clipped" highlights. What DxOMark measures is more subtle: if you make an exposure series, what quality level will the best image have? In photographer-speak, what shadow noise do you get if you do an ideal "expose to the right" exposure. A high Dynamic Range sensor is good, but chances are that you can't print or even view this without special software. The Landscape/Sport/Portrait terms can easily confuse people who take this literally. I am tempted to interpret the 3 metrics as Dynamic Range (as DxO does), Luminance Noise (instead of Low-Light), and Chroma Noise (instead of Color Sensitivity). Those are quantities you find more often in reviews

Here are the problems as I see them. First, DPReview does not consistently apply a noise threshold. So when a DR test hits the 2% luminance threshold before it hits the noise threshold, there is actually more DR available from the sensor and JPEG image and you can actually use it if you bring the in-camera JPEG into Photoshop. In these cases, the reader doesn't get a DR test from DPReview. The reader gets a nice accurate report on the camera's tone curve instead. If the test hits the noise threshold and that is used as the cuttoff point instead, you do have a somewhat reasonable standard of comparison. But that only seems to happen on the high ISO tests, and you have to look carefully at the graphs to determine which tests hit a noise threshold and which ones don't. The noise threshold is a "standard" that isn't always applied. Probably all of the DSLR based lower ISO tests are under-reporting the actual DR because they all hit a tone curve dictated luminance threshold before they hit the noise threshold.

If you want a good indication of DR, you should develop the camera's raw files using no sharpening, no noise reduction, and preferably no gamma correction. Then you can analyze the image using a program that will measure the response and measure the noise. There is no need to concern yourself with luminance levels as a threshold. Those values are completely manipulatable. Testing using a basic linear develope raw like this isn't a perfect solution. [8-13-2010 - A better test is to use the raw data without demosaicing it. In such a test you would simply test only red, blue or green pixels and they would have no processing at all.] Everybody doesn't have the same idea of what is or isn't an acceptable level of noise. Furthermore, noise that measures the same doesn't always look the same. But doing DR tests this way will be far better and have far fewer variables than anything being done by online review sites right now [8-13-2010 - with the possible exception of dxomark testing]. What we have now is a jumbled mess of sorta-sometimes-maybe-right test results.

in the end... i guess you have no clue at all at what noise level you have measured?
and i don´t mean this offensive.

 

[Astro]

I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

the problem with wedge test is, that most people measure what they SEE not what can be PROCESSED against a given noise threshold.
and processing does not mean cheating here, the info is in the files.

in the case of dpreview you will sure get the same results if you make the same mistakes.

 

[Spriter]

The main problem with DXO metrics is that they interpret the results based on a 8 megapixels downsampling of the measurements they made. I find this inapropriate.

Most of us are cropping our pictures.

I'd certainly prefered having an indicator about pixel quality. I know how many megapixel I need and how many megapixel there is on the body I am considering.

 

[dtaylor]

that is why i have posted the link.

but i guess the 90% are to lazy to read it.....

I've read it, but it's little more than empty theorizing. Step wedge tests yield results that are consistent with the real world, whatever amount of noise you're willing to accept, as long as you have the same noise standard for the wedge test and the real world. I can't find any combination of variables which cause real world results to match DxO's calculations.

According to DxO there's not much difference in DR between the 10D, 20D, and 7D (less than 1 stop). Before I even tested these bodies I could have told you, from real world experience shooting and processing thousands of frames, that their measurements were ridiculous. The amount by which DxO is off varies with each body, so it's not just a simple matter of DxO having a different noise floor. Their results are simply wrong.

But anyone who has compared a Nikon 5100 (DxO score: 80) with a Hasselblad H3DII 50 (DxO score: 78) could tell you that DxO is ridiculous.

BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

 

[Spriter]

BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

Well done, I agree. Reducing to 8MP does not really help appreciate the value of the sensor. DXO is wrong.

 

[Astro]

BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

i guess most of the guys are engineers like myself who see the logic behind it.

 

[Astro]

The main problem with DXO metrics is that they interpret the results based on a 8 megapixels downsampling of the measurements they made. I find this inapropriate.

it´s a normalization.

As discussed above, the DxOMark Sensor score is "normalized" to compensate for differences in sensor resolution. To summarize: the DxOMark Sensor benchmark doesn't "punish" high-resolution sensors for having lots of small pixels that are each individually noisier. And similarly, the benchmark doesn't favor using large pixels despite their lower per-pixel noise. This is not some kind of ideology: it is just estimating the resulting noise level when viewing the entire image

for a fixed print size that makes sense. as downsampling is applied anyway.
once you exceed the acceptable noise for the printsize we can discuss if it is usefull.

 

[Gothmoth]

BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

but you measure the combo: sensor + software demosaicing + your individual develop settings.

how will you get a objectiv (comparable) result about the sensor that way?
apply a different gamma curve and you will have more DR, but what about the noise level if you do so? it will raise into regions that are not acceptable. luminance can be pushed around easily. you need some fixation and that is the acceptable noise level.

DXO measures the RAW info and noise levels.

 

[elflord]

DxO does not know how to properly measure dynamic range. If you want accurate DR values, please view test results at DPReview.

What specifically is wrong with the way DxOMark test dynamic range ?

The DxOMark tests basically show Canon's sensors flattening out in dynamic range by ISO 400 (that is, dynamic range doesn't improve for low ISO) whereas the Sony sensors (also used in Nikon sensors) continue to improve at ISO 200-100. At higher ISOs it goes more or less as you'd expect -- the bigger sensor wins by about a stop.

Are you saying that this is due to some flaw in DxOMarks method ? If so, what is the flaw ?

What Sony APS-C camera would that be? It looks to me like they are no better than Canon's 18 MP bodies when DR optimization is on or when RAW is used under best possible settings. That would leave Sony's APS-C sensors a solid 1 stop behind the Canon 5D2. I doubt the 5D3 has less DR.

DxOMark has the NEX 5N and Sony Alpha 77 beating the 5D Mark II for dynamic range at low ISOs.

 

[wickidwombat]

i would have read all of this thread but i've been out all weekend shooting with my 5Dmk3
since i don't really follow the lab tests or particularly care about them for that matter, 95 is good is it?

 

[briansquibb]

i would have read all of this thread but i've been out all weekend shooting with my 5Dmk3
since i don't really follow the lab tests or particularly care about them for that matter, 95 is good is it?

Most good bodies score in the 80's

Personally I feel the score is so much higher that this cynical old man wonders about the accuracy