August 30, 2014, 02:14:35 AM

Author Topic: Mother of God - D800 scores 95 DxOMark  (Read 45693 times)

Canon-F1

  • 5D Mark III
  • ******
  • Posts: 715
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #75 on: March 24, 2012, 03:25:27 PM »
I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

well it´s open to discussion if THAT is the right way to measure DR.

but there is enough pro and contra talk about it on the web already.


http://www.luminous-landscape.com/essays/dxomark_sensor_for_benchmarking_cameras.shtml
« Last Edit: March 24, 2012, 03:27:37 PM by Canon-F1 »
6D, 5D MK2, 7D, 550D... a lot of Glass.

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #75 on: March 24, 2012, 03:25:27 PM »

dtaylor

  • 6D
  • *****
  • Posts: 575
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #76 on: March 24, 2012, 03:52:13 PM »
I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

well it´s open to discussion if THAT is the right way to measure DR.

That's a short discussion. If I carefully spot meter a scene to know its full brightness range, shoot it, then come home and see how much my camera captured, will the result match the step wedge test or the DxO test? The answer is the step wedge test.

Step wedge test results matched up nicely with real world results and the zone system back in the days of film as well.

Canon-F1

  • 5D Mark III
  • ******
  • Posts: 715
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #77 on: March 24, 2012, 03:56:58 PM »
well you must understand the DXO scores and how to interpret them.. that is for sure.
unfortunately 90% who look at the scores have no clue at all at what they are looking.  ::)

that is why i have posted the link.

but i guess the 90% are to lazy to read it.....


Quote
Dynamic Range corresponds to the ratio between the highest brightness a camera can capture [..] and the lowest brightness [..] when noise is [as strong as the actual signal].

Photographers run out of Dynamic Range once in a while: usually in terms of "burnt" or "clipped" highlights. What DxOMark measures is more subtle: if you make an exposure series, what quality level will the best image have? In photographer-speak, what shadow noise do you get if you do an ideal "expose to the right" exposure. A high Dynamic Range sensor is good, but chances are that you can't print or even view this without special software. The Landscape/Sport/Portrait terms can easily confuse people who take this literally. I am tempted to interpret the 3 metrics as Dynamic Range (as DxO does), Luminance Noise (instead of Low-Light), and Chroma Noise (instead of Color Sensitivity). Those are quantities you find more often in reviews


Quote
Here are the problems as I see them. First, DPReview does not consistently apply a noise threshold. So when a DR test hits the 2% luminance threshold before it hits the noise threshold, there is actually more DR available from the sensor and JPEG image and you can actually use it if you bring the in-camera JPEG into Photoshop. In these cases, the reader doesn't get a DR test from DPReview. The reader gets a nice accurate report on the camera's tone curve instead. If the test hits the noise threshold and that is used as the cuttoff point instead, you do have a somewhat reasonable standard of comparison. But that only seems to happen on the high ISO tests, and you have to look carefully at the graphs to determine which tests hit a noise threshold and which ones don't. The noise threshold is a "standard" that isn't always applied. Probably all of the DSLR based lower ISO tests are under-reporting the actual DR because they all hit a tone curve dictated luminance threshold before they hit the noise threshold.

If you want a good indication of DR, you should develop the camera's raw files using no sharpening, no noise reduction, and preferably no gamma correction. Then you can analyze the image using a program that will measure the response and measure the noise. There is no need to concern yourself with luminance levels as a threshold. Those values are completely manipulatable. Testing using a basic linear develope raw like this isn't a perfect solution. [8-13-2010 - A better test is to use the raw data without demosaicing it. In such a test you would simply test only red, blue or green pixels and they would have no processing at all.] Everybody doesn't have the same idea of what is or isn't an acceptable level of noise. Furthermore, noise that measures the same doesn't always look the same. But doing DR tests this way will be far better and have far fewer variables than anything being done by online review sites right now [8-13-2010 - with the possible exception of dxomark testing]. What we have now is a jumbled mess of sorta-sometimes-maybe-right test results.

in the end... i guess you have no clue at all at what noise level you have measured?
and i don´t mean this offensive.
« Last Edit: March 24, 2012, 04:27:43 PM by Canon-F1 »
6D, 5D MK2, 7D, 550D... a lot of Glass.

Astro

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #78 on: March 24, 2012, 04:51:51 PM »
I have personally checked DxO's dynamic range results for 3 bodies (10D, 20D, 7D) against a Stouffer transmission step wedge in essentially the same test that is performed at DPReview.com.

the problem with wedge test is, that most people measure what they SEE not what can be PROCESSED against a given noise threshold.
and processing does not mean cheating here, the info is in the files.

in the case of dpreview you will sure get the same results if you make the same mistakes.


Spriter

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #79 on: March 24, 2012, 05:17:39 PM »
The main problem with DXO metrics is that they interpret the results based on a 8 megapixels downsampling of the measurements they made.  I find this inapropriate.

Most of us are cropping our pictures.

I'd certainly prefered having an indicator about pixel quality.  I know how many megapixel I need and how many megapixel there is on the body I am considering.

dtaylor

  • 6D
  • *****
  • Posts: 575
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #80 on: March 24, 2012, 05:19:00 PM »
that is why i have posted the link.

but i guess the 90% are to lazy to read it.....

I've read it, but it's little more than empty theorizing. Step wedge tests yield results that are consistent with the real world, whatever amount of noise you're willing to accept, as long as you have the same noise standard for the wedge test and the real world. I can't find any combination of variables which cause real world results to match DxO's calculations.

According to DxO there's not much difference in DR between the 10D, 20D, and 7D (less than 1 stop). Before I even tested these bodies I could have told you, from real world experience shooting and processing thousands of frames, that their measurements were ridiculous. The amount by which DxO is off varies with each body, so it's not just a simple matter of DxO having a different noise floor. Their results are simply wrong.

But anyone who has compared a Nikon 5100 (DxO score: 80) with a Hasselblad H3DII 50 (DxO score: 78) could tell you that DxO is ridiculous.

BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

Spriter

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #81 on: March 24, 2012, 05:22:27 PM »


BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

Well done, I agree.  Reducing to 8MP does not really help appreciate the value of the sensor.  DXO is wrong.

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #81 on: March 24, 2012, 05:22:27 PM »

Astro

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #82 on: March 24, 2012, 05:22:53 PM »
BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

i guess most of the guys are engineers like myself who see the logic behind it. :)


Astro

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #83 on: March 24, 2012, 05:34:30 PM »
The main problem with DXO metrics is that they interpret the results based on a 8 megapixels downsampling of the measurements they made.  I find this inapropriate.
 

it´s a normalization.

Quote
As discussed above, the DxOMark Sensor score is "normalized" to compensate for differences in sensor resolution. To summarize: the DxOMark Sensor benchmark doesn't "punish" high-resolution sensors for having lots of small pixels that are each individually noisier. And similarly, the benchmark doesn't favor using large pixels despite their lower per-pixel noise. This is not some kind of ideology: it is just estimating the resulting noise level when viewing the entire image

for a fixed print size that makes sense. as downsampling is applied anyway.
once you exceed the acceptable noise for the printsize we can discuss if it is usefull.

« Last Edit: March 24, 2012, 05:42:26 PM by Astro »

Gothmoth

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #84 on: March 24, 2012, 05:50:00 PM »
BTW, every time someone points out how wrong DxO's test results are, someone else jumps in and claims "well, you have to understand the test. Read this link." No. Sorry. It doesn't work that way. DxO can't present a score which explicitly says A is better than B, then backtrack and say we don't understand the score if we think it means A is better than B. Same for individual tests with results stated using industry standard terms (i.e. DR in stops). If their scoring is confusing that's their fault, not the reader's. And if their test results fail to correspond to real world use, again that is their fault, not the reader's.

but you measure the combo: sensor + software demosaicing + your individual develop settings.

how will you get a objectiv (comparable) result about the sensor that way?
apply a different  gamma curve and you will have more DR, but what about the noise level if you do so? it will raise into regions that are not acceptable. luminance can be pushed around easily. you need some fixation and that is the acceptable noise level.

DXO measures the RAW info and noise levels.

« Last Edit: March 24, 2012, 06:03:37 PM by Gothmoth »

elflord

  • 5D Mark III
  • ******
  • Posts: 705
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #85 on: March 24, 2012, 09:36:39 PM »

DxO does not know how to properly measure dynamic range. If you want accurate DR values, please view test results at DPReview.


What specifically is wrong with the way DxOMark test dynamic range ?

The DxOMark tests basically show Canon's sensors flattening out in dynamic range by ISO 400 (that is, dynamic range doesn't improve for low ISO) whereas the Sony sensors (also used in Nikon sensors) continue to improve at ISO 200-100.  At higher ISOs it goes more or less as you'd expect -- the bigger sensor wins by about a stop.

Are you saying that this is due to some flaw in DxOMarks method ? If so, what is the flaw ?

Quote
What Sony APS-C camera would that be? It looks to me like they are no better than Canon's 18 MP bodies when DR optimization is on or when RAW is used under best possible settings. That would leave Sony's APS-C sensors a solid 1 stop behind the Canon 5D2. I doubt the 5D3 has less DR.

DxOMark has the NEX 5N and Sony Alpha 77 beating the 5D Mark II for dynamic range at low ISOs.

wickidwombat

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4484
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #86 on: March 25, 2012, 04:43:15 AM »
i would have read all of this thread but i've been out all weekend shooting with my 5Dmk3 ;)
since i don't really follow the lab tests or particularly care about them for that matter, 95 is good is it?
APS-H Fanboy

briansquibb

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #87 on: March 25, 2012, 04:47:14 AM »
i would have read all of this thread but i've been out all weekend shooting with my 5Dmk3 ;)
since i don't really follow the lab tests or particularly care about them for that matter, 95 is good is it?

Most good bodies score in the 80's

Personally I feel the score is so much higher that this cynical old man wonders about the accuracy

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #87 on: March 25, 2012, 04:47:14 AM »

ssrdd

  • Rebel SL1
  • ***
  • Posts: 90
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #88 on: March 25, 2012, 05:38:57 AM »
finally 5dmark 3 has proved to be S___ for the price point.
ha ha haaa...

briansquibb

  • Guest
Re: Mother of God - D800 scores 95 DxOMark
« Reply #89 on: March 25, 2012, 05:44:52 AM »
finally 5dmark 3 has proved to be S___ for the price point.
ha ha haaa...

I can tell you have done your research - you spotted the 'knock the 5DIII and praise the D800' bandwagon and jumped on.

DXOMark comparing the 5DIII and the D800

http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/%28appareil1%29/795|0/%28brand%29/Canon/%28appareil2%29/792|0/%28brand2%29/Nikon

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #89 on: March 25, 2012, 05:44:52 AM »