May 19, 2013, 03:43:31 AM

Author Topic: Mother of God - D800 scores 95 DxOMark  (Read 33648 times)

lonebear

  • PowerShot G15
  • **
  • Posts: 23
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #135 on: June 09, 2012, 03:00:10 PM »
Quote
The D800 has 14 bits A/D converters, so how could it have more than 14 bits of dynamic range?

A JPEG file is 8-bit, but that doesn't mean that the maximum scene dynamic range it can encode is only 8 stops, correct?

I'm not so sure.


Think about printing greyscale images with only pure black ink. So you basically have only two colors pure white and pure black, one stop DR only? And it is possible to print a fine greyscale image if the resolution is fine enough. The technique is some kind old.

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #135 on: June 09, 2012, 03:00:10 PM »

sarangiman

  • EOS M
  • ****
  • Posts: 207
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #136 on: June 09, 2012, 03:13:49 PM »
Quote
Think about printing greyscale images with only pure black ink. So you basically have only two colors pure white and pure black, one stop DR only?

Sorry, but that is not at all a valid analogy. You have many shades of grey due to the ability to overlay black dots to get darker areas (when viewed from a distance). B&W film, for example, isn't binary in nature... the size & density of silver deposites within a given area of film determine the dynamic range & tonality for that area... this quickly becomes a very complicated analysis b/c those numbers change based on the size of the area you're sampling. But now we're way OT.

The point is that you can still encode 14 stops of scene data with, say, an 8-bit ADC. Your tonality may suffer though; whether or not that is of any practical importance is still debated. Ideally, though, you want a high enough bit depth ADC such that the read noise of the sensor is not smaller than the quantization step (1 ADU). This ensures an accurate representation of the original data off the sensor. It could be argued that Nikon's D7000 & D800 actually undersample noise (noise in ADU is less than 1 for a dark frame, compared to 6ADU for Canon's 5D Mark III which oversamples noise) & so may benefit from a higher bit depth ADC... whether that would make any real world difference... I can't say at this point.

briansquibb

  • 1D X
  • *******
  • Posts: 3434
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #137 on: June 09, 2012, 03:40:39 PM »
Quote
Neuro has effectively destroyed the credibility of the DxO sensor marks.

Do you mind pointing us to where he did this? And, in the face of all the evidence of the ability to push D800 shadows stops & stops above 5D Mark III shadows, are you really disinclined to believe DxO's numbers that quantify exactly what is being observed & reported all over the internets?

I understand it's less of an issue for you Brian, with your enviable lack-of-banding shadows from your 1DsIII :) That really was a clean dark frame you sent me!


I am not debating the issue of the D800 merely pointing out that the DxO sensor scorings are dubious which were explained in a link from Dilbert and taken apart by Neuro. Whilst there is this confusion/debate going on I would suggest that quoting DxO sensor figures to prove a point is fairly meaningless as we cannot be certain of their validity

You are right in that my 1DS3 gives me far better IQ than my 5D2 and also the 1D4 for low ISO. As the 5D3 is not appreciably better than the 5D2 at low iso then I can only assume the 1DS3 is better than the 5D3. It may even be getting close to the D800 at low ISO. Using the histogram in DPP it is clear that I am usually have a DR of 8-10 - which is well within the capability of the 1DS3 (and probably all other modern DSLRs) so what is the benefit of buying a camera for even more DR potential when I am not using what I have at the moment.

I also shoot to the right which means that extracting shadow detail is, for me, fairly rare.

Here is an example which I took today

1Ds3, 135@f/2, 1/60 (yep, no IS needed), iso 100

DR = 7

D30, tse-24L II

sarangiman

  • EOS M
  • ****
  • Posts: 207
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #138 on: June 09, 2012, 03:49:00 PM »
Quote
Whilst there is this confusion/debate going on I would suggest that quoting DxO sensor figures to prove a point is fairly meaningless as we cannot be certain of their validity

I was curious enough to test DxO's numbers myself. My 13.2 stop wedge tests pretty much agreed with DxO's numbers for the 5D3 & D7000 within 0.5 stops if you use SNR of 1 as the lower boundary.

I then did side-by-side comparisons of a sunset which reflected my findings in the more controlled ('lab') environment above.

So as far as I'm concerned, DxO's dynamic range numbers are absolutely valid. But I understand that you have no reason to believe my tests until I actually put them up on a blog post & make the RAW files available, etc. I intend to do so I just keep getting swamped with work...

briansquibb

  • 1D X
  • *******
  • Posts: 3434
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #139 on: June 09, 2012, 04:02:14 PM »
Quote
Whilst there is this confusion/debate going on I would suggest that quoting DxO sensor figures to prove a point is fairly meaningless as we cannot be certain of their validity

I was curious enough to test DxO's numbers myself. My 13.2 stop wedge tests pretty much agreed with DxO's numbers for the 5D3 & D7000 within 0.5 stops if you use SNR of 1 as the lower boundary.

I then did side-by-side comparisons of a sunset which reflected my findings in the more controlled ('lab') environment above.

So as far as I'm concerned, DxO's dynamic range numbers are absolutely valid. But I understand that you have no reason to believe my tests until I actually put them up on a blog post & make the RAW files available, etc. I intend to do so I just keep getting swamped with work...

I think it is important to have definitive and undisputed measures for the baseline of a discussion.

However on the high DR front I wonder how many get close to DR 10 and therefore would benefit from, say, a DR of 14 body over a DR of 13 body? If it isn't many then there is little point focussing on what is for most a theoretical benefit.

D30, tse-24L II

sarangiman

  • EOS M
  • ****
  • Posts: 207
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #140 on: June 09, 2012, 04:27:50 PM »
Quote
However on the high DR front I wonder how many get close to DR 10 and therefore would benefit from, say, a DR of 14 body over a DR of 13 body? If it isn't many then there is little point focussing on what is for most a theoretical benefit.

Well, photographers are creative people, so I, for one, am looking forward to what capable photographers might do with the expanded DR of the D800. For example landscapes/cityscapes not amenable to grad ND filters or multiple exposures (which become costly when you're doing 30s exposures and the sun is setting/lighting changing constantly). Moonscapes. Or environmental portraits shooting into the sun... even aided with a flash, these could benefit from increased DR.

And let's not forget that sometimes during live event shooting you just get the exposure wrong. Or a flash doesn't fire. Or someone walks in the path of your flash. The ability to rescue such shots should not be overlooked.

lonebear

  • PowerShot G15
  • **
  • Posts: 23
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #141 on: June 09, 2012, 04:28:56 PM »
Quote
Think about printing greyscale images with only pure black ink. So you basically have only two colors pure white and pure black, one stop DR only?

Sorry, but that is not at all a valid analogy. You have many shades of grey due to the ability to overlay black dots to get darker areas (when viewed from a distance). B&W film, for example, isn't binary in nature... the size & density of silver deposites within a given area of film determine the dynamic range & tonality for that area... this quickly becomes a very complicated analysis b/c those numbers change based on the size of the area you're sampling. But now we're way OT.

The point is that you can still encode 14 stops of scene data with, say, an 8-bit ADC. Your tonality may suffer though; whether or not that is of any practical importance is still debated. Ideally, though, you want a high enough bit depth ADC such that the read noise of the sensor is not smaller than the quantization step (1 ADU). This ensures an accurate representation of the original data off the sensor. It could be argued that Nikon's D7000 & D800 actually undersample noise (noise in ADU is less than 1 for a dark frame, compared to 6ADU for Canon's 5D Mark III which oversamples noise) & so may benefit from a higher bit depth ADC... whether that would make any real world difference... I can't say at this point.

Yes, it is valid (and it's not analogy but industry standard way to handle it). Beside printing industry, this technique had been used in the earliest raster displays as well. The original doubt was whether we can generate higher DR given a limited single pixel DR, and the answer is yes. The example I given is an extreme with only two colors. If given 8 bit pixel formats I should be easy to double the DR using 2X2 dithering.

When the resolution goes high beyond certain threshold, human eyes will do the blending automatically. That is how the current printing technique works. Ink printers will blend some droplets together to some degree (and thus more preferable in photo printing), but tuner printers have to rely on the remaining white spots among the black array to create different shades. Putting the print-outs under microscope (kind of pixel peeping), the uneven particles should be easy to see. Film is different as it is not only with much higher resolution (to the size of cluster of molecules) but also with various silver deposit density. However, if you can place the finest film photo under electronic microscope (that's the king of pixel peeping in current technology ;-) ), the most smooth tonal changes will reveal their uneven dithering instantly. Basically, I think you know what I mean.

Back to the original DXO DR score, I don't think DXO will take the dithering into consideration. No normal photographers will EVER sacrifice IQ that much for some occasionally needed extra DR. However, given a fixed formula of DR calculation, it may allow some unintended dithering scenario passing through, and generating a higher score. Of course, this is only one explanation among others.
« Last Edit: June 09, 2012, 04:40:42 PM by lonebear »

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #141 on: June 09, 2012, 04:28:56 PM »

DZY

  • PowerShot G15
  • **
  • Posts: 12
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #142 on: June 09, 2012, 04:31:53 PM »
Quote
Quote
the result becomes a system overall performance.

... which is exactly what we should care about.
So the overall performance should include the entire camera, ie. noise, ISO, FPS, AF, lens, etc., not represented by DXO only.

Quote
Quote
It has been well known in the astrophotography that Nikon does NOT output real raw data, some degrees of NR is applied to their RAW.

This is a good point, something I didn't want to get into earlier b/c of the complexity of the issue. But since you brought it up... what one can do is actually fit the quantitated data from the RAW file (of the wedge shot) to the theoretical luminosity curve of the wedge. You can then find the point at which the fit deviates significantly from the model, & set that as your lower end. To standardize results, maybe set a 'maximum deviance' criterion. From this fit vs. model you can also detect 'shadow crushing'... that is, if data is significantly clipped on the lower end (as some suspect is the case for Nikon RAWs), the quantitated data will deviate from the model quicker (instead of continuing to be linear on a log scale, the quantitated data will level out). I've done these measurements for a D7000 vs. my 5D II & 5D III; both cameras show this 'toe' on the darker end, & even if you pick the lower acceptable end based on this toe, the D7000 is still ~2 stops better than any of the 5D series bodies.
This hokey stick type of curve looking like a check mark will tell you what the minimal DN vs Electron of the sensor is, it is again at least the overall of the QE+Opamp gain+ADC gain+processor algorithm. The processor can do a magic trick here to minimize the noise. I do agree, by inspecting Nikon vs Canon photo, that Nikon is better than Canon on the algorithm of this, they used another method. However, this does not mean the two sensor is of that much difference. DXO score does not indicate the sensor performance, as they claims, and this is my point. I think many of us are mislead by DXO score. Nikon, or Canon, or any one can design their camera targeting the DXO score if they want, but with bad actual picture, refer the following notes.

Quote
Quote
Let's put another grey color filter, there is no maximum white to be represented by the sensor any more

  • First of all, that's an extreme case.
  • Secondly, of course there's still maximum white... you adjust the exposure. Like I said, you take exposures right around the exposure that blows the brightest patch. An ND filter, e.g., simply does not trip up this methodology whatsoever.
What you do is to find the max and min of the electrons (photo) the sensor takes. You method can eliminate the effect of the optics, and I hope you did NOT use camera's shutter/aperture/ISO in this test. But, the problem is explain as my direct quote from another forum dvxuser.com:
"trez 04-25-2012, 04:06 AM
Speaking about DR, people often mix two things - the ratio of the brightness and the ratio of the coded values (digital numbers). These are not interchangeable terms.
Imagine 1-bit codec. The value can only be 1 or 0. Now, this doesn't mean much, unless we know which brightness levels that "1" corresponds to.
In general case, the camera has to squeeze its DR (or part of it) somehow into the available bits. There are various ways to achieve that and none is perfect - one of the reasons we have so many gamma curves. Assume 8-bit codec - even linear gamma curve can squeeze more than 8 f-stops into them - all it needs to do is to 'compress' the brightness range, so that, f.ex. digital 255 represents the brightness of 12 f-stops above what's represented by digital 0. It's just the mapping.
The problem is that the more we compress, the worse the tonal resolution becomes in the mids (and shadows), which is of great importance, f.ex. for skin tones - there's risk of severe banding, when there are not enough distinct (coded) levels to smoothly represent skin gradation. That's why non-linear gamma curves are used - they assign more coding space (distinct values representing various brightness levels) to where it makes more sense, taking into account things like logarithmic tonal perception of human vision, anticipated viewing conditions, noise, gradability, being able to use standard displays for monitoring etc. F.ex, LOG-based gammas sacrifices easy monitoring in order to improve on other areas.
So, while 8-bit codec doesn't necessarily limit the DR it can represent, there's no point in squeezing too much DR into it, as severe banding will occur in post, if we're trying to stretch it to get back to the original scene DR. Of course, some compression is fine - we don't need too much tonal resolution in highlights so we can squeeze them - this is similar to what film does, known as "shoulder".

Quote
Quote
I just don't see how you can say the DXO score IS the sensor only properties.

I never claimed anything of the sort.
Good, we are on the same boat then.

Quote
Quote
to downsize the picture? What a joke to these downsize from nikon, first, why not you downsize both 5D3 and D800 to 800x600, I bet they will be the same. To be extreme, we can downsize any photo to 1x1 size and all picture will be same for same the scene and exposure. Second, are you buying a 36MP camera and use as a 22MP each every time?

Since this has been covered extensively in other threads, I won't belabor the point... but, in a nutshell, the point is simply that the 36MP camera offers you the advantage of more resolution if you want it, while giving you just as clean images at the resolution of the 5D Mark III.

I will say though that it would've been nice for Nikon to include mRAW functionality for lower resolution RAW images right off the camera to ease the workflow of photographers. But that has its own issues, since mRAWs require demosaicing prior to downsizing (and so you don't benefit from the freedom of choice of demosaicing algorithm, or the evolution of such algorithms in the future). So perhaps it behooves RAW converters (ACR, Aperture, etc.) to offer this option in their software as we get higher & higher megapixel cameras... but that raises other issues -- e.g. do you save the original RAW file or not?
I don't clip/reframe my photo too much, I don't need THAT much MP unless I am a spy. You finish your composition before the shutter, do you? if not, you are really not a photographer. BTW, any MP greater than 10MP is fine for me already. You may say my lens is not a tele so I have to shoot than enlarge. Well, if you can no see, you can not shoot, simple as this. Again, I am not a spy.
BUT, I do need higher ISO and FPS, such as D7000, 7D, 5D3, etc.

dilbert

  • 1D X
  • *******
  • Posts: 1772
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #143 on: June 09, 2012, 04:45:14 PM »
However on the high DR front I wonder how many get close to DR 10 and therefore would benefit from, say, a DR of 14 body over a DR of 13 body? If it isn't many then there is little point focussing on what is for most a theoretical benefit.

I for one definitely would benefit from greater DR.

Along with Lightroom 4, I think that there's a greater chance that a single photo with a DR of 14 or more would let me do some photos in one shot rather than multiple and then HDR it.

DZY

  • PowerShot G15
  • **
  • Posts: 12
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #144 on: June 09, 2012, 04:52:55 PM »
Quote
Whilst there is this confusion/debate going on I would suggest that quoting DxO sensor figures to prove a point is fairly meaningless as we cannot be certain of their validity

I was curious enough to test DxO's numbers myself. My 13.2 stop wedge tests pretty much agreed with DxO's numbers for the 5D3 & D7000 within 0.5 stops if you use SNR of 1 as the lower boundary.

I then did side-by-side comparisons of a sunset which reflected my findings in the more controlled ('lab') environment above.

So as far as I'm concerned, DxO's dynamic range numbers are absolutely valid. But I understand that you have no reason to believe my tests until I actually put them up on a blog post & make the RAW files available, etc. I intend to do so I just keep getting swamped with work...
Thank you for you good work to verify DXO numbers, I have no doubt about the numbers. That is science and mathematics, and they don't lie. What is wrong is that we, human beings, make wrong assumption and wrong judgment. As I said, DXO score does not indicates the D800 is the best . So do not worry and be serious, especially don't be mislead by DXO.
You can put a 4-cylinder or a 16-cylinder engine into a Lexus and tune up to as same as the 8-cylinder. Then it is the personal taste issue.
My personal taste: I think the Nikon DR curve at shadow is more usable than Canon in the real word, compare to Canon DR curve at high light, which is their tradition.
As you said on another post, more DR or better tonality, is still in debate.
« Last Edit: June 09, 2012, 04:56:56 PM by DZY »

sarangiman

  • EOS M
  • ****
  • Posts: 207
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #145 on: June 09, 2012, 04:55:52 PM »
Quote
I hope you did NOT use camera's shutter/aperture/ISO in this test

I believe I mentioned in my initial posting re: this that I do NOT use the same shutter/aperture/ISO in my test. Instead, I take a variety of exposures & select the one that is just short of clipping. Which eliminates effects of optics (within reason, of course).

Quote
So the overall performance should include the entire camera, ie. noise, ISO, FPS, AF, lens, etc., not represented by DXO only.

I agree. DXO is only addressing certain features of the camera. Important ones for some people. And they do it in a very objective manner. I was suspicious of their numbers as well until I did my own tests.

Quote
This hokey stick type of curve looking like a check mark will tell you what the minimal DN vs Electron of the sensor is, it is again at least the overall of the QE+Opamp gain+ADC gain+processor algorithm. The processor can do a magic trick here to minimize the noise. I do agree, by inspecting Nikon vs Canon photo, that Nikon is better than Canon on the algorithm of this, they used another method. However, this does not mean the two sensor is of that much difference.

Yes, it's a measure of sensor + downstream electronics + DSP. And in the end, that's what matters to the photographer, not sensor only performance. The 'hockey stick type curve', even with DSP tricks to minimize noise, allows you to select a lower end that still manages to distinguish between the darkest patches of the wedge. So you, the evaluator, can select what you deem is acceptable in the face of shadow crushing due to tricky signal processing. And yet, still, the Nikon D7000 will distinguish many more dark patches than the 5D sensor will (while SNR>1)... and this translates to my real world tests as well. So I respectfully disagree that it's just a magic algorithm in the Nikon that allows for this. Their sensor+electronics are simply cleaner and, therefore, do not much with your lower signals much.

Another way to evaluate this is to do the test of ISO-less capability. Take a proper exposure at ISO 6400, then, without changing shutter speed or aperture, shoot at ISO 3200, 1600, ... 100. Boost exposures properly in post to match ISO 6400 image (ISO 3200 + 1stop | ISO 1600 + 2stops | ISO 800 + 3stops | ISO 400 + 4stops, etc.). The 5D series shots fall apart by ISO 1600 + 2 stops compared to the ISO 6400 image. The D7000 ISO 200 + 5 stops looks virtually identical to the original ISO 6400 shot. That speaks volumes as to how clean the sensor & downstream electronics are.

And, as I said, this is easily reflected in real world shots of high dynamic range scenes.

Quote
Again, I am not a spy. BUT, I do need higher ISO and FPS, such as D7000, 7D, 5D3, etc.

That is totally valid. In fact, the awesome AF & higher FPS of the 5D Mark III is what's making me hold on to it. Once I get my D800 (if ever, ha!), I'll compare the AF & if it's just as good as the 5D III (I doubt it, b/c it lacks side cross-type AF points... but only experience over a number of shoots will tell me how much this matters to me), I may take the hit of lower FPS for better image quality.

My only point was that it was unfair to call DXO biased. I see no evidence to support this hypothesis. My only gripe about them is that they don't fully publish their protocols/methodology.
« Last Edit: June 09, 2012, 04:58:55 PM by sarangiman »

DZY

  • PowerShot G15
  • **
  • Posts: 12
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #146 on: June 09, 2012, 05:19:50 PM »
Quote
Again, I am not a spy. BUT, I do need higher ISO and FPS, such as D7000, 7D, 5D3, etc.

That is totally valid. In fact, the awesome AF & higher FPS of the 5D Mark III is what's making me hold on to it. Once I get my D800 (if ever, ha!), I'll compare the AF & if it's just as good as the 5D III (I doubt it, b/c it lacks side cross-type AF points... but only experience over a number of shoots will tell me how much this matters to me), I may take the hit of lower FPS for better image quality.

My only point was that it was unfair to call DXO biased. I see no evidence to support this hypothesis. My only gripe about them is that they don't fully publish their protocols/methodology.
Yes! I also want to know how DXO does their test. All scientific studies have to publish how and what, not just the result. Giving a number is just commercial ad and marketing.
When you compare the AF, please notice the AF lag time also.
BTW,
1. the shutter/Aperture/ISO meant not the setting on the body b/c they might be not reliable.
2. by bias I am saying the assumption/pre-condition DXO sets up might be biased. In other words, Nikon un-intentionally take advantage of it.
3. I agree with you on the D7000 low noise performance, although I don't know if it comes from the circuit or the DSP. I doubt Canon can not do this. They just don't do as Nikon. Canon's approach seems different and hurts the result at low light which is not appreciated by me.
« Last Edit: June 09, 2012, 05:22:37 PM by DZY »

neuroanatomist

  • 1D X
  • *******
  • Posts: 8591
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #147 on: June 09, 2012, 06:27:33 PM »
So as far as I'm concerned, DxO's dynamic range numbers are absolutely valid.

I agree.  Allow me to clarify.  I see no issues with DxOMark's measurements - they are useful and accurate, and they disclose their methods to generate those measurements. (FWIW, I personally prefer and use DxO Optics Pro as a RAW converter.)

The problem is the DxOMark Sensor Score.  Here's what Luminous Landscape states:

Documentation about the way the final DxOMark Sensor score is computed from Dynamic Range, Color Sensitivity and Low-light ISO scores is not currently available.

That same article (linked in a thread started by dilbert) indicates there is some 'non-linearity' in the way the Score is calculated.  So...a weighted average, but undisclosed weighting factors, no confirmation that those unknown factors are even consistently applied, non-linear calculations, etc., to me, that makes the DxOMark Sensor Score a meaningless and useless number (not measurement, just an irrelevant number).
EOS 1D X, EOS 7D, and lots of lenses
_____________________________
Flickr | TDP Profile/Gear List

Marsu42

  • 1D X
  • *******
  • Posts: 2796
  • 60d / 100L / 70-300L / 17-40L / 600rt / 430ex2
    • View Profile
    • 6D positive spec list
Re: Mother of God - D800 scores 95 DxOMark
« Reply #148 on: June 09, 2012, 06:32:33 PM »
to me, that makes the DxOMark Sensor Score a meaningless and useless number

While not transparent, the score isn't meaningless to me if it usually reflects the iq differences that a multitude of reviews express in words sooner or later. You can see it this way, too: Google's search engine algorithm is not transparent, too, but does that make Google useless?

briansquibb

  • 1D X
  • *******
  • Posts: 3434
    • View Profile
Re: Mother of God - D800 scores 95 DxOMark
« Reply #149 on: June 09, 2012, 06:36:23 PM »
to me, that makes the DxOMark Sensor Score a meaningless and useless number

While not transparent, the score isn't meaningless to me if it usually reflects the iq differences that a multitude of reviews express in words sooner or later. You can see it this way, too: Google's search engine algorithm is not transparent, too, but does that make Google useless?

IQ deduced from a sensor score that reflects numerous measurements not relating to IQ?

Google's algorithm does not give a score that influences spending habits nor does it pretend to be 100% accurate.
D30, tse-24L II

canon rumors FORUM

Re: Mother of God - D800 scores 95 DxOMark
« Reply #149 on: June 09, 2012, 06:36:23 PM »