August 01, 2014, 01:15:27 PM

Author Topic: 5D3 same max dynamic range as the 5D2???  (Read 27785 times)

moreorless

  • 6D
  • *****
  • Posts: 580
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #30 on: March 09, 2012, 03:46:15 AM »
That would be awesome. Hope you're right. Can't help wondering why Canon would be keeping such performance a secret.

Perhaps similar to the pixel binning issue on video? in order to hype it on one product you need to trash your other products.

In areas such as AF and FPS many users simpley don't need them and ISO tend to be hyped by max numbers rather than pointing out the weakness of lesser models at similar levels. DR and video quality on the other hand are issue that relate to most users and talking openly about weaknesses probabley isnt something there keen on.

canon rumors FORUM

Re: 5D3 same max dynamic range as the 5D2???
« Reply #30 on: March 09, 2012, 03:46:15 AM »

altenae

  • EOS M2
  • ****
  • Posts: 185
  • Enjoy Wildlife
    • View Profile
    • Wildlife-photos
Re: 5D3 same max dynamic range as the 5D2???
« Reply #31 on: March 09, 2012, 03:47:04 AM »
We are all spending hours and hours on numbers.
Very sad. Very sad.

Some time ago I saw an image taken from a MF and I was blown away.
I checked dxo and the DR was worse than the Sony Nex 10.

There are two things :
Numbers and what your eyes see.

I am going out now to take some pictures.
Or should I stay home to do some more calculations.

Please guys use the camera for where it was made for.

Nice article here:(eyes vs. numbers)

http://www.luminous-landscape.com/essays/eyes-vs-numbers.shtml
« Last Edit: March 09, 2012, 03:51:33 AM by altenae »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13624
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #32 on: March 09, 2012, 04:02:36 AM »
I listed my entire procedure in detail! I wasted like ten minutes typing it out. It's the same thing DxO does.

Really?  It doesn't sound like you did this:

Quote from: DxO Labs
We perform noise measurement using a transmission target placed on top of a uniform light box. The transmission target, designed by DxO Labs, is made of a plate of thick black plastic with precision-drilled holes which are equipped with a range of neutral density filters designed to absorb the light identically for all wavelengths.

The filters are made of pure optical glass with no structures that can be measured as noise. (While other image quality measurement solutions make use of printed targets, we believe such targets are inappropriate for noise measurement testing, as the intrinsic noise pattern of the print paper may be recorded by the tested camera and then confused with the camera’s own noise pattern.)

We place high-density filters on neighboring positions to limit reciprocal illumination of the patches.
The light box (placed behind the target) is composed of two fluorescent daylight spectrum tubes with a diffusing sheet on top, achieving a perfect uniformity on each filter. The luminance is about 1500cd/m2.

We use filters having different light absorption levels ranging from 0% to 99.99% in order to test across a dynamic range of 4 density steps (= 13.3 f-stops — a dynamic range much greater than today’s digital cameras). When shooting such a chart, the sensor of the camera being tested sees a wide range of light levels, with a 1/10,000 ratio from minimum to maximum. For comparison, a printed target dynamic is typically 2 density steps (6.65 f-stops), which is inadequate to simulate high dynamic range or back-lit scenes.

Each uniform zone on the chart (a “patch”) is measured for luminance (cd/m2) with a certified luminance-meter; then all the values are input into DxO Analyzer software.

Once we measure the target and calibrate the DxO Analyzer software, the selected camera shoots an image of the noise target at different ISO settings, and we measure the noise for each color channel of the target image (R, Gr, Gb, B). We compute the mean gray level and noise values for each patch and for all images shot at different ISO settings. We then interpolate these numerical values for all gray levels to calculate and plot signal-to-noise ratio (SNR) curves, from which DxO Analyzer extracts the SNR 18%, the dynamic range, and the tonal range.

Perhaps the 10 minutes you spent typing out your 'procedure' wasn't the only time you've wasted on this 'analysis'...
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3752
  • POTATO
    • View Profile
    • Nature Photography
Re: 5D3 same max dynamic range as the 5D2???
« Reply #33 on: March 09, 2012, 04:12:55 AM »
Somehow i doubt you would be so sensitive about its methodology and findings if LetTheRightLensIn's thread title was "Zomg 5d3 has 14DR u guys!"

In my case i tend to believe him; his approach -with the amount of data we have so far- seems reasonable, his results fits other people tests, and since Canon itself hasnt bothered to announce something about low iso performance, tends to point in the direction LetTheRightLensIn is.

If he insisted his means of arriving at such a conclusion was "exactly the same as DXO's", yes, I'd have issue with it. Its not so much about what he is saying as much as it is about the fact that he is claiming his approach and results are just as good and trustworthy as an institution that has a demonstrated track record of using consistent, repeatable, verifiable techniques to produce valid results.

I'd figure most of the people on this forum realistically expect about a 1 stop improvement in DR, but hope for a full two stop improvement that would maximize the potential of a 14-bit sensor. Realistically, I think the 5D III will exhibit around a 1-stop improvement in DR once tested by DXO (12.7-13 stops). However...thats all just expectation and hope. If someone told me they had run some kind of test that used "masked off" dark pixels from a RAW file and was able to prove with as much accuracy as DXO that the 5D III achieved exactly 14.0 stops of DR...I'd be extremely skeptical. (Having an understanding of the CR2 format, sounds a bit sketchy to me as the masked pixels are intended to be used to set black level by RAW converters, and they are purposely fixed at around 1024, rather than around 0 (don't ask me why Canon does that...its just what they do)...I don't believe they are a valid basis to use for measurement of DR.)

At BEST, I think Canon might be able to achieve the same as Sony, 13.86 stops of DR, which is within the margin of error for 3 electron read noise and quantization error in ADC with a 14-bit sensor. Assuming Canon has not actually moved to a column parallel on-die ADC system in their sensors like Sony did (since Sony currently holds all the patents for such technology), realistically I'd be doubtful that DR will improve much past 13 stops, let alone 13.8. Given Canon's track record with improvements to DR, they have consistently improved it with each successive camera in any one of their lines. Taking the 1D APS-H series as an example using DXO data, the II had 11.1 stops, the III had 11.7 stops, and the IV had 12 stops. Read noise for those cameras was 29.7, 22.3, 16.6 e-, respectively. Canon has shown a notable ability to improve their low ISO dynamic range with reductions in read noise, and they have made some significant claims about improved DR that could bite them in the ass if it turns out to be false, or worse, the 5D III exhibited worse DR than the 5D II.

Realistically, I'd expect the 5D III to approach 13 stops of DR, especially if they reduce read noise to 7D levels (8.7 e-). If Canon has been able to reduce read noise to Sony Exmor levels of about 3 e-, I'd expect the 5D III to push 13.5 stops of DR or more. However its all just educated expectation...even if I thought I could get accurate measurements from .CR2 files from sample images on the net, I wouldn't make any bold claims about my measurements being accurate to any degree, and especially not as accurate as DXO. I have no idea what kind of lighting was used to illuminate any sample image I might get my hands on, whether the camera was a pre-production model or not, whether the exposure settings were set exactly correct to measure DR for the type and intensity of the bulb used to illuminate the scene. So far no sample image on the net that I've come across is really ideal for measuring DR with any degree of accuracy as there are no guarantees about the viability of the information contained within them.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13624
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #34 on: March 09, 2012, 04:15:59 AM »
his approach -with the amount of data we have so far- seems reasonable...

It's basing a conclusion on a questionable analysis of a single image.  It's like asking one person who they plan to vote for in the next presidential election, then predicting that candidate will be the next President, with the data gathered thus far (n=1), that seems reasonable, too.

Sweeping conclusions based on very limited data, combined with outright false statements ("It's the same thing DxO does"), adds up to a lack of credibility.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

traveller

  • 5D Mark III
  • ******
  • Posts: 655
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #35 on: March 09, 2012, 05:11:36 AM »
Oh boy, it's started again!   ;)

Guys, I appreciate that people have spent quite a bit of time trying to analyse this, but can we not just hold off on this whole debate for a month until DXO Mark publishes their report on the 5D MkIII's and D800's sensors? That way we won't have to argue over the validity of test images, method etc.

This isn't meant as criticism of anyone, it's just that I don't think it's worth getting riled up at this early stage... 

Tijn

  • Guest
Re: 5D3 same max dynamic range as the 5D2???
« Reply #36 on: March 09, 2012, 05:16:45 AM »
Does your method involved the use of a specialized test device designed just for the purpose of testing DR, that is finely calibrated to produce consistent test images containing tonal swatches of exact known intensities, lit by a very specific kind of light bulb that is capable of emitting a very specific amount of light, with a very specific color balance, at a very specific intensity?
Look, you clearly want to introduce a research hardware setup into this story. You're indicating that his method, lacking all this, is therefore not good enough. What exactly about his method is wrong, if you will, and how is it he's getting consistent results with the RAW files he's been "fiddling" with? Consistent results that are in line with "specialized test device" measurements, even?

Quote
Claiming that your results are as accurate as theirs comes with a certain responsibility, and I'm sorry...but you repeatedly claiming that your as accurate and reliable as DXO, well its actually rather sad at this point.
He actually doesn't claim to be reliable, and he hopes he's wrong. He's listed out his entire procedure. We can take it with as much of a grain of salt as we like, as we have all the information. We can see what he did, we can see what he did wrong and we can think of the limitations of his findings ourselves. All of that does not take away the fact that his method of measuring these RAW files comes up with consistent results of the DxO measurements. That means that the chance that his method is "valid" has increased. Consistency is a good thing in different approaches to a similar kind of measurement.

Quote
As it stands, all we really have at this point is your personal word...but someones word only has meaning to those who have reason to trust it.
No. He's told you how to do it, and he told you what he found. If you don't trust the man, go do it yourself. It's totally reproducable, which he also noted himself by the way because 4 other bloggers on these things came up with the same results themselves.

Quote
We all know what your opinion is
In this discussion between him and you, you're the only one whose level of discussion does not come above that of "opinion". He has no personal opinion, just a method and an interest in the matter. If he made an error, be so kind as to point out that error to him. You're the only opinion here, and that opinion seems to be close to sticking your fingers in your ears and saying lalala I don't hear you; because you have faith in Specialized Test Devices© but not in people doing manual measurements in RAW files and getting the same results. If you don't understand the research methods, I can imagine you only really TRUST the people in white lab coats and the big bleepy complex-looking machines. But nobody's asking you to TRUST this research. You are however trying to discredit it simply for not being a complex-machine lab coat. Give this guy some credit, he's getting consistent results. You clearly don't understand it, so really, you might as well just not comment unless you have something to say that will help us evaluate this method of DR data gathering.
« Last Edit: March 09, 2012, 05:19:45 AM by Tijn »

canon rumors FORUM

Re: 5D3 same max dynamic range as the 5D2???
« Reply #36 on: March 09, 2012, 05:16:45 AM »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13624
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #37 on: March 09, 2012, 05:30:00 AM »
...something to say that will help us evaluate this method of DR data gathering.

Still not sure I'd call it data gathering.  As for consistent results, the method guarantees them.  As for results that match proper empirical tests, a broken analog clock is correct twice a day. But whatever.

Here's a question: when 'measuring' the dark noise level in the 'side masked area', what is the probability that the numbers provided for the dark signal are meaningful?  He reports average values of 1024 for the 5DII and 2048 for the 5DIII - exact 2n values.  Personally, I find that suspicious, and it seems more likely that those values are not actual signal, but rathet result from the camera firmware setting those pixels, which are outside the image area, to an arbitrary value as it writes out the .CR2 file.  In that case, both the absolute value of the dark signal and the noise of that signal are not valid for DR determination.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

altenae

  • EOS M2
  • ****
  • Posts: 185
  • Enjoy Wildlife
    • View Profile
    • Wildlife-photos
Re: 5D3 same max dynamic range as the 5D2???
« Reply #38 on: March 09, 2012, 05:31:20 AM »
Numbers , numbers, numbers.
Go outside take pictures.
If you are happy and your customer is happy, then who cares if the DR is 10 or 11.

The same story everytime a new camera is about to release.

Tijn

  • Guest
Re: 5D3 same max dynamic range as the 5D2???
« Reply #39 on: March 09, 2012, 05:34:47 AM »
It's basing a conclusion on a questionable analysis of a single image.  It's like asking one person who they plan to vote for in the next presidential election, then predicting that candidate will be the next President, with the data gathered thus far (n=1), that seems reasonable, too.
Now that's just mean, considering the DR measurement to be a binary value of 1 or 0 in a comparison... At least give a fair example.

Frog population. It's said frogs are heavier on island B than they are on island A. Island A frogs weigh 100g, with a standard deviation of about 2.
Give a guy a frog from island B, have him measure a frog's weight for 15 minutes, and then take his weight measurement to be a representative weight for that type of frog on island B.
Frog weighs 98g. Hey, this frog is almost just as heavy! Therefore, it looks like frogs on Island B aren't heavier at all; perhaps even lighter.

And even with all that, I'm not really sure if his measurements isn't actually more reliable than that.

Perhaps a ruler length measurement would be more fair. A ruler is found in some unknown metrics, but with descernible equally-distanced lines on it. You measure the distance between 10 lines to be 3.22 inches. From this measurement you deduce that for the next ruler of that type you find, the distance will also be 3.22 inches when you measure 10 lines.

For this guy's measurement, he's measuring the darkest bit of a RAW file (masked sensor area) and the brightest bit of the file (blown-out highlight). My question would be: if his masked sensor area measurement is valid (and may be taken as "black") and if his highlight may be taken as valid as well; would there be any significant difference examining 20 images rather than 1? For a RAW file linked to a sensor, isn't "blown-out white" always the same value in the RAW? It's blown out at some value, right? Does that value change from picture to picture?

Tijn

  • Guest
Re: 5D3 same max dynamic range as the 5D2???
« Reply #40 on: March 09, 2012, 05:36:35 AM »
Here's a question: when 'measuring' the dark noise level in the 'side masked area', what is the probability that the numbers provided for the dark signal are meaningful?  He reports average values of 1024 for the 5DII and 2048 for the 5DIII - exact 2n values.  Personally, I find that suspicious, and it seems more likely that those values are not actual signal, but rathet result from the camera firmware setting those pixels, which are outside the image area, to an arbitrary value as it writes out the .CR2 file.  In that case, both the absolute value of the dark signal and the noise of that signal are not valid for DR determination.
Good point. Do you reckon the darkest meaningful parts of an image (blown-out black) would have a significantly lower value?
Also, if(?) you can take the blown-out highlights to be the same value every time (i.e. blown-out white for a specific sensor will always be the same value for any picture with that sensor in RAW files), is it not strange that he actually got the same DR number as DxO predicted (for the Mk2)?
« Last Edit: March 09, 2012, 05:48:06 AM by Tijn »

MazV-L

  • Guest
Re: 5D3 same max dynamic range as the 5D2???
« Reply #41 on: March 09, 2012, 05:38:03 AM »
Most of this technical jargon is way above my head  ::)

However, I do wonder how anyone can fairly test 2 cameras, one they have or have used, the other they've never laid their hands on or even know anyone who has for any length of time ???

It'd be interesting to hear all the various opinions of the people who will be using the 5Diii AFTER they've have it and have been using them for at least a month or so.
« Last Edit: March 09, 2012, 05:39:54 AM by MazV-L »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13624
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #42 on: March 09, 2012, 05:46:15 AM »
For a RAW file linked to a sensor, isn't "blown-out white" always the same value in the RAW? It's blown out at some value, right? Does that value change from picture to picture?

Do we know it's 'blown out white'?  You might have noticed that reds tend to blow out more easily than blues, and a blown out red/green will show up as blown white, but may still be below peak for blue, due to the spectral sensitivities of the Bayer mask. That's one reason DxO measures noise in separate color channels then averages them.  But since this guy is testing 'the same way as DxO' maybe he's doing that, but just didn't waste another 5 minutes to include that in the description of his method.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

canon rumors FORUM

Re: 5D3 same max dynamic range as the 5D2???
« Reply #42 on: March 09, 2012, 05:46:15 AM »

Tijn

  • Guest
Re: 5D3 same max dynamic range as the 5D2???
« Reply #43 on: March 09, 2012, 05:54:05 AM »
Do we know it's 'blown out white'?  You might have noticed that reds tend to blow out more easily than blues, and a blown out red/green will show up as blown white, but may still be below peak for blue, due to the spectral sensitivities of the Bayer mask. That's one reason DxO measures noise in separate color channels then averages them.  But since this guy is testing 'the same way as DxO' maybe he's doing that, but just didn't waste another 5 minutes to include that in the description of his method.
Good, so there's also the possibility that his blown out white area wasn't blown out on all channels. I think that's a good reason to take his results with a big grain of salt. And hope they are indeed wrong.  :)

Still I think you should give him a bit of credit for trying, he's not a 'bad guy' at all. With the concerns you've just expressed about the black measurement and the blown-out whites measurements, this whole discussion (flame war..?) could have been quite a bit shorter and there wouldn't be so much ad personam pollution. No need for yelling at one another, they're valid concerns. It's something completely different from someone yelling "your machine doesn't have dials, therefore your method must be faulty". Still interesting he got the same values with the 5Dmk2, however. Perhaps he had better raw files for those.
« Last Edit: March 09, 2012, 05:55:53 AM by Tijn »

neuroanatomist

  • CR GEEK
  • ********
  • Posts: 13624
    • View Profile
Re: 5D3 same max dynamic range as the 5D2???
« Reply #44 on: March 09, 2012, 06:00:50 AM »
Good point. Do you reckon the darkest meaningful parts of an image (blown-out black) would have a significantly lower value?

Also, if(?) you can take the blown-out highlights to be the same value every time (i.e. blown-out white for a specific sensor will always be the same value for any picture with that sensor in RAW files), is it not strange that he actually got the same DR number as DxO predicted (for the Mk2)?

The dark signal may or may not be lower, but more importantly, the noise may be different, and that's the denominator in the equation (in fact, I'm not sure why he's subtracting the dark signal from the max first - the standard calculation is full well capacity over read noise, which translates to log2 (average peak / dark std dev).

Semantics, but I wouldn't say DxO 'predicted' DR for the 5DII - they measured it. What this guy is doing is trying to predict it, and I'll take a proper measurement over a prediction any day.  I'm not saying he's definitely wrong, just that there's not enough data to go on.  If he downloads the IR RAW files for 10 more cameras, does the same test, and comes out spot on with DxO's measurements for them, that would be convincing.
EOS 1D X, EOS M, and lots of lenses
______________________________
Flickr | TDP Profile/Gear List

canon rumors FORUM

Re: 5D3 same max dynamic range as the 5D2???
« Reply #44 on: March 09, 2012, 06:00:50 AM »