June 25, 2018, 02:25:26 AM

Author Topic: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?  (Read 14197 times)

bludragon

  • PowerShot G7 X Mark II
  • **
  • Posts: 9
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #30 on: February 12, 2018, 01:32:02 AM »
Can anyone explain the math behind this one?  I mean, if the normal image is made up by summing the two values from each half of a dual pixel, which each collect half the light, how is it better than just halving brightness in the original image?

The only way I could see it helping is if the file format runs out of bits to store the max value from combining the dual pixels.  That is the case for jpegs which drop some of the highlight detail, but I'd be surprised if it is the case for regular RAW files.

canon rumors FORUM

Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #30 on: February 12, 2018, 01:32:02 AM »

SecureGSM

  • EOS 5D Mark IV
  • ******
  • Posts: 744
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #31 on: February 12, 2018, 02:29:49 AM »

https://www.rawdigger.com/howtouse/canon-dual-pixel-mode-highlights-are-there

Can anyone explain the math behind this one?  I mean, if the normal image is made up by summing the two values from each half of a dual pixel, which each collect half the light, how is it better than just halving brightness in the original image? ...

hne

  • EOS 80D
  • ****
  • Posts: 282
  • Gear limits your creativity
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #32 on: February 12, 2018, 04:25:50 AM »
Can anyone explain the math behind this one?  I mean, if the normal image is made up by summing the two values from each half of a dual pixel, which each collect half the light, how is it better than just halving brightness in the original image?

The only way I could see it helping is if the file format runs out of bits to store the max value from combining the dual pixels.  That is the case for jpegs which drop some of the highlight detail, but I'd be surprised if it is the case for regular RAW files.

Each half-pixel is going through an analog-to-digital converter and ends up in a 14 bit memory slot as a number between 0 and 16383, for example 8375 being 10 0000 1011 0111 and 8402 being 10 0000 1101 0010. The two half-pixels are then added together into a new 14 bit memory slot, 8375+8402=16777 or 100 0001 1000 1001 which wouldn't fit in 14 bits. The value is clamped to largest number you can encode with 14 bits: 16383=11 1111 1111 1111. Blown higlight.

If you have dual pixel raw turned on, you'd have two pictures in your .CR2, one with blown hilights being clamped to 11 1111 1111 1111 and one with 10 0000 1101 0010. The number of extra stops of hilights is 0.034285: (ln(16777)-ln(16383))/ln(2)

The largest gain from this is when you almost blow out both half-pixels: 11 1111 1111 1110+11 1111 1111 1110=111 1111 1111 1100 (3276 in decimal) which gives .9999 stops extra hilight detail which means very close to double the brightness recorded. The math behind the number of stops: (ln(32764)-ln(16383))/ln(2) ≃ .999912

In short, as long as your hilights are at most blown by just a hair under a stop, this would make them recoverable, albeit with a minor parallax shift. Anything over 1 stop would still give an extra stop of hilights in the gradient towards the blown-out region.

Canon could implement this hilight recovery directly in camera, if they were to add a 15th bit to the .CR2 format.
But then, the .CR2 format also wastes a few bits on a non-zero black point (which is really good for shadow denoise) so we wouldn't get 15 bits of DR just like that.
Preferred gear: 5D IV, 35/1.4L, 85/1.4L IS, 70-200/2.8L IS, Elinchrom RX
Backup: 5D II, 17-40/4L, 85/1.8, 100-400/4.5-5.6L, Yongnuo + 622

Sharlin

  • EOS 5D Mark IV
  • ******
  • Posts: 655
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #33 on: February 12, 2018, 07:17:39 AM »
So the (sub)photosites have some headroom in well capacity, being able to represent higher values before saturating than the ADC and post-ADC digital electronics. Basically the sensor has a ”native” ISO lower than 100.

privatebydesign

  • Canon EF 800mm f/5.6L IS
  • ***********
  • Posts: 6893
  • Would you take advice from a cartoons stuffed toy?
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #34 on: February 12, 2018, 08:40:05 AM »
So the (sub)photosites have some headroom in well capacity, being able to represent higher values before saturating than the ADC and post-ADC digital electronics. Basically the sensor has a ”native” ISO lower than 100.

No, they have half the well capacity, it is a maths issue.

If a 14 bit ADC had a range from 0-10, 10 being clipped, then if each half is below 5 the highlight will have detail in the main frame. If both sub pixels register 8 then in the main frame the value would be >10 so blown, but the value from the sub frame is 8, so it retains highlight detail.

The point is the main frame has the FWC of two sub pixels added together to make the output value limited at 14 bits, this means if the value of the sub frames pixel is less than 1/2 the 14 bit ceiling you will get additional highlight detail out of the sub frame. As we all know 1/2 or x2 is equal to 1 stop, so in theory there is a possible 1 stop improvement in highlight detail using this technique.
Too often we lose sight of the fact that photography is about capturing light, if we have the ability to take control of that light then we grow exponentially as photographers. More often than not the image is not about lens speed, sensor size, DR, MP's or AF, it is about the light.

Iliah Borg

  • PowerShot G7 X Mark II
  • **
  • Posts: 6
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #35 on: February 12, 2018, 10:43:20 AM »
Gentlemen:

We made some changes,
http://updates.fastrawviewer.com/data/DPRSplit/DPRSplit-0.8.2.10-x64-Setup.exe
http://updates.fastrawviewer.com/data/DPRSplit/DPRSplit-0.8.2.10.dmg

- font size on the first run (and margins for Windows) are set based on the screen size. If the base font size setting is changed to smaller, the application window is also changed to smaller, accordingly. Works OK with 1200 by 800 pixels screens. The control is in Preferences.

To ease the pain for those who want to use both frames to get less noisy shadows from the "main" (composite, A+B) frame:
•   Adjust EXIF shutter speed by -1EV for second frame (for HDR merge) – simulates reduction of exposure by 1 stop by setting the shutter speed in the EXIF of the auxiliary frame 1 stop faster. This may be useful (and even necessary) while merging first and second frame in certain HDR stacking applications that analyze frames for exposure parameters.


For the first beta, we were careful not to confuse the raw converters that do not process the extreme highlights correctly. Now we offer a finer control in Preferences:
•   Data Maximum: controls the content of the DNG White Level tag:
o   Normal White –«white» will be set to the Normal White value found in makernotes of CR2 file.
o   Specular White (default) – «white» will be set to the SPecular White value found in makernotes of CR2 file.
o   Data Max (or specular) – «white» will be set to the actual maximum found in raw data of the CR2 file or to Specular White, if Specular White is higher.
o   Data format max – 16383 (minus the black level if the processing mode is set to subtracting black). This option may result in color artifacts in the highlights (such as pink clouds) if the raw converter incorrectly clips the highlights after applying white balance.
« Last Edit: February 12, 2018, 10:52:15 AM by Iliah Borg »

Mt Spokane Photography

  • CR GEEK
  • ***************
  • Posts: 14244
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #36 on: February 12, 2018, 12:19:56 PM »
I tried the updated app.  I used lightroom, helicon, and photomatix on the same image.  I liked it for the test because of the very dark bird feeder and the almost blown out white post.

Once again, Lightroom = Fail, no need to show another repeat.

Helicon, excellent output, it will need a little tweaking to boost the shadows.

Photomatix, I liked the "Realistic" output because it is warmer and boosts the shadows.

Top = Helicon, Bottom = Photomatix..


canon rumors FORUM

Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #36 on: February 12, 2018, 12:19:56 PM »

Iliah Borg

  • PowerShot G7 X Mark II
  • **
  • Posts: 6
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #37 on: February 12, 2018, 12:47:02 PM »
Thank you for testing.

Here is the "main" frame, magenta highlighting indicates the green channels of raw data being blown out, red highlighting is where both green and blue channels are clipped, black spots are indicating clipping in all the channels. From "Statistics" you can see how many pixels / % of each of the colour channels are clipped.


Now, the "auxiliary" frame, clipping is very tolerable:


Settings:


zim

  • EOS-1D X Mark II
  • *******
  • Posts: 1716
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #38 on: February 12, 2018, 03:21:19 PM »
Sounds like a really interesting ‘development’ but FWIW here’s the file processed through Affinity Photo develop module. I’m a big fan of AP but not of the develop module, it has IMHO many issues I much prefer DXO Pro for RAW development but my version doesn’t support the 5D4.
Anyway my take on this image is that it’s no test for the 5D4 sensor which with the most basic of adjustments (highlight adjustment and a little tone curve) in AP can recover those highlights. I didn’t really pay much attention to shadows and I didn’t use Overlays which would have given me much more control of the image.

Don’t you need something a bit more extreme to show if there are any benifits to dual pixel RAW?

Mt Spokane Photography

  • CR GEEK
  • ***************
  • Posts: 14244
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #39 on: February 12, 2018, 09:30:02 PM »
Thank you for testing.

Here is the "main" frame, magenta highlighting indicates the green channels of raw data being blown out, red highlighting is where both green and blue channels are clipped, black spots are indicating clipping in all the channels. From "Statistics" you can see how many pixels / % of each of the colour channels are clipped.


Now, the "auxiliary" frame, clipping is very tolerable:


Settings:



So it looks like I was able to recover almost all of the 18% clipped pixels - Impressive!  I focused on the black portion of the feeder, so the camera did boost the exposure because of the dark area.  I expected that.  It was set to evaluative metering, but still gave some priority to the center of the frame.
« Last Edit: February 12, 2018, 09:34:02 PM by Mt Spokane Photography »

Iliah Borg

  • PowerShot G7 X Mark II
  • **
  • Posts: 6
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #40 on: February 13, 2018, 04:17:25 PM »
So it looks like I was able to recover almost all of the 18% clipped pixels - Impressive!  I focused on the black portion of the feeder, so the camera did boost the exposure because of the dark area.  I expected that.  It was set to evaluative metering, but still gave some priority to the center of the frame.

I think yes, the pole looks natural, without dull grey spots resulting from reconstructing highlights through interpolation.

Some folks prefer Lumariver HDR ( http://www.lumariver.com ) because it results in DNG file with virtually no ghosting.

Mt Spokane Photography

  • CR GEEK
  • ***************
  • Posts: 14244
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #41 on: February 13, 2018, 09:34:59 PM »

Some folks prefer Lumariver HDR ( http://www.lumariver.com ) because it results in DNG file with virtually no ghosting.

Unfortunately, its Mac only.  I was really unhappy in 1992 when my company standardized on pc's, but I've become so adept at using one that a Mac might seem strange 25+ years later.

Iliah Borg

  • PowerShot G7 X Mark II
  • **
  • Posts: 6
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #42 on: February 13, 2018, 10:44:03 PM »

Some folks prefer Lumariver HDR ( http://www.lumariver.com ) because it results in DNG file with virtually no ghosting.

Unfortunately, its Mac only.  I was really unhappy in 1992 when my company standardized on pc's, but I've become so adept at using one that a Mac might seem strange 25+ years later.

Ah. I'm sorry for forgetting to mention this limitation in my post.

canon rumors FORUM

Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #42 on: February 13, 2018, 10:44:03 PM »

IglooEater

  • EOS 5DS R
  • ******
  • Posts: 899
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #43 on: February 14, 2018, 07:37:11 AM »
So the (sub)photosites have some headroom in well capacity, being able to represent higher values before saturating than the ADC and post-ADC digital electronics. Basically the sensor has a ”native” ISO lower than 100.

No, they have half the well capacity, it is a maths issue.

If a 14 bit ADC had a range from 0-10, 10 being clipped, then if each half is below 5 the highlight will have detail in the main frame. If both sub pixels register 8 then in the main frame the value would be >10 so blown, but the value from the sub frame is 8, so it retains highlight detail.

The point is the main frame has the FWC of two sub pixels added together to make the output value limited at 14 bits, this means if the value of the sub frames pixel is less than 1/2 the 14 bit ceiling you will get additional highlight detail out of the sub frame. As we all know 1/2 or x2 is equal to 1 stop, so in theory there is a possible 1 stop improvement in highlight detail using this technique.

So could canon have given the sensor an extra stop of DR by using a 16 bit ADC?  Edit: removed a dumb question
« Last Edit: February 14, 2018, 07:48:29 AM by IglooEater »

privatebydesign

  • Canon EF 800mm f/5.6L IS
  • ***********
  • Posts: 6893
  • Would you take advice from a cartoons stuffed toy?
Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #44 on: February 14, 2018, 08:01:15 AM »
So the (sub)photosites have some headroom in well capacity, being able to represent higher values before saturating than the ADC and post-ADC digital electronics. Basically the sensor has a ”native” ISO lower than 100.

No, they have half the well capacity, it is a maths issue.

If a 14 bit ADC had a range from 0-10, 10 being clipped, then if each half is below 5 the highlight will have detail in the main frame. If both sub pixels register 8 then in the main frame the value would be >10 so blown, but the value from the sub frame is 8, so it retains highlight detail.

The point is the main frame has the FWC of two sub pixels added together to make the output value limited at 14 bits, this means if the value of the sub frames pixel is less than 1/2 the 14 bit ceiling you will get additional highlight detail out of the sub frame. As we all know 1/2 or x2 is equal to 1 stop, so in theory there is a possible 1 stop improvement in highlight detail using this technique.

So could canon have given the sensor an extra stop of DR by using a 16 bit ADC?  Edit: removed a dumb question

Yes. 14 bit files are holding Canon sensor capabilities back nowadays, more bit depth would realize more DR even with todays dual pixel sensors.
Too often we lose sight of the fact that photography is about capturing light, if we have the ability to take control of that light then we grow exponentially as photographers. More often than not the image is not about lens speed, sensor size, DR, MP's or AF, it is about the light.

canon rumors FORUM

Re: Want to Increase The Dynamic Range of Your EOS 5D Mark IV Raw Files?
« Reply #44 on: February 14, 2018, 08:01:15 AM »