1Dx ISO100 high DR shadows SNR improvement

[Neutral]

Hi Neutral, thanks for sharing the technique and outcomes.... interesting (& understandable).

The only thing is, that the final output file you show, has significantly more noise and vertical banding than the base file "1DX_ISO100_EV0-Crop200%.JPG".

The file 1DX_ISO100_EV0-Crop200%.JPG is the cleanest for me... in fact very good for a 200% crop.

At least, that is what displays on my PC & Monitor.  Anyone else?

Paul

As I mentioned above this method gives about 2 stops of visible SNR improvements, not full 3 stops and this above is "stress" test to see  camera limits and what could be dome with that.
  So definitely 0EV ISO100 image IQ will still be better than -3EV image pulled up 3 stops using this shooting method.
But even this is significant improvement.
 At 100% crop this difference is almost invisible and for normal shots with 0EV it gives enough room for shadows details recovery - almost full 2 stops
Just pull shadows up to the level where banding pattern is just below black point and get almost perfect image
Vertical banding:
 As I mentioned - you can filter down only random noise thus pulling fixed details up from the random noise. Low amplitude fixed image details becomes visible as they not any more masked out by random noise. Image banding are fixed sensor imperfections which also filtered out from random noise and become more visible as well.
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop  - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

[Neutral]

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling

[Neutral]

@OP - if you're that desperate to pull +3EV in shadows you could use split-toning action in PShop with (obviously) using blurred layer to pull those shadows up.

here's a dload link> http://www.mediafire.com/?po4x8dcgg5msr6a for that action, let me know if it works for ya.

---------------
Sorry, I am afraid that you are a bit confused about the whole subject.
---
1. Subject is not that "someone is desperate to pull up shadows to +3EV" )))
  Subject is that 1Dx has some limitations at low ISO range  and how to push limits of the camera  so that it could be more usable in some photography scenarios for fixed still images (e.g. evening/night city landscapes, some interiors shots etc) to get better IQ  - to make it close to Nikon D800 in this area.
---
2. Subject is not how to blur  noise at problematic image areas reducing both visible noise and also smearing fine image detail.
 Subject is just opposite - how to preserve that fine details when pulling up shadows by filtering noise out.
---
3. Subject is not about post processing - but how to get best possible image IQ directly out of camera which required minimum post processing thus saving extremely valuable time.
 As a matter of fact blurring noise (together with details) is the ancient method used when modern high quality NR S/W features were not available.
 I used that (blurring problematic areas) many years back with my first digital camera.
  Also no need to go to PS now for that - all could be done in LR. One just need to  have adequate amount of NR applied to whole image to keep fine details and then apply more NR and de-sharpening to the specific local areas using local brush tool.
  For me since I upgraded from LR2 to LR3 and then to LR4 I almost stopped using PS - only for something very special.  For 99% of shots I can do all required with LR4 or DXO Optic Pro (ver. 8 now) or with Phase One  C1 when I need to see highest rendering IQ  and to work selectively with colors  - very useful with local adjustment tools which are  also available in C1 - similar to LR.

[Neutral]

This is routinely applied in astrophotograpy if combined with precision German Equatorial Mounts. It is also used when combining many video frames into one photo (see registax). But I doubt it can be used elsewhere. You have to exclude leaves, people, water, clouds maybe and I am sure I haven't thought of everything...

Definitely this is not applicable to everything.
Only applicable for fixed still images and one  need to use steady tripod for that.
But for night/evening city shots - this is something that really helps - you can recover fine  details in very deep shadows.
As for water – this is 50-50 - this could be used as substitution for ND filter to capture water motion instead of freezing it - e.g. shooting waterfalls.

[Neutral]

  In order to save time on explanations I just searched internet  for “averaging noise” and found very nice article – exactly about the same subject where everything explained to full extent.
http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm
So here is also typical photography application to for this - which I was referring in my posts.
Luckily it could be easily done with 1Dx using method I described above.
------------
And now to scientific numbers)))
  From article above :  “ In general, magnitude of noise fluctuation drops by the square root of the number of images averaged, so you need to average 4 images in order to cut the magnitude in half”
  So averaging 9 photos reduce noise fluctuations 3 times which is equivalent to 10*log(Sqrt(9)) = 4.7dB.
  Look at DXOMark SNR curves for 1Dx and add 4.7 db to that at the flat left shoulder part at low ISOs and compare that to Nikon D800.
  Does not look  too bad after all that.
Not sure if D800 has the same multi-exposure mode. 
If yes then it could get even more amazing results.
As for banding – I believe it could be easily removed as well:
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

[nightbreath]

I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

I heard sensor heat correlates with sensor banding, so it's not that simple 

[nightbreath]

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling

I didn't find the message with sample images that lacked visible noise, but here is another message about the same thing:

http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent

the techradar test:

TIFF images (after conversion from raw) have a consistently good signal to noise ratio across the sensitivity range comparing closely to the Canon EOS 1Ds Mk III and just having the edge over the Nikon D800. It also shows a slight improvement at all sensitivities over the Canon EOS 5D Mk II.

TIFF files (after conversion from raw) have a high dynamic range with results comparing closely to the Nikon D4 and D800. Compared with the Canon EOS 5D Mk II, there is a marked improvement showing over 2EV greater range at the lower end of the sensitivity scale.

i thought techradar is a DxO certified lab?   
http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588

[Neutral]

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling

I didn't find the message with sample images that lacked visible noise, but here is another message about the same thing:

http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent

the techradar test:

TIFF images (after conversion from raw) have a consistently good signal to noise ratio across the sensitivity range comparing closely to the Canon EOS 1Ds Mk III and just having the edge over the Nikon D800. It also shows a slight improvement at all sensitivities over the Canon EOS 5D Mk II.

TIFF files (after conversion from raw) have a high dynamic range with results comparing closely to the Nikon D4 and D800. Compared with the Canon EOS 5D Mk II, there is a marked improvement showing over 2EV greater range at the lower end of the sensitivity scale.

i thought techradar is a DxO certified lab?   
http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588

I do not know if TechRadar  is certified DXO lab, I think they just use some DXO equipment but how they use  that and how they interpret results I do not know. Have seen that article before but did not trust much  what they were publishing .
 I more rely on pure Math - all the time in my professional life that was first stage in any systems development.  And then implementation results were compared against initial math results to see if design was OK or something was missed or done wrong - initially or at later stage.
 So as you suggested I did noise comparison for original RAW and  TIFF (lossless converted from RAW, to16 bit TIFF, color space is the same Adobe RGB, no re-sampling - the same pixel count).

 Noise measurements were done using Noise Ninja plug-in PS which gives NUMERIC noise profile of the image.
For initial RAW noise with shadows raised by +3EV   index is 56 (Luminance=15, Chroma=41)
For RAW converted to TIFF and then shadows raised by +3EV noise  index is 54 (Luminance=15 and Chroma=39).
As expected these are same results for  lossless conversion  with normal statistical distribution as Noise Ninja was selecting areas for noise profiling automatically and they a bit different.
See below results two 200% crops for RAW and TIFF and that snapshots of NJ noise profiler screen
Anyone who has NJ plug-in in PS can do the same and see numeric results

[Neutral]

I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

I heard sensor heat correlates with sensor banding, so it's not that simple 

I am sure you are right with this and it depends on sensor heat.
So it would be not possible totally eliminate banding only to suppres it to some extent.
To improve results it would  required to do calibration shot just before shooting session - similar what is done with Gray/wite/black card before session for later WB adjustement

[sanj]

Thank you!

[Neutral]

Thank you!

Appreciate that )))
   On of my favorite TV programms is Discovery Channel  and one of themes there is  Myth Breakers .
They take any Myth from anywhere  and do research to see if it right or no more than fiction.
Very funny)))

[Neutral]

  If anyone is interested I can publish  Noise Ninja  noise profile results for original 0EV image , and for shot with noise averaging in 1Dx  (the same  multi-exposure shot  with-3EV pulled up 3EV in LR  which was shown initially – there was no post processing NR applied to it at all) .
 It was interesting to see and compare noise profiles numeric results for both.
 Also how shot with noise averaging looks after LR NR and  after  Noise Ninja NR applied in PS – interesting to see the difference

[V8Beast]

If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times

[rpt]

If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times

But seriously, the best shot ever is with the lens cap on. It does not even have to focus... Or need IS. And that is not the definition of IS Neuro had a few threads back!

[V8Beast]

But seriously, the best shot ever is with the lens cap on. It does not even have to focus... Or need IS. And that is not the definition of IS Neuro had a few threads back!

Yes, I am a huge fan of lens-cap-on photography. It makes my $100 lens work just as well as a $14,000 Canon 800mm f/5.6L.