1Dx ISO100 high DR shadows SNR improvement

[Neutral]

One of the weak sides of the 1Dx is noise in deep shadows at base ISO.
It seems that nobody is arguing about that any more.
DXO 1Dx measurement results also show that very clearly on their measurement curves.
And I believe that nobody would be arguing that performance in this area is very critical for not just few people but for many people - for landscape photography, evening/night city shots and other sort of photography.
Though it is easy to argue that that Nikon D800 would be the better choice BUT why not to see how we could get better deep shadow SNR from 1Dx.
In one of my posts I mentioned that 1Dx has a very nice feature that could be used for that.
I will not name it now (will do at SOME next post) but here is some magic - example how it works (snapshots from LR screen - noise reduction is totally disabled )
1. First image shows 1dX shot at IS100 with EV=(-3) the Data Color noise calibration chart which was used for Noise Ninja NR S/W and PS plug-in. Shot with manual Out of Focus to blur target texture so it would not interfere with noise
2. Second one is the same with Exposure slider in LR set to +3 - looks the same as normal shot with EV=0
3. Third image is 200% crop of shot with EV=0 showing normal 1Dx ISO shadow noise
4. Fourth one is 200% crop of the EV=-3 shot with LR exposure set to +3 to raise shadows to normal level.
Level of 1Dx noise at ISO100 at -3 EV shadows is very clearly seen

And next post is little bit of magic - the similar EV=(-3) shoot from 1Dx recovered in LR - 200% crop to see the difference in SNR for random nose with the previous shot.

 

[Neutral]

OK, now is 200% crop of the similar EV=(-3) shoot from 1Dx using some special shooting method with 1Dx .
Shadows are recovered in LR with Exposure slider set to +3 to compensate for in camera EV =(-3) . No noise reduction in LR.
Note how SNR for random noise was improved.

So we have almost two stops improvement here for random noise SNR.
This means that with normal exposure it is possible to successfully recover details in very deep shadows at ISO100.
We still see some slight pattern noise on the image but I believe that there is some NR software that doing very well with reducing pattern noise - might be Topaz De-noise but not very sure as I do not have that.
If all combined together with some special shooting method with 1Dx it is possible to get shadow SNR on par with Nikon D800.
So now quiz - how this was done?

 

[victorwol]

No clue, but would be nice if you can share it. It's very interesting for sure.

 

[neuroanatomist]

The suspense isn't killing me. : But feel free to share, when you're ready...

 

[Viggo]

Bring it on!

 

[nightbreath]

Resizing down to 8Mpx file dimensions? ;D

 

[Neutral]

Resizing down to 8Mpx file dimensions? ;D

No, full sensor resolution )))

 

[Neutral]

OK, no magic - just "feeling" what is behind numbers )))

But seriously this method used for decades for filtering noise out in different applications.
This is use of simple math in real life use.
Noise is random - image is fixed.
So what is required is just to integrate information element for a longer time and random noise would be filtered out.
I believe that this is something that also was being used in astronomy to separate low level image details that could be even below random noise level. Also used in communication systems.
=========
So how this is done with 1Dx ???
------
1Dx has one feature which is intended for some artistic work but it could be used for something completely different.
This feature is multi-exposure – in one of the posts I mentioned about this for filtering out noise.
You can take up to 9 shots and 1Dx will average these shots into one.
As each shot has it's own random noise distribution then averaging 9 shots into one would essentially suppress noise - only fixed elements will be kept - so random electronic noise is significantly reduced but fixed patterns - real image or sensor defects are kept and cleared out of noise. This is equivalent to 9 times of longer integration time of the information element in the image detector.

How to do this in 1Dx:
Enable multi-exposure continuous shooting, number of exposures to 9, multi-exposure control to Average (to integrate over 9 images) , put mirror lock up with the set button release to reduce camera vibration (will be returned back by set button) , put camera in high speed continuous shooting mode, press shutter - mirror will be locked, press shutter again and keep it pressed - camera will take quickly 9 images in less than 1 sec , stops and then merge it ( integrate) into one image.
And you are done.
Now press set button to return mirror back and then you could see the result.
So nothing special - just to see things a bit deeper)))
Method is suitable for only still images .
In normal conditions such kind of shot will take about 0.75 sec with FPS =12
And of course could be only shot using strong stable tripod for camera.

 

[nightbreath]

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

 

[neuroanatomist]

Thought that might be the 'trick'. I use it all the time in photomicroscopy. Works great for test charts and fixed tissue. Not so well with anything that moves, which limits its utility for >95% of what I shoot...

 

[tron]

This is routinely applied in astrophotograpy if combined with precision German Equatorial Mounts. It is also used when combining many video frames into one photo (see registax). But I doubt it can be used elsewhere. You have to exclude leaves, people, water, clouds maybe and I am sure I haven't thought of everything...

 

[pj1974]

Hi Neutral, thanks for sharing the technique and outcomes.... interesting (& understandable).

The only thing is, that the final output file you show, has significantly more noise and vertical banding than the base file "1DX_ISO100_EV0-Crop200%.JPG".

The file 1DX_ISO100_EV0-Crop200%.JPG is the cleanest for me... in fact very good for a 200% crop.

At least, that is what displays on my PC & Monitor. Anyone else?

Paul

 

[PVS]

@OP - if you're that desperate to pull +3EV in shadows you could use split-toning action in PShop with (obviously) using blurred layer to pull those shadows up.

here's a dload link> http://www.mediafire.com/?po4x8dcgg5msr6a for that action, let me know if it works for ya.

 

[RLPhoto]

The d800 currently has the best 35mm sensor ever made. We all know this.

The 1Dx currently the best professional camera ever made. We all know this.

Two tools for two uses.

Canon could use some newer tech on the sensor base, and Nikon currently could use a newer AF system from the 3500 series. If you nail your exposures on either camera, only good photos will result.

 

[Tammy]

it's also the same method used in the "handheld night shooting" mode in the latest compact cameras, first introduced by Sony..

 

[Neutral]

Hi Neutral, thanks for sharing the technique and outcomes.... interesting (& understandable).

The only thing is, that the final output file you show, has significantly more noise and vertical banding than the base file "1DX_ISO100_EV0-Crop200%.JPG".

The file 1DX_ISO100_EV0-Crop200%.JPG is the cleanest for me... in fact very good for a 200% crop.

At least, that is what displays on my PC & Monitor. Anyone else?

Paul

As I mentioned above this method gives about 2 stops of visible SNR improvements, not full 3 stops and this above is "stress" test to see camera limits and what could be dome with that.
So definitely 0EV ISO100 image IQ will still be better than -3EV image pulled up 3 stops using this shooting method.
But even this is significant improvement.
At 100% crop this difference is almost invisible and for normal shots with 0EV it gives enough room for shadows details recovery - almost full 2 stops
Just pull shadows up to the level where banding pattern is just below black point and get almost perfect image
Vertical banding:
As I mentioned - you can filter down only random noise thus pulling fixed details up from the random noise. Low amplitude fixed image details becomes visible as they not any more masked out by random noise. Image banding are fixed sensor imperfections which also filtered out from random noise and become more visible as well.
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

 

[Neutral]

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling

 

[Neutral]

@OP - if you're that desperate to pull +3EV in shadows you could use split-toning action in PShop with (obviously) using blurred layer to pull those shadows up.

here's a dload link> http://www.mediafire.com/?po4x8dcgg5msr6a for that action, let me know if it works for ya.

---------------
Sorry, I am afraid that you are a bit confused about the whole subject.
---
1. Subject is not that "someone is desperate to pull up shadows to +3EV" )))
Subject is that 1Dx has some limitations at low ISO range and how to push limits of the camera so that it could be more usable in some photography scenarios for fixed still images (e.g. evening/night city landscapes, some interiors shots etc) to get better IQ - to make it close to Nikon D800 in this area.
---
2. Subject is not how to blur noise at problematic image areas reducing both visible noise and also smearing fine image detail.
Subject is just opposite - how to preserve that fine details when pulling up shadows by filtering noise out.
---
3. Subject is not about post processing - but how to get best possible image IQ directly out of camera which required minimum post processing thus saving extremely valuable time.
As a matter of fact blurring noise (together with details) is the ancient method used when modern high quality NR S/W features were not available.
I used that (blurring problematic areas) many years back with my first digital camera.
Also no need to go to PS now for that - all could be done in LR. One just need to have adequate amount of NR applied to whole image to keep fine details and then apply more NR and de-sharpening to the specific local areas using local brush tool.
For me since I upgraded from LR2 to LR3 and then to LR4 I almost stopped using PS - only for something very special. For 99% of shots I can do all required with LR4 or DXO Optic Pro (ver. 8 now) or with Phase One C1 when I need to see highest rendering IQ and to work selectively with colors - very useful with local adjustment tools which are also available in C1 - similar to LR.

 

[Neutral]

This is routinely applied in astrophotograpy if combined with precision German Equatorial Mounts. It is also used when combining many video frames into one photo (see registax). But I doubt it can be used elsewhere. You have to exclude leaves, people, water, clouds maybe and I am sure I haven't thought of everything...

Definitely this is not applicable to everything.
Only applicable for fixed still images and one need to use steady tripod for that.
But for night/evening city shots - this is something that really helps - you can recover fine details in very deep shadows.
As for water – this is 50-50 - this could be used as substitution for ND filter to capture water motion instead of freezing it - e.g. shooting waterfalls.

 

[Neutral]

In order to save time on explanations I just searched internet for “averaging noise” and found very nice article – exactly about the same subject where everything explained to full extent.
http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm
So here is also typical photography application to for this - which I was referring in my posts.
Luckily it could be easily done with 1Dx using method I described above.
------------
And now to scientific numbers)))
From article above : “ In general, magnitude of noise fluctuation drops by the square root of the number of images averaged, so you need to average 4 images in order to cut the magnitude in half”
So averaging 9 photos reduce noise fluctuations 3 times which is equivalent to 10*log(Sqrt(9)) = 4.7dB.
Look at DXOMark SNR curves for 1Dx and add 4.7 db to that at the flat left shoulder part at low ISOs and compare that to Nikon D800.
Does not look too bad after all that.
Not sure if D800 has the same multi-exposure mode.
If yes then it could get even more amazing results.
As for banding – I believe it could be easily removed as well:
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.