canon rumors FORUM

Gear Talk => EOS Bodies - For Stills => Topic started by: Neutral on November 04, 2012, 12:28:45 PM

Title: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 04, 2012, 12:28:45 PM
One of the weak sides of the 1Dx is noise   in deep shadows at base ISO.
It seems that nobody is arguing about that any more.
 DXO 1Dx measurement results also show that very clearly on their measurement curves.
And I believe that nobody would be arguing that performance in this area is very critical for not just few people but for many people  - for landscape photography, evening/night  city shots and other sort of photography.
Though it is easy to argue that that Nikon D800 would be the better choice BUT why not to see how we could get better deep shadow SNR from 1Dx.
In one of my posts I mentioned that 1Dx has a very nice feature that could be used for that.
I will not name it now (will do at SOME next post) but here is some magic - example how it works (snapshots from LR screen - noise reduction is totally disabled  )
1.  First image shows 1dX  shot at IS100  with EV=(-3)   the Data Color noise calibration chart  which was  used  for Noise Ninja NR S/W and PS plug-in. Shot  with  manual Out of Focus  to blur target texture so it would not interfere with noise
2. Second one is the same with Exposure slider in LR set to +3  - looks the same as normal shot with EV=0
3. Third  image is 200% crop  of shot with EV=0 showing normal 1Dx ISO shadow noise
4. Fourth one is 200% crop of the EV=-3 shot with LR exposure set to +3 to raise shadows to normal level.
     Level of 1Dx noise at ISO100 at  -3 EV shadows is very clearly seen
 
 And next post is  little bit of magic  - the similar EV=(-3) shoot from 1Dx  recovered in LR - 200% crop  to see the difference  in SNR for random nose with the previous shot.
 
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 04, 2012, 12:30:15 PM
OK, now is 200% crop of the similar EV=(-3) shoot from 1Dx  using some  special shooting  method  with 1Dx . 
Shadows are recovered in LR with Exposure slider set to +3 to compensate for in camera EV =(-3) . No noise reduction in LR.
Note how SNR for random noise was improved.

 So we have almost two stops  improvement here  for random noise SNR.
This means that with normal exposure it is possible to successfully recover details in very deep shadows at ISO100.
We still  see some slight pattern noise on the image but I believe that there is some NR software that doing very well with reducing pattern  noise - might be Topaz  De-noise  but not very sure  as I do not have that.
  If all combined together with some special shooting method with 1Dx  it  is possible to get shadow SNR  on par with Nikon D800.
So now quiz - how this was done?
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: victorwol on November 04, 2012, 12:42:42 PM
No clue, but would be nice if you can share it. It's very interesting for sure.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: neuroanatomist on November 04, 2012, 01:03:46 PM
The suspense isn't killing me.   ::)  But feel free to share, when you're ready...
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Viggo on November 04, 2012, 01:13:29 PM
Bring it on!  :D
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: nightbreath on November 04, 2012, 02:19:17 PM
Resizing down to 8Mpx file dimensions?  ;D
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 04, 2012, 02:34:07 PM
Resizing down to 8Mpx file dimensions?  ;D

No, full sensor resolution )))
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 04, 2012, 02:42:59 PM
OK, no magic - just "feeling" what is behind numbers )))

But seriously  this method used for decades for filtering noise out in different applications.
This is use of simple math in real life use.
Noise is random - image is fixed.
So what is required is just to integrate information element for a longer time and random noise would be filtered out.
  I believe that  this is something that  also was being used in astronomy to separate low level image details that could be even below random noise level.  Also used in communication systems.
=========
So how this is done with 1Dx ???
------
1Dx has one feature which is  intended for some artistic work but it could be used for something completely different.
This feature is multi-exposure – in one of the posts I mentioned about this for filtering out noise.
You can take up to 9 shots and 1Dx will average these shots into one.
 As each shot has it's own random noise distribution then averaging 9 shots into one would  essentially suppress noise - only fixed elements will  be kept - so random electronic noise is significantly reduced but fixed patterns - real image or sensor defects are kept and cleared out of noise. This is equivalent  to 9 times of longer integration time of the information element in the image detector.

How to  do this in 1Dx:
Enable  multi-exposure continuous shooting, number of exposures to 9, multi-exposure control to  Average (to integrate over 9 images) , put mirror lock up with the set button release  to reduce camera vibration (will be returned back by set button) , put camera in high speed continuous shooting mode, press shutter - mirror will be locked, press shutter again and keep it pressed - camera will take quickly 9 images in less than 1 sec , stops and then merge it ( integrate)  into one image.
And you are done.
Now press set button to return mirror back and then you could see the result.
So nothing special - just to see things a bit deeper)))
Method is suitable for only still images .
In normal conditions such kind of shot will take about 0.75 sec with FPS =12
And of course could be only shot using strong stable tripod for camera.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: nightbreath on November 04, 2012, 02:59:48 PM
Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: neuroanatomist on November 04, 2012, 03:18:36 PM
Thought that might be the 'trick'. I use it all the time in photomicroscopy.  Works great for test charts and fixed tissue.  Not so well with anything that moves, which limits its utility for >95% of what I shoot...
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 04, 2012, 03:41:42 PM
This is routinely applied in astrophotograpy if combined with precision German Equatorial Mounts. It is also used when combining many video frames into one photo (see registax). But I doubt it can be used elsewhere. You have to exclude leaves, people, water, clouds maybe and I am sure I haven't thought of everything...
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: pj1974 on November 04, 2012, 04:34:22 PM
Hi Neutral, thanks for sharing the technique and outcomes.... interesting (& understandable).

The only thing is, that the final output file you show, has significantly more noise and vertical banding than the base file "1DX_ISO100_EV0-Crop200%.JPG".

The file 1DX_ISO100_EV0-Crop200%.JPG is the cleanest for me... in fact very good for a 200% crop.

At least, that is what displays on my PC & Monitor.  Anyone else?

Paul
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: PVS on November 04, 2012, 04:38:40 PM
@OP - if you're that desperate to pull +3EV in shadows you could use split-toning action in PShop with (obviously) using blurred layer to pull those shadows up.

here's a dload link> http://www.mediafire.com/?po4x8dcgg5msr6a (http://www.mediafire.com/?po4x8dcgg5msr6a) for that action, let me know if it works for ya.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: RLPhoto on November 04, 2012, 06:45:15 PM
The d800 currently has the best 35mm sensor ever made. We all know this.

The 1Dx currently the best professional camera ever made. We all know this.

Two tools for two uses.

Canon could use some newer tech on the sensor base, and Nikon currently could use a newer AF system from the 3500 series. If you nail your exposures on either camera, only good photos will result.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Tammy on November 05, 2012, 01:38:40 AM
it's also the same method used in the "handheld night shooting" mode in the latest compact cameras, first introduced by Sony..
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 05, 2012, 10:14:41 AM
Hi Neutral, thanks for sharing the technique and outcomes.... interesting (& understandable).

The only thing is, that the final output file you show, has significantly more noise and vertical banding than the base file "1DX_ISO100_EV0-Crop200%.JPG".

The file 1DX_ISO100_EV0-Crop200%.JPG is the cleanest for me... in fact very good for a 200% crop.

At least, that is what displays on my PC & Monitor.  Anyone else?

Paul
As I mentioned above this method gives about 2 stops of visible SNR improvements, not full 3 stops and this above is "stress" test to see  camera limits and what could be dome with that.
  So definitely 0EV ISO100 image IQ will still be better than -3EV image pulled up 3 stops using this shooting method.
But even this is significant improvement.
 At 100% crop this difference is almost invisible and for normal shots with 0EV it gives enough room for shadows details recovery - almost full 2 stops
Just pull shadows up to the level where banding pattern is just below black point and get almost perfect image
Vertical banding:
 As I mentioned - you can filter down only random noise thus pulling fixed details up from the random noise. Low amplitude fixed image details becomes visible as they not any more masked out by random noise. Image banding are fixed sensor imperfections which also filtered out from random noise and become more visible as well.
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop  - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 05, 2012, 10:16:10 AM
Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.
Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 05, 2012, 10:18:57 AM
@OP - if you're that desperate to pull +3EV in shadows you could use split-toning action in PShop with (obviously) using blurred layer to pull those shadows up.

here's a dload link> http://www.mediafire.com/?po4x8dcgg5msr6a (http://www.mediafire.com/?po4x8dcgg5msr6a) for that action, let me know if it works for ya.
---------------
Sorry, I am afraid that you are a bit confused about the whole subject.
---
1. Subject is not that "someone is desperate to pull up shadows to +3EV" )))
  Subject is that 1Dx has some limitations at low ISO range  and how to push limits of the camera  so that it could be more usable in some photography scenarios for fixed still images (e.g. evening/night city landscapes, some interiors shots etc) to get better IQ  - to make it close to Nikon D800 in this area.
---
2. Subject is not how to blur  noise at problematic image areas reducing both visible noise and also smearing fine image detail.
 Subject is just opposite - how to preserve that fine details when pulling up shadows by filtering noise out.
---
3. Subject is not about post processing - but how to get best possible image IQ directly out of camera which required minimum post processing thus saving extremely valuable time.
 As a matter of fact blurring noise (together with details) is the ancient method used when modern high quality NR S/W features were not available.
 I used that (blurring problematic areas) many years back with my first digital camera.
  Also no need to go to PS now for that - all could be done in LR. One just need to  have adequate amount of NR applied to whole image to keep fine details and then apply more NR and de-sharpening to the specific local areas using local brush tool.
  For me since I upgraded from LR2 to LR3 and then to LR4 I almost stopped using PS - only for something very special.  For 99% of shots I can do all required with LR4 or DXO Optic Pro (ver. 8 now) or with Phase One  C1 when I need to see highest rendering IQ  and to work selectively with colors  - very useful with local adjustment tools which are  also available in C1 - similar to LR.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 05, 2012, 10:20:30 AM
This is routinely applied in astrophotograpy if combined with precision German Equatorial Mounts. It is also used when combining many video frames into one photo (see registax). But I doubt it can be used elsewhere. You have to exclude leaves, people, water, clouds maybe and I am sure I haven't thought of everything...
Definitely this is not applicable to everything.
Only applicable for fixed still images and one  need to use steady tripod for that.
But for night/evening city shots - this is something that really helps - you can recover fine  details in very deep shadows.
As for water – this is 50-50 - this could be used as substitution for ND filter to capture water motion instead of freezing it - e.g. shooting waterfalls.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 05, 2012, 10:30:44 AM
  In order to save time on explanations I just searched internet  for “averaging noise” and found very nice article – exactly about the same subject where everything explained to full extent.
http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm (http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm)
So here is also typical photography application to for this - which I was referring in my posts.
Luckily it could be easily done with 1Dx using method I described above.
------------
And now to scientific numbers)))
  From article above :  “ In general, magnitude of noise fluctuation drops by the square root of the number of images averaged, so you need to average 4 images in order to cut the magnitude in half”
  So averaging 9 photos reduce noise fluctuations 3 times which is equivalent to 10*log(Sqrt(9)) = 4.7dB.
  Look at DXOMark SNR curves for 1Dx and add 4.7 db to that at the flat left shoulder part at low ISOs and compare that to Nikon D800.
  Does not look  too bad after all that.
Not sure if D800 has the same multi-exposure mode. 
If yes then it could get even more amazing results.
As for banding – I believe it could be easily removed as well:
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: nightbreath on November 05, 2012, 03:10:22 PM
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.
I heard sensor heat correlates with sensor banding, so it's not that simple  :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: nightbreath on November 05, 2012, 03:29:31 PM
Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.
Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling
I didn't find the message with sample images that lacked visible noise, but here is another message about the same thing:

http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent (http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent)

the techradar test:
Quote
TIFF images (after conversion from raw) have a consistently good signal to noise ratio across the sensitivity range comparing closely to the Canon EOS 1Ds Mk III and just having the edge over the Nikon D800. It also shows a slight improvement at all sensitivities over the Canon EOS 5D Mk II.
Quote
TIFF files (after conversion from raw) have a high dynamic range with results comparing closely to the Nikon D4 and D800. Compared with the Canon EOS 5D Mk II, there is a marked improvement showing over 2EV greater range at the lower end of the sensitivity scale.
i thought techradar is a DxO certified lab?   ???
http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588 (http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 06, 2012, 09:55:12 AM
Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.
Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling
I didn't find the message with sample images that lacked visible noise, but here is another message about the same thing:

http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent (http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent)

the techradar test:
Quote
TIFF images (after conversion from raw) have a consistently good signal to noise ratio across the sensitivity range comparing closely to the Canon EOS 1Ds Mk III and just having the edge over the Nikon D800. It also shows a slight improvement at all sensitivities over the Canon EOS 5D Mk II.
Quote
TIFF files (after conversion from raw) have a high dynamic range with results comparing closely to the Nikon D4 and D800. Compared with the Canon EOS 5D Mk II, there is a marked improvement showing over 2EV greater range at the lower end of the sensitivity scale.
i thought techradar is a DxO certified lab?   ???
http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588 (http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588)

I do not know if TechRadar  is certified DXO lab, I think they just use some DXO equipment but how they use  that and how they interpret results I do not know. Have seen that article before but did not trust much  what they were publishing .
 I more rely on pure Math - all the time in my professional life that was first stage in any systems development.  And then implementation results were compared against initial math results to see if design was OK or something was missed or done wrong - initially or at later stage.
 So as you suggested I did noise comparison for original RAW and  TIFF (lossless converted from RAW, to16 bit TIFF, color space is the same Adobe RGB, no re-sampling - the same pixel count).

 Noise measurements were done using Noise Ninja plug-in PS which gives NUMERIC noise profile of the image.
For initial RAW noise with shadows raised by +3EV   index is 56 (Luminance=15, Chroma=41)
For RAW converted to TIFF and then shadows raised by +3EV noise  index is 54 (Luminance=15 and Chroma=39).
As expected these are same results for  lossless conversion  with normal statistical distribution as Noise Ninja was selecting areas for noise profiling automatically and they a bit different.
See below results two 200% crops for RAW and TIFF and that snapshots of NJ noise profiler screen
Anyone who has NJ plug-in in PS can do the same and see numeric results

Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 06, 2012, 10:01:51 AM
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.
I heard sensor heat correlates with sensor banding, so it's not that simple  :)

I am sure you are right with this and it depends on sensor heat.
So it would be not possible totally eliminate banding only to suppres it to some extent.
To improve results it would  required to do calibration shot just before shooting session - similar what is done with Gray/wite/black card before session for later WB adjustement
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: sanj on November 06, 2012, 11:40:12 AM
Thank you!
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 06, 2012, 12:05:05 PM
Thank you!
Appreciate that )))
   On of my favorite TV programms is Discovery Channel  and one of themes there is  Myth Breakers .
They take any Myth from anywhere  and do research to see if it right or no more than fiction.
Very funny)))
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 06, 2012, 12:09:15 PM
  If anyone is interested I can publish  Noise Ninja  noise profile results for original 0EV image , and for shot with noise averaging in 1Dx  (the same  multi-exposure shot  with-3EV pulled up 3EV in LR  which was shown initially – there was no post processing NR applied to it at all) .
 It was interesting to see and compare noise profiles numeric results for both.
 Also how shot with noise averaging looks after LR NR and  after  Noise Ninja NR applied in PS – interesting to see the difference
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: V8Beast on November 06, 2012, 12:16:00 PM
If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: rpt on November 06, 2012, 12:24:42 PM
If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times :)
:)

But seriously, the best shot ever is with the lens cap on. It does not even have to focus... Or need IS. And that is not the definition of IS Neuro had a few threads back!

Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: V8Beast on November 06, 2012, 12:30:59 PM


But seriously, the best shot ever is with the lens cap on. It does not even have to focus... Or need IS. And that is not the definition of IS Neuro had a few threads back!

Yes, I am a huge fan of lens-cap-on photography. It makes my $100 lens work just as well as a $14,000 Canon 800mm f/5.6L.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 06, 2012, 12:51:41 PM
If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times :)

  That all above is  not about the fun of making underexposed shots and pulling them up but about  normal 0EV shots for high DR scenes  and where it is desired  to see details in deep shadows instead of noise or instead of totally black areas.   And how to push 1Dx limits in this area.
   If one is trying to make fun of something  and this one does not fully realize what is the whole subject about  then yes  this making people to smile but not about the joke itself  but rather about the person who tried to make such joke))
Have fun  )))
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: RLPhoto on November 06, 2012, 12:53:55 PM
If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times :)

  That all above is  not about the fun of making underexposed shots and pulling them up but about  normal 0EV shots for high DR scenes  and where it is desired  to see details in deep shadows instead of noise or instead of totally black areas.   And how to push 1Dx limits in this area.
   If one is trying to make fun of something  and this one does not fully realize what is the whole subject about  then yes  this making people to smile but not about the joke itself  but rather about the person who tried to make such joke))
Have fun  )))

I love quotes taken out of context.   ::)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: MarkII on November 06, 2012, 01:20:56 PM
To improve results it would  required to do calibration shot just before shooting session - similar what is done with Gray/wite/black card before session for later WB adjustement
It is not quite constant. I did some experiments merging hundreds of images (don't ask), and the areas where banding seems to occur are clustered but not constant (over periods of a few hours).

However, if your exposure is more than 1s, turn on long exposure noise reduction. On my 5DIII at least, this removes most of the banding. The downside is that the general noise level rises (which also helps hide the banding), so you will need to average several such frames to get the noise back down again.

Another thing that seems to work well with multi-exposures is for images that need contrast stretching. For example, if you have an image with very low contrast (fog), combining multiple frames (eg in PS) can give you more latitude for  contrast adjustment.

In general though, if shadow noise is a problem, you are better off shooting multiple exposures, as for HDR. With a three-shot bracket, you can extend the usable dynamic range by six stops. To do the same by averaging frames you would need an awful lot more - and if the pattern noise is correlated the result will be worse.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: nightbreath on November 06, 2012, 03:03:17 PM
So as you suggested I did noise comparison for original RAW and  TIFF (lossless converted from RAW, to16 bit TIFF, color space is the same Adobe RGB, no re-sampling - the same pixel count).
Quick question. Did you create TIFF image via DPP as opposed by this method?
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: V8Beast on November 06, 2012, 03:08:53 PM
If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times :)

  That all above is  not about the fun of making underexposed shots and pulling them up but about  normal 0EV shots for high DR scenes  and where it is desired  to see details in deep shadows instead of noise or instead of totally black areas.   And how to push 1Dx limits in this area.
   If one is trying to make fun of something  and this one does not fully realize what is the whole subject about  then yes  this making people to smile but not about the joke itself  but rather about the person who tried to make such joke))
Have fun  )))

The joke is that you can't figure out that I quoted RL Photo, not you. Therefore, my response was directed at RL Photo's, not yours.

For future reference, when a post reads "Quote from: RL Photo," unless your user name is RL Photo, the comments in it aren't something for you to get overly defensive about. Failing to understand something this simple is funny, and I didn't even have to make a joke about it :) Thanks for making my job easier :)


Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: arcanej on November 06, 2012, 05:13:14 PM
After the election today, can we have a four year moratorium on posts extolling or dumping-on a camera based on pictures that are 3+ stops underexposed?
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: sandymandy on November 07, 2012, 09:25:03 AM
evening/night  city shots

Who is gonna do low light shots with ISO 100? Seriously? TBH i think people with an 1DX ususally dont f___ up their photos so badly like -3EV by accident.

@ "cap on lens photography"

check this guy:

How to NOT use a Leica (http://www.youtube.com/watch?v=X5IYahsPK2k#ws)

:D
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: neuroanatomist on November 07, 2012, 09:32:50 AM
evening/night  city shots
Who is gonna do low light shots with ISO 100? Seriously?

Mad dogs and Norsemen?   ::)

Oh, and me...

(http://farm6.staticflickr.com/5171/5515466348_cab556dcf6_z.jpg) (http://www.flickr.com/photos/dr_brain/5515466348/in/set-72157627187589316/lightbox/)
EOS 5D Mark II, TS-E 24mm f/3.5L II, 6 s, f/8, ISO 100

(http://farm6.staticflickr.com/5051/5514545687_9370f22b77_z.jpg) (http://www.flickr.com/photos/dr_brain/5514545687/in/set-72157627063033457/lightbox/)
EOS 5D Mark II, EF 35mm f/1.4L USM, 2.5 s, f/5.6, ISO 100
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 07, 2012, 11:40:31 AM
To improve results it would  required to do calibration shot just before shooting session - similar what is done with Gray/wite/black card before session for later WB adjustement
It is not quite constant. I did some experiments merging hundreds of images (don't ask), and the areas where banding seems to occur are clustered but not constant (over periods of a few hours).

However, if your exposure is more than 1s, turn on long exposure noise reduction. On my 5DIII at least, this removes most of the banding. The downside is that the general noise level rises (which also helps hide the banding), so you will need to average several such frames to get the noise back down again.

Another thing that seems to work well with multi-exposures is for images that need contrast stretching. For example, if you have an image with very low contrast (fog), combining multiple frames (eg in PS) can give you more latitude for  contrast adjustment.

In general though, if shadow noise is a problem, you are better off shooting multiple exposures, as for HDR. With a three-shot bracket, you can extend the usable dynamic range by six stops. To do the same by averaging frames you would need an awful lot more - and if the pattern noise is correlated the result will be worse.
Thanks, interesting info about banding.
Also I agree, one could use HDR to increase DR.
And I was using that occasionally (in Photomatix or PS) but did not like that much for reasons listed below.
Of course if 1Dx could do in-camera HDR similar to 5DMIII than this would be very nice.
One of the problems with HDR is that this requires careful post-processing which is time consuming.
And “time is money” – very valuable thing.
 Also known problem with HDR is that though it allows to increase image DR significantly it also increases image noise which becomes more visible.  So one of the practices for HDR is to do NR on each image before merging them into final HDR. This also additional time in post processing
 My goal was to try to get out of 1Dx cleanest image that practically does not require any post processing .    Almost 2 stops SNR improvement for shadow areas (actually 4.7 dB)  is sufficient for many if not most of  high DR evening/night shots.   And all that with just one button press ( or two if you want to have mirror lock to reduce camera vibrations to get best possible resolution).
  And if a little bit of NR applied then you can get very clean image with good details and low noise level  in deep shadows.   
  As for long exposure noise reduction – sure - this is very useful feature. 
I do not know how Canon implementing this but in my first digital camera (Sony DSC-828) it was done by subtracting pure noise shot ( sensor read with shutter closed ) from image done after that.
And that was working very well.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 07, 2012, 11:52:22 AM
So as you suggested I did noise comparison for original RAW and  TIFF (lossless converted from RAW, to16 bit TIFF, color space is the same Adobe RGB, no re-sampling - the same pixel count).
Quick question. Did you create TIFF image via DPP as opposed by this method?
No DPP,  just lossless TIFF export from LR.
   I removed DPP from my PC long long time back as what I have (LR, DXO Optic Pro and Phase One Capture One)  together give me all functionality and quality I need. Use one of them which is better suited for what I want to do at the moment - but with latest LR improvement use other two less and less/
Mathematically any lossless conversion from RAW to TIFF should not change any image metrics – this is why it called lossless.  If it changes something then it is not lossless – it either changing something or distort something. Change in noise level only possible if high special frequencies are suppressed – e.g. with down sampling which introduce some image info losses.
I can try to export RAW to TIFF in C1 but 100% sure will have the same result.
So I do not believe that converting RAW to TIFF does any image SNR improvements –just from pure math angle of view.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 07, 2012, 11:58:50 AM
evening/night  city shots
Who is gonna do low light shots with ISO 100? Seriously?

Mad dogs and Norsemen?   ::)

Oh, and me...

Nice pictures by the way.
Evening/night shots always carry on some special mood and feeling.
Also it seems that you have set of all the best lenses from Canon including TS ))))
Was always thinking of getting TS lens )))
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 07, 2012, 12:25:40 PM
Who is gonna do low light shots with ISO 100? Seriously?
Eh, hmmm, there is an accessory called tripod that facilitates this ::)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: neuroanatomist on November 07, 2012, 12:42:48 PM
Eh, hmmm, there is an accessory called tripod that facilitates this ::)

FWIW, that 35L shot above with the 2.5 s exposure was 'assisted' handheld, resting the camera on a bridge railing.   :D
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 07, 2012, 12:46:58 PM
Eh, hmmm, there is an accessory called tripod that facilitates this ::)

FWIW, that 35L shot above with the 2.5 s exposure was 'assisted' handheld, resting the camera on a bridge railing.   :D
I do remember this from another thread of yours  :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: nightbreath on November 07, 2012, 01:35:23 PM
So as you suggested I did noise comparison for original RAW and  TIFF (lossless converted from RAW, to16 bit TIFF, color space is the same Adobe RGB, no re-sampling - the same pixel count).
Quick question. Did you create TIFF image via DPP as opposed by this method?
No DPP,  just lossless TIFF export from LR.
   I removed DPP from my PC long long time back as what I have (LR, DXO Optic Pro and Phase One Capture One)  together give me all functionality and quality I need. Use one of them which is better suited for what I want to do at the moment - but with latest LR improvement use other two less and less/
Mathematically any lossless conversion from RAW to TIFF should not change any image metrics – this is why it called lossless.  If it changes something then it is not lossless – it either changing something or distort something. Change in noise level only possible if high special frequencies are suppressed – e.g. with down sampling which introduce some image info losses.
I can try to export RAW to TIFF in C1 but 100% sure will have the same result.
So I do not believe that converting RAW to TIFF does any image SNR improvements –just from pure math angle of view.
Canon software might be dealing with RAW image even better than the method you suggested. Just wanted to check  :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 10, 2012, 08:38:20 AM
  So far in this thread I was describing concept how 1DX performance could be significantly improved for low light high DR shots shadows at ISO100   - with some stress test shots showing to what extent (strength) this is applicable in real life and what SNR improvement we can get out of that   - just with a single output RAW file directly from the camera without no post processing we have additional 4.7dB of SNR improvement
OK, now to the final part of the subject – how all that works in real life combined together with long exposure NR that is applied by camera to every shot merged in final ME shot. I believe that that long exp NR is done by subtraction of calibration shot with shutter closed from the next normal shot thus eliminating banding patterns.

So now here is a real life example.

1. First shot is AS IS from the camera - properly exposed multi-exposure shot with -0.7EV shift to allow highlights to be recovered. Exposure shift was selected after few test shots to see what is the best EV
   But this is not what I see in reality and does not convey mood and feeling of what I see in real life.
Shot is dark, flat, no feeling of space and distance –as nothing seen on the foreground, reflections in the water not seen well – as water is a bit rough.  Shot just allows recognizing the place – no more than that.
-----------
2. Second image is original ME shot  which is processed to my feelings and of course a bit exaggerated to be a little bit more catchy.
   All was done within LR.   Local brush was used to raise EV and shadow in the foreground so something could be seen close to the camera to give feeling of distance and space and be close as I was seeing that.  The same for the sea surface to bring it from the dark.
Both above  are exported from LR4  as 800px sRGB  100% jpeg with preserved camera EXIF data so that anyone could check that these images are really from 1Dx and not from the medium format camera.
And now is most interesting part - two 200% crop snapshots from the LR screen using Win7 snip tool) to show image quality.
-------------
3.   200% scale  crop of the far city towers .  Please note absolute lack of noise in the sky – it is absolutely clean, transparent and it has natural “liquid” look.
Note that noise in shadows is also  not visible  but fine details are clean and clear. You even could notice two ropes between two towers – from the top of the blue tower going diagonally down towards white/yellow tower on the left.
 And this is from the distance of about 3km away from those towers!!!
 This is image quality which is close to the IQ that could be observed on medium format cameras.
For sharpness there are also several additional factors which are playing significant role in reaching such IQ in low light high DR environment:
- mirror is locked for all duration of multi- exposure sequence – released manually after completing the  last ME shot
 - amazing  sharpness and  micro-contrast of the new 24-70 F/2,8L II USM lens – my piece is just like unbelievable miracle  - I will post something related to that on 1DX body  in new thread some time later.
- averaging of 9 ME shots each is  2.5 sec  and few seconds interval between each ME shot  results  in almost 1 min total exposure and this effectively  filters out atmospheric  turbulence  which could be noticeable on single shot image for such distance.
--------------
4. Final is 200% of the crop of the foreground grass raised from absolute darkness.
   Note that there are no traces of any noise and banding here at all.
   This is not in lens focus – the same as I see it when look at far distance. 
    Foreground was raised from shadows to the level to be able to see something close in order to give feeling of the distance and space.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: neuroanatomist on November 10, 2012, 11:42:18 AM
Thanks, Neutral. Nice shot, and definitely improved by the post processing.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 10, 2012, 11:56:10 AM
Thanks, Neutral. Nice shot, and definitely improved by the post processing.
  And all that became possible (to have noise free shadows with all details) as result of SNR improvement using ME shot.  Basically there was a very little bit of post processing for that in LR.
Just few basic things.
  Using method described above it is possible to improve 1Dx low ISO SNR by 4.7db plus what is added by long exposure NR  ( I believe that s about 10*Log(SQRT(3)=1.5 dB.
So in total about 6 db SNR improvement directly from the camera single RAW file  without any post processing for strong NR and all that allow easily recover of shadows without need to spend time on  HDR.
I can post couple of more real life shots using this method a bit later – demonstrating the same virtually noise free recovered shadows.   Just single short HDR imagedirectly  out of camera – just pull down highlights and raise shadow in LR – this is  all what is required

Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 11, 2012, 01:27:49 PM
One more example of 1Dx ISO100  ME shot in action.
From series  “Another world dimensions “   -  through reflections in water.
Small snapshot from LR screen is shows level of noise
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 12, 2012, 09:45:43 AM
 Was checking  latest updates for new Sony DSC-RX1 which is expected mid december and found something very interesting.
Sony DCS-RX1  will be using the same ME noise averaging method  described above for two stops SNR improvement.

  Clever Sony engineers – they use whatever possible and available (both TECHNOLOGY and METHODS) to improve SNR performance and grabbing ideas directly from the air and putting all that in camera as standard features.
Seems that Sony DCS-RX1 could be very nice walk around camera...
---------------------- 
http://www.bhphotovideo.com/c/product/891105-REG/Sony_dscrx1_DSC_RX1_Full_Frame_Point.html/c/product/#inpage:IN+STOCK (http://www.bhphotovideo.com/c/product/891105-REG/Sony_dscrx1_DSC_RX1_Full_Frame_Point.html/c/product/#inpage:IN+STOCK)

“Low-light Performance
  The maximum selectable ISO setting is 25600, which enables low-light shooting without a flash. In addition, DSC-RX1 can capture six images in a fraction of a second and combine the data from all six to create a single image with a reduction in noise equivalent to two additional steps of ISO sensitivity.”
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: DrDeano on November 12, 2012, 11:08:44 PM
Tried this trick out tonight and it works a treat!

I never noticed how much noise was in the shadows at 100 ISO before. This method really helps bring out the details.

Thanks for sharing, I can't wait to use this more.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 13, 2012, 01:30:00 PM
Tried this trick out tonight and it works a treat!

I never noticed how much noise was in the shadows at 100 ISO before. This method really helps bring out the details.

Thanks for sharing, I can't wait to use this more.

My pleasure)))
Glad if that was useful to any of 1Dx users.
It would be nice if Canon could put that in firmware upgrade as single button press feature.
This is very simple to implement.
For me  to make it easy to use  I just registered ME  and Mirror lockup in the1Dx  custom menu (last tab).

And now one more real life comparison of 1DX single shot and ME shot using method described above.
  Two 200% crop with no noise reduction in LR showing how clean 1Dx  ISO100 ME shot  compared to the single shot.
 And also 1Dx ME shooting mode  performance shown on DxoMark  SNR measurement curves  with comparison to Nikon D800 and D4 single shot. (I do not know if they have the same ME shooting mode).
Hope that seeing all that could make many 1DX users more happier about the camera)))
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: slowcarIX on November 13, 2012, 08:17:53 PM

For me  to make it easy to use  I just registered ME  and Mirror lockup in the1Dx  custom menu (last tab).


Neutral: could you please give a step by step instruction how this can be done?

Thanks!!
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 13, 2012, 08:49:56 PM
Glad if that was useful to any of 1Dx users.
I guess you helped 5DMkIII users too as the Multiple exposure capabilities of the two cameras seem similar  ;D
So thank you on behalf of the 5DMkIII users too ...
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 13, 2012, 09:19:39 PM
Just  a question. You mentioned that you kept the mirror locked until the end of the sequence. Do you mean that the mirror was up the whole time ?Assuming that 5D3 and 1Dx are exactly the same in ME this cannot happen unless you are using LiveView. Unless you meant you had activated mirror locking and you deactivated it afterwards. But in that case the mirror moves up before and then down after every shot.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: sach100 on November 14, 2012, 12:24:23 AM
Great examples Neutral. Thanks. I think i should be able to try this on my 5d3.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 14, 2012, 10:28:28 AM
Glad if that was useful to any of 1Dx users.
I guess you helped 5DMkIII users too as the Multiple exposure capabilities of the two cameras seem similar  ;D
So thank you on behalf of the 5DMkIII users too ...
My pleasure again)))
At some point it is more pleasure to give people something useful for them rather then  get something for himself)))
There is also very interesting side-effect of this ME method.
Probably someone has already noticed that in the real life examples posted above.
This side effect is also extremely useful.
Because image is so clean (extremely low noise level) this allows applying significantly more aggressive sharpening to the image without any noticeable problems.
Just on two last examples above - with no noise reduction in LR sharpness slider is set to 40 and details to 100.
Single shot is practically unusable for big prints
And second ME shot is still clean and full of details (see the curtains in the left window).
With just small amount of NR applied (about 10 -15) and using properly set sharpen mask slider it is possible to move safely sharpness slider up  to 50-60  (also reducing radius  below 1 ) and get perfect per pixel sharpness  which is practically impossible using single shot in such high DR low light environment.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 14, 2012, 10:31:52 AM
Just  a question. You mentioned that you kept the mirror locked until the end of the sequence. Do you mean that the mirror was up the whole time ?Assuming that 5D3 and 1Dx are exactly the same in ME this cannot happen unless you are using LiveView. Unless you meant you had activated mirror locking and you deactivated it afterwards. But in that case the mirror moves up before and then down after every shot.
  Actually  I was activating mirror lock before ME sequence and then deactivating it afterwards - this was more convenient for me. To keep it up until sequence is completed it should be set to be released by set button instead of deactivating after each shot.
But I think live view mode also could be used for that.
Here is some more explanations for Mirror Lock:
As a matter of fact use of mirror lock for ME shot is not required but very desirable.
  My main goal was not only to improve 1DX SNR performance in high DR environment but also to get maximum possible image quality regarding sharpness and clarity - actually to squeeze whatever possible from 1Dx regarding IQ.
  So I activate mirror lock just  before ME shot sequence  (done automatically after camera focused on the subject)  and then deactivate it after ME shot sequence  (using SET button - selectable option in mirror lock function) or it is deactivated itself later by camera itself .
Below is the outcome of using Mirror Lock
1.      This reduces camera vibrations thus contributing to best image sharpness -nothing new in that.
2.      The other result of using mirror lock is that camera focuses once before ME sequence and does not try to refocus for each shot in ME sequence thus preventing losing sharpness as result that for each ME shot focus could be slightly different.  This is equivalent that you set manual focus for all sequence in ME shot.
  Of course if image sharpness is not required and only shadows SNR is the concern ( e.g. for small print) then ME shot could be done without Mirror Lock.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 14, 2012, 10:35:19 AM

For me  to make it easy to use  I just registered ME  and Mirror lockup in the1Dx  custom menu (last tab).


Neutral: could you please give a step by step instruction how this can be done?

Thanks!!

This is how to  do this in 1Dx:
1.      Enable multi-exposure
1.1. Set ME mode to continuous .
1.2.  Set number of exposures to 9  (or any between 4 an 9 - smaller number will give less SNR improvement)
1.3.  Set multi-exposure control to  Average (to integrate over 9 images) ,
2.      (Optional) Switch on  mirror lock up function with the SET button deactivation to reduce camera vibration (will be deactivated  by SET button).  See explanation for this in my post above

3.      Switch on Long exposure NR - set it to auto - will be used for shots longer than 1 sec or set it ON -"forced" .  This adds additional SNR improvements and significantly reduce banding

4.      Put camera in high speed continuous shooting mode  ( optional - if you are shooting in high DR daylight environment to make ME shot very fast) ,  press shutter - mirror will be locked, press shutter again  to start ME shot  and keep it  pressed for each next shot in ME sequence until ME sequence is completed
 Camera will take quickly 9 images in less than 1 sec (that for daylight - at night will be longer ), stops and then merge it ( integrate)  into one image.
And you are done.
Now press set button to return mirror back (if mirror lock was used)  and then you could see the result.
Method is suitable for only still images.
In normal (daylight) conditions such kind of shot will take about 0.75 sec with FPS =12
And of course method should be used with strong stable tripod for camera and remote shutter control (to prevent image blurring due to camera vibrations and get highest possible resolution)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: wickidwombat on November 14, 2012, 06:59:36 PM
Nice work, but does this do a better job than exposure bracketing and combining in LR enfuse?
perhaps doing both might be interesting using the ME process for several different exposures
that way you are combining 3 or 5 super clean files in enfuse to start with.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 14, 2012, 09:01:21 PM
So I activate mirror lock just  before ME shot sequence  (done automatically after camera focused on the subject)  and then deactivate it after ME shot sequence  (using SET button - selectable option in mirror lock function) or it is deactivated itself later by camera itself .
Unfortunately this is not supported by my 5DMkIII that's why I was wondering how you can do that. So it's only Live View for me  :(  (OK I know it is sufficient and actually a super set of Mirror Lock for this purpose but still ...)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: slowcarIX on November 14, 2012, 09:03:06 PM
Thank you!

Will single long exposure achieve the same effect keeping iso constant?

9 exposure with total integration of 2.5secs = smaller aperture and keeping shutter open for 2.5secs?


For me  to make it easy to use  I just registered ME  and Mirror lockup in the1Dx  custom menu (last tab).


Neutral: could you please give a step by step instruction how this can be done?

Thanks!!

This is how to  do this in 1Dx:
1.      Enable multi-exposure
1.1. Set ME mode to continuous .
1.2.  Set number of exposures to 9  (or any between 4 an 9 - smaller number will give less SNR improvement)
1.3.  Set multi-exposure control to  Average (to integrate over 9 images) ,
2.      (Optional) Switch on  mirror lock up function with the SET button deactivation to reduce camera vibration (will be deactivated  by SET button).  See explanation for this in my post above

3.      Switch on Long exposure NR - set it to auto - will be used for shots longer than 1 sec or set it ON -"forced" .  This adds additional SNR improvements and significantly reduce banding

4.      Put camera in high speed continuous shooting mode  ( optional - if you are shooting in high DR daylight environment to make ME shot very fast) ,  press shutter - mirror will be locked, press shutter again  to start ME shot  and keep it  pressed for each next shot in ME sequence until ME sequence is completed
 Camera will take quickly 9 images in less than 1 sec (that for daylight - at night will be longer ), stops and then merge it ( integrate)  into one image.
And you are done.
Now press set button to return mirror back (if mirror lock was used)  and then you could see the result.
Method is suitable for only still images.
In normal (daylight) conditions such kind of shot will take about 0.75 sec with FPS =12
And of course method should be used with strong stable tripod for camera and remote shutter control (to prevent image blurring due to camera vibrations and get highest possible resolution)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: weixing on November 14, 2012, 11:31:34 PM
Hi,
    This method is commonly use in Astrophotography to reduce noise (mainly to reduce random noise) and improve image quality.

    By the way, Panasonic had this feature built-in in some of their compact camera. 

   Have a nice day.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: rpt on November 14, 2012, 11:45:19 PM
Thank you!

Will single long exposure achieve the same effect keeping iso constant?

9 exposure with total integration of 2.5secs = smaller aperture and keeping shutter open for 2.5secs?


For me  to make it easy to use  I just registered ME  and Mirror lockup in the1Dx  custom menu (last tab).


Neutral: could you please give a step by step instruction how this can be done?

Thanks!!

This is how to  do this in 1Dx:
1.      Enable multi-exposure
1.1. Set ME mode to continuous .
1.2.  Set number of exposures to 9  (or any between 4 an 9 - smaller number will give less SNR improvement)
1.3.  Set multi-exposure control to  Average (to integrate over 9 images) ,
2.      (Optional) Switch on  mirror lock up function with the SET button deactivation to reduce camera vibration (will be deactivated  by SET button).  See explanation for this in my post above

3.      Switch on Long exposure NR - set it to auto - will be used for shots longer than 1 sec or set it ON -"forced" .  This adds additional SNR improvements and significantly reduce banding

4.      Put camera in high speed continuous shooting mode  ( optional - if you are shooting in high DR daylight environment to make ME shot very fast) ,  press shutter - mirror will be locked, press shutter again  to start ME shot  and keep it  pressed for each next shot in ME sequence until ME sequence is completed
 Camera will take quickly 9 images in less than 1 sec (that for daylight - at night will be longer ), stops and then merge it ( integrate)  into one image.
And you are done.
Now press set button to return mirror back (if mirror lock was used)  and then you could see the result.
Method is suitable for only still images.
In normal (daylight) conditions such kind of shot will take about 0.75 sec with FPS =12
And of course method should be used with strong stable tripod for camera and remote shutter control (to prevent image blurring due to camera vibrations and get highest possible resolution)
As I understand it, the pixel data will be averaged and not summed as that is what is selected in step 1.3. So my understanding is that the exposure should be sufficient to capture all the light for the subject. It is only in the dark areas that the random noise will be evened out.

Neutral, do correct me if I did not get it right.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 15, 2012, 10:30:07 AM
    This method is commonly use in Astrophotography to reduce noise (mainly to reduce random noise) and improve image quality...
This correct. And they could use thousands and thousands of images for averaging to extract details that far below noise level.   And have supercomputers to do this using sophisticated processing algorithms – not just simple averaging
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 15, 2012, 10:33:41 AM
Thank you!
Will single long exposure achieve the same effect keeping iso constant?
9 exposure with total integration of 2.5secs = smaller aperture and keeping shutter open for 2.5secs?
Unfortunately this will not help.
 Basically by doing so you will be just reducing amount of light per time unit to the sensor to keep exposure longer  to obtain the same amount of light. 
    Mutli-Exposure  method is different  - it allows to capture more light - up 9 times more for 1Dx (and 10 times more for  Nikon D800) than in single shot and then averaging that  (normalizing)  to one single shot image . But random noise is not increased by 9 times but only  by  Sqrt(9)=3
 So after averaging 9 shots in ME sequence  (normalizing it to one shot) we will have  10*log(9/sqrt(9))=10*log(3)= 4.7dB  image SNR improvements.

 On the link below you can find detailed explanations - I was referring this in one of my previous post .
Found it when was searching for something that could answer all the questions.
http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm (http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm)
 " In general, magnitude of noise<http://www.cambridgeincolour.com/tutorials/image-noise.htm> fluctuation drops by the square root of the number of images averaged, so you need to average 4 images in order to cut the magnitude in half."
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 15, 2012, 10:36:51 AM
  As I understand it, the pixel data will be averaged and not summed as that is what is selected in step 1.3. So my understanding is that the exposure should be sufficient to capture all the light for the subject. It is only in the dark areas that the random noise will be evened out.

Neutral, do correct me if I did not get it right.
Averaging is summing up all the values and then dividing result by number of samples.
Mathematically averaging is (A1+A2+A3 +...+AN)/N where N is number of samples.
  By using ME method Image SNR will be increased for every pixel on the image - regardless if this shadows or highlights where SNR is already high and will be even better.
As result of cleaner image you can use more aggressive sharpening without any visible problems on the image
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: rpt on November 15, 2012, 10:46:33 AM
  As I understand it, the pixel data will be averaged and not summed as that is what is selected in step 1.3. So my understanding is that the exposure should be sufficient to capture all the light for the subject. It is only in the dark areas that the random noise will be evened out.

Neutral, do correct me if I did not get it right.
Averaging is summing up all the values and then dividing result by number of samples.
Mathematically averaging is (A1+A2+A3 +...+AN)/N where N is number of samples.
  By using ME method Image SNR will be increased for every pixel on the image - regardless if this shadows or highlights where SNR is already high and will be even better.
As result of cleaner image you can use more aggressive sharpening without any visible problems on the image
You are right. The noise will be addressed for each pixel. It will be more noticeable in the shadows and darker areas of the image.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 15, 2012, 10:48:29 AM
So I activate mirror lock just  before ME shot sequence  (done automatically after camera focused on the subject)  and then deactivate it after ME shot sequence  (using SET button - selectable option in mirror lock function) or it is deactivated itself later by camera itself .
Unfortunately this is not supported by my 5DMkIII that's why I was wondering how you can do that. So it's only Live View for me  :(  (OK I know it is sufficient and actually a super set of Mirror Lock for this purpose but still ...)
  Actually mirror lock is not required – just an option to get better image sharpness – reducing camera vibrations and preventing refocusing for each ME shot in sequence.
If for first one nothing could be done with 5DM3 but for second one it is possible to focus automatically then switch lens to manual focus to keep focus unchanged during  ME sequence
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: rpt on November 15, 2012, 12:14:37 PM
Ok, I just tried it. Set it for 9 exposures. Set it on live view. After the first shot, I set the AF off and took the next 8. The noise reduction was significant. I shot decorative lights in the garden. The iso was 25600.
Yay!

Thanks Neutral :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: rpt on November 15, 2012, 01:11:39 PM
Just tried another with iso 100. I am happy. No NR needed in LR :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: tron on November 15, 2012, 06:02:19 PM
So I activate mirror lock just  before ME shot sequence  (done automatically after camera focused on the subject)  and then deactivate it after ME shot sequence  (using SET button - selectable option in mirror lock function) or it is deactivated itself later by camera itself .
Unfortunately this is not supported by my 5DMkIII that's why I was wondering how you can do that. So it's only Live View for me  :(  (OK I know it is sufficient and actually a super set of Mirror Lock for this purpose but still ...)
  Actually mirror lock is not required – just an option to get better image sharpness – reducing camera vibrations and preventing refocusing for each ME shot in sequence.
If for first one nothing could be done with 5DM3 but for second one it is possible to focus automatically then switch lens to manual focus to keep focus unchanged during  ME sequence
Mirror lock is required for sharpness. There is no point in trying to lower the noise merely to induce camera shake. But is is achieved with LiveView. Anyway for shots on tripod - when it is getting dark - I use Mirror Lock and 2 second delay. Focusing is either AF and then turned to manual or Live View (with manual focus and 10x). So the only additional function is the Multiple Exposure (Mirror is Locked in LV).
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: PeterD on November 16, 2012, 02:04:35 AM
Neutral - thanks for the in-depth explanation - I'm anxious to try out your ME method.

BTW - my 1DX is my first 1 series camera - I'm loving it but there is a learning - it's threads like yours that I find helpful - inspired me to register at canonrumors. Thanks again. :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 16, 2012, 02:19:55 AM
Just tried another with iso 100. I am happy. No NR needed in LR :)
One more happy user  :)
 That's nice and gives some good feeling to me  :)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 16, 2012, 02:30:00 AM
Neutral - thanks for the in-depth explanation - I'm anxious to try out your ME method.

BTW - my 1DX is my first 1 series camera - I'm loving it but there is a learning - it's threads like yours that I find helpful - inspired me to register at canonrumors. Thanks again. :)
Thanks for good words :)
My pleasure ))
 This is just illustration to what I was telling several times in some other posts – instead of useless arguments over subject(s)  that some most involved have no ideas or knowledge – better do something useful for others if you can do that )))
  And this was the reason I join forum recently – I got very tired of reading all that heated disputes about DXO sensor measurements  between people who do not understand that at all and wanted to show how  simple MATH is applied to real life.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: wickidwombat on November 16, 2012, 02:32:37 AM
Neutral have you tried mixing this with other HDR techniques? say -2EV and +2EV then using enfuse to blend
so a total of 27 shots? 3 combined super clean files blended together I think there is lots of potential with adapting this technique. Thanks for the find I cant wait to give it a go
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 16, 2012, 02:32:56 AM
So I activate mirror lock just  before ME shot sequence  (done automatically after camera focused on the subject)  and then deactivate it after ME shot sequence  (using SET button - selectable option in mirror lock function) or it is deactivated itself later by camera itself .
Unfortunately this is not supported by my 5DMkIII that's why I was wondering how you can do that. So it's only Live View for me  :(  (OK I know it is sufficient and actually a super set of Mirror Lock for this purpose but still ...)
  Actually mirror lock is not required – just an option to get better image sharpness – reducing camera vibrations and preventing refocusing for each ME shot in sequence.
If for first one nothing could be done with 5DM3 but for second one it is possible to focus automatically then switch lens to manual focus to keep focus unchanged during  ME sequence
Mirror lock is required for sharpness. There is no point in trying to lower the noise merely to induce camera shake. But is is achieved with LiveView. Anyway for shots on tripod - when it is getting dark - I use Mirror Lock and 2 second delay. Focusing is either AF and then turned to manual or Live View (with manual focus and 10x). So the only additional function is the Multiple Exposure (Mirror is Locked in LV).
I could not imagine that 5DM3 does not have similar as 1Dx option to keep mirror locked up for 30 seconds with early deactivation by SET button.
That’s good that you found a way for 5DM3 to keep mirror locked during ME sequence.
With that 5DM3 could have an edge over 1Dx for such kind of ME night/evening shots due to better sensor resolution.
   The only one thing I do not understand why Canon did not  do that as F/W feature with single button press as done in some other cameras (e.g. upcoming Sony DCS-RX1).  This is so simple to add that in F/W to give user easy way to cope with deep shadow noise issues at ISO100 for 1Dx and 5DM3.  They know the problem they know (I hope) how to deal with that so why not to make life easier to camera users who like night/evening shooting.
  It would be also interesting to see outcome of this noise averaging method on Nikon D800.
D800 could have up to 10 shots in ME sequence.  Combined with camera 36mpx sensor resolution and high DR outcome using this method could be really amazing (with good Hi-Res lens)
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 16, 2012, 02:39:39 AM
Neutral have you tried mixing this with other HDR techniques? say -2EV and +2EV then using enfuse to blend
so a total of 27 shots? 3 combined super clean files blended together I think there is lots of potential with adapting this technique. Thanks for the find I cant wait to give it a go
I did not try that  but I think this could be very interesting.
Known issue for HDR is increasing image noise.
So having virtually noise free images from camera using ME could help get cleaner and better HDR images. When I look at some HDR images posted around I frequently want to take wet tissue and wipe that noise away as this is causing pain for my eyes)))
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 16, 2012, 03:35:59 AM
Neutral - thanks for the in-depth explanation - I'm anxious to try out your ME method...
As a matter of fact this is not my method  :)

This technique  was being used around for decades in different areas.

  I just was not very happy with 1Dx low ISO SNR performance  (as I like night/evening shooting)  and when saw that ME function in 1Dx – my first thought was that this could be workaround until we get new rumored Hi-Res camera that should bring some new sensor technologies.  I was tempting to buy Nikon D800 but thought that I can wait and do something with 1Dx.   

 Actually Multi-Exposure term is coming from old film times when in order  to mix together several  images you had to do multiple exposure from different frames on the same piece of photo paper.
Same functionality was main goal of introducing ME in digital cameras – and one of the application of this to catch phases of object movement in one shot - but there are also many other interesting artistic applications for that

Noise averaging using ME for improving SNR is just very useful side effect of this functionality.

And the other side effect which is also very useful -  is to simulate ND filter for long exposure.
Using ME method one will be getting much cleaner images than using ND filter to reduce amount of light into camera sensor.
If both are combined together (ND filter and ME method) you can get very interesting artistic effects due to ability to have very long exposures at day time.  I did not tried that yet but going to do so at some time.
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: wickidwombat on November 16, 2012, 06:52:30 AM
Neutral have you tried mixing this with other HDR techniques? say -2EV and +2EV then using enfuse to blend
so a total of 27 shots? 3 combined super clean files blended together I think there is lots of potential with adapting this technique. Thanks for the find I cant wait to give it a go
I did not try that  but I think this could be very interesting.
Known issue for HDR is increasing image noise.
So having virtually noise free images from camera using ME could help get cleaner and better HDR images. When I look at some HDR images posted around I frequently want to take wet tissue and wipe that noise away as this is causing pain for my eyes)))
Lr enfuse doesn't make eye rending abominations like photomatix it takes the exposures and blends them so no noise gain anyway. I suspect in combination with your method it will enable some super clean files
Title: Re: 1Dx ISO100 high DR shadows SNR improvement
Post by: Neutral on November 16, 2012, 10:12:21 AM
Neutral have you tried mixing this with other HDR techniques? say -2EV and +2EV then using enfuse to blend
so a total of 27 shots? 3 combined super clean files blended together I think there is lots of potential with adapting this technique. Thanks for the find I cant wait to give it a go
I did not try that  but I think this could be very interesting.
Known issue for HDR is increasing image noise.
So having virtually noise free images from camera using ME could help get cleaner and better HDR images. When I look at some HDR images posted around I frequently want to take wet tissue and wipe that noise away as this is causing pain for my eyes)))
Lr enfuse doesn't make eye rending abominations like photomatix it takes the exposures and blends them so no noise gain anyway. I suspect in combination with your method it will enable some super clean files
I think this will definitely work as expected.
Will try that as well when have chance to do so.