Messages

31

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 07, 2012, 11:40:31 AM »

To improve results it would  required to do calibration shot just before shooting session - similar what is done with Gray/wite/black card before session for later WB adjustement

It is not quite constant. I did some experiments merging hundreds of images (don't ask), and the areas where banding seems to occur are clustered but not constant (over periods of a few hours).

However, if your exposure is more than 1s, turn on long exposure noise reduction. On my 5DIII at least, this removes most of the banding. The downside is that the general noise level rises (which also helps hide the banding), so you will need to average several such frames to get the noise back down again.

Another thing that seems to work well with multi-exposures is for images that need contrast stretching. For example, if you have an image with very low contrast (fog), combining multiple frames (eg in PS) can give you more latitude for  contrast adjustment.

In general though, if shadow noise is a problem, you are better off shooting multiple exposures, as for HDR. With a three-shot bracket, you can extend the usable dynamic range by six stops. To do the same by averaging frames you would need an awful lot more - and if the pattern noise is correlated the result will be worse.

Thanks, interesting info about banding.
Also I agree, one could use HDR to increase DR.
And I was using that occasionally (in Photomatix or PS) but did not like that much for reasons listed below.
Of course if 1Dx could do in-camera HDR similar to 5DMIII than this would be very nice.
One of the problems with HDR is that this requires careful post-processing which is time consuming.
And “time is money” – very valuable thing.
 Also known problem with HDR is that though it allows to increase image DR significantly it also increases image noise which becomes more visible.  So one of the practices for HDR is to do NR on each image before merging them into final HDR. This also additional time in post processing
 My goal was to try to get out of 1Dx cleanest image that practically does not require any post processing .    Almost 2 stops SNR improvement for shadow areas (actually 4.7 dB)  is sufficient for many if not most of  high DR evening/night shots.   And all that with just one button press ( or two if you want to have mirror lock to reduce camera vibrations to get best possible resolution).
  And if a little bit of NR applied then you can get very clean image with good details and low noise level  in deep shadows.   
  As for long exposure noise reduction – sure - this is very useful feature. 
I do not know how Canon implementing this but in my first digital camera (Sony DSC-828) it was done by subtracting pure noise shot ( sensor read with shutter closed ) from image done after that.
And that was working very well.

32

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 06, 2012, 12:51:41 PM »

If you nail your exposures on either camera, only good photos will result.

.....but why would you do this when it's so much fun to intentionally underexpose an image by 5 stops then crank on the sliders in Lightroom. Yeeehawww, keep pushing those sliders over to the right!

Good times

  That all above is  not about the fun of making underexposed shots and pulling them up but about  normal 0EV shots for high DR scenes  and where it is desired  to see details in deep shadows instead of noise or instead of totally black areas.   And how to push 1Dx limits in this area.
   If one is trying to make fun of something  and this one does not fully realize what is the whole subject about  then yes  this making people to smile but not about the joke itself  but rather about the person who tried to make such joke))
Have fun  )))

33

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 06, 2012, 12:09:15 PM »

  If anyone is interested I can publish  Noise Ninja  noise profile results for original 0EV image , and for shot with noise averaging in 1Dx  (the same  multi-exposure shot  with-3EV pulled up 3EV in LR  which was shown initially – there was no post processing NR applied to it at all) .
 It was interesting to see and compare noise profiles numeric results for both.
 Also how shot with noise averaging looks after LR NR and  after  Noise Ninja NR applied in PS – interesting to see the difference

34

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 06, 2012, 12:05:05 PM »

Thank you!

Appreciate that )))
   On of my favorite TV programms is Discovery Channel  and one of themes there is  Myth Breakers .
They take any Myth from anywhere  and do research to see if it right or no more than fiction.
Very funny)))

35

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 06, 2012, 10:01:51 AM »

I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

I heard sensor heat correlates with sensor banding, so it's not that simple 

I am sure you are right with this and it depends on sensor heat.
So it would be not possible totally eliminate banding only to suppres it to some extent.
To improve results it would  required to do calibration shot just before shooting session - similar what is done with Gray/wite/black card before session for later WB adjustement

36

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 06, 2012, 09:55:12 AM »

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling

I didn't find the message with sample images that lacked visible noise, but here is another message about the same thing:

http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-5d-mark-iii-1074186/review/page:5#articleContent

the techradar test:

TIFF images (after conversion from raw) have a consistently good signal to noise ratio across the sensitivity range comparing closely to the Canon EOS 1Ds Mk III and just having the edge over the Nikon D800. It also shows a slight improvement at all sensitivities over the Canon EOS 5D Mk II.

TIFF files (after conversion from raw) have a high dynamic range with results comparing closely to the Nikon D4 and D800. Compared with the Canon EOS 5D Mk II, there is a marked improvement showing over 2EV greater range at the lower end of the sensitivity scale.

i thought techradar is a DxO certified lab?   
http://www.techradar.com/news/photography-video-capture/cameras/noise-and-dynamic-range-results-explained-1027588

I do not know if TechRadar  is certified DXO lab, I think they just use some DXO equipment but how they use  that and how they interpret results I do not know. Have seen that article before but did not trust much  what they were publishing .
 I more rely on pure Math - all the time in my professional life that was first stage in any systems development.  And then implementation results were compared against initial math results to see if design was OK or something was missed or done wrong - initially or at later stage.
 So as you suggested I did noise comparison for original RAW and  TIFF (lossless converted from RAW, to16 bit TIFF, color space is the same Adobe RGB, no re-sampling - the same pixel count).

 Noise measurements were done using Noise Ninja plug-in PS which gives NUMERIC noise profile of the image.
For initial RAW noise with shadows raised by +3EV   index is 56 (Luminance=15, Chroma=41)
For RAW converted to TIFF and then shadows raised by +3EV noise  index is 54 (Luminance=15 and Chroma=39).
As expected these are same results for  lossless conversion  with normal statistical distribution as Noise Ninja was selecting areas for noise profiling automatically and they a bit different.
See below results two 200% crops for RAW and TIFF and that snapshots of NJ noise profiler screen
Anyone who has NJ plug-in in PS can do the same and see numeric results

37

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 05, 2012, 10:30:44 AM »

  In order to save time on explanations I just searched internet  for “averaging noise” and found very nice article – exactly about the same subject where everything explained to full extent.
http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm
So here is also typical photography application to for this - which I was referring in my posts.
Luckily it could be easily done with 1Dx using method I described above.
------------
And now to scientific numbers)))
  From article above :  “ In general, magnitude of noise fluctuation drops by the square root of the number of images averaged, so you need to average 4 images in order to cut the magnitude in half”
  So averaging 9 photos reduce noise fluctuations 3 times which is equivalent to 10*log(Sqrt(9)) = 4.7dB.
  Look at DXOMark SNR curves for 1Dx and add 4.7 db to that at the flat left shoulder part at low ISOs and compare that to Nikon D800.
  Does not look  too bad after all that.
Not sure if D800 has the same multi-exposure mode. 
If yes then it could get even more amazing results.
As for banding – I believe it could be easily removed as well:
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

38

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 05, 2012, 10:20:30 AM »

This is routinely applied in astrophotograpy if combined with precision German Equatorial Mounts. It is also used when combining many video frames into one photo (see registax). But I doubt it can be used elsewhere. You have to exclude leaves, people, water, clouds maybe and I am sure I haven't thought of everything...

Definitely this is not applicable to everything.
Only applicable for fixed still images and one  need to use steady tripod for that.
But for night/evening city shots - this is something that really helps - you can recover fine  details in very deep shadows.
As for water – this is 50-50 - this could be used as substitution for ND filter to capture water motion instead of freezing it - e.g. shooting waterfalls.

39

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 05, 2012, 10:18:57 AM »

@OP - if you're that desperate to pull +3EV in shadows you could use split-toning action in PShop with (obviously) using blurred layer to pull those shadows up.

here's a dload link> http://www.mediafire.com/?po4x8dcgg5msr6a for that action, let me know if it works for ya.

---------------
Sorry, I am afraid that you are a bit confused about the whole subject.
---
1. Subject is not that "someone is desperate to pull up shadows to +3EV" )))
  Subject is that 1Dx has some limitations at low ISO range  and how to push limits of the camera  so that it could be more usable in some photography scenarios for fixed still images (e.g. evening/night city landscapes, some interiors shots etc) to get better IQ  - to make it close to Nikon D800 in this area.
---
2. Subject is not how to blur  noise at problematic image areas reducing both visible noise and also smearing fine image detail.
 Subject is just opposite - how to preserve that fine details when pulling up shadows by filtering noise out.
---
3. Subject is not about post processing - but how to get best possible image IQ directly out of camera which required minimum post processing thus saving extremely valuable time.
 As a matter of fact blurring noise (together with details) is the ancient method used when modern high quality NR S/W features were not available.
 I used that (blurring problematic areas) many years back with my first digital camera.
  Also no need to go to PS now for that - all could be done in LR. One just need to  have adequate amount of NR applied to whole image to keep fine details and then apply more NR and de-sharpening to the specific local areas using local brush tool.
  For me since I upgraded from LR2 to LR3 and then to LR4 I almost stopped using PS - only for something very special.  For 99% of shots I can do all required with LR4 or DXO Optic Pro (ver. 8 now) or with Phase One  C1 when I need to see highest rendering IQ  and to work selectively with colors  - very useful with local adjustment tools which are  also available in C1 - similar to LR.

40

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 05, 2012, 10:16:10 AM »

Could you do one thing just to test another approach? Please export the RAW file to TIFF and push the shadows in the resulted file. I saw very good results someone shared here using this method.

Mathematically this should not give any difference as noise is already in the image.
You might see improvements in SNR  a little bit but only if during this conversion from RAW to TIFF some small per pixel details are lost e.g due to down sampling

41

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 05, 2012, 10:14:41 AM »

Hi Neutral, thanks for sharing the technique and outcomes.... interesting (& understandable).

The only thing is, that the final output file you show, has significantly more noise and vertical banding than the base file "1DX_ISO100_EV0-Crop200%.JPG".

The file 1DX_ISO100_EV0-Crop200%.JPG is the cleanest for me... in fact very good for a 200% crop.

At least, that is what displays on my PC & Monitor.  Anyone else?

Paul

As I mentioned above this method gives about 2 stops of visible SNR improvements, not full 3 stops and this above is "stress" test to see  camera limits and what could be dome with that.
  So definitely 0EV ISO100 image IQ will still be better than -3EV image pulled up 3 stops using this shooting method.
But even this is significant improvement.
 At 100% crop this difference is almost invisible and for normal shots with 0EV it gives enough room for shadows details recovery - almost full 2 stops
Just pull shadows up to the level where banding pattern is just below black point and get almost perfect image
Vertical banding:
 As I mentioned - you can filter down only random noise thus pulling fixed details up from the random noise. Low amplitude fixed image details becomes visible as they not any more masked out by random noise. Image banding are fixed sensor imperfections which also filtered out from random noise and become more visible as well.
I think but not 100% sure that banding is fixed specific pattern for each specific sensor .
If so it could be easily removed as well - just one calibration short with(-3EV) against dark gray surface using the same method. Save this calibration shot and then subtract this from the normal image in Photoshop  - add calibration shot as a second layer above the image, invert this layer and adjust intensity, transparency and contrast for maximum banding removal.
I think all this technique could be easily incorporated in camera firmware for night shot mode.

42

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 04, 2012, 02:42:59 PM »

OK, no magic - just "feeling" what is behind numbers )))

But seriously  this method used for decades for filtering noise out in different applications.
This is use of simple math in real life use.
Noise is random - image is fixed.
So what is required is just to integrate information element for a longer time and random noise would be filtered out.
  I believe that  this is something that  also was being used in astronomy to separate low level image details that could be even below random noise level.  Also used in communication systems.
=========
So how this is done with 1Dx
------
1Dx has one feature which is  intended for some artistic work but it could be used for something completely different.
This feature is multi-exposure – in one of the posts I mentioned about this for filtering out noise.
You can take up to 9 shots and 1Dx will average these shots into one.
 As each shot has it's own random noise distribution then averaging 9 shots into one would  essentially suppress noise - only fixed elements will  be kept - so random electronic noise is significantly reduced but fixed patterns - real image or sensor defects are kept and cleared out of noise. This is equivalent  to 9 times of longer integration time of the information element in the image detector.

How to  do this in 1Dx:
Enable  multi-exposure continuous shooting, number of exposures to 9, multi-exposure control to  Average (to integrate over 9 images) , put mirror lock up with the set button release  to reduce camera vibration (will be returned back by set button) , put camera in high speed continuous shooting mode, press shutter - mirror will be locked, press shutter again and keep it pressed - camera will take quickly 9 images in less than 1 sec , stops and then merge it ( integrate)  into one image.
And you are done.
Now press set button to return mirror back and then you could see the result.
So nothing special - just to see things a bit deeper)))
Method is suitable for only still images .
In normal conditions such kind of shot will take about 0.75 sec with FPS =12
And of course could be only shot using strong stable tripod for camera.

43

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 04, 2012, 02:34:07 PM »

Resizing down to 8Mpx file dimensions? 

No, full sensor resolution )))

44

EOS Bodies - For Stills / Re: 1Dx ISO100 high DR shadows SNR improvement

« on: November 04, 2012, 12:30:15 PM »

OK, now is 200% crop of the similar EV=(-3) shoot from 1Dx  using some  special shooting  method  with 1Dx . 
Shadows are recovered in LR with Exposure slider set to +3 to compensate for in camera EV =(-3) . No noise reduction in LR.
Note how SNR for random noise was improved.

 So we have almost two stops  improvement here  for random noise SNR.
This means that with normal exposure it is possible to successfully recover details in very deep shadows at ISO100.
We still  see some slight pattern noise on the image but I believe that there is some NR software that doing very well with reducing pattern  noise - might be Topaz  De-noise  but not very sure  as I do not have that.
  If all combined together with some special shooting method with 1Dx  it  is possible to get shadow SNR  on par with Nikon D800.
So now quiz - how this was done?

45

EOS Bodies - For Stills / 1Dx ISO100 high DR shadows SNR improvement

« on: November 04, 2012, 12:28:45 PM »

One of the weak sides of the 1Dx is noise   in deep shadows at base ISO.
It seems that nobody is arguing about that any more.
 DXO 1Dx measurement results also show that very clearly on their measurement curves.
And I believe that nobody would be arguing that performance in this area is very critical for not just few people but for many people  - for landscape photography, evening/night  city shots and other sort of photography.
Though it is easy to argue that that Nikon D800 would be the better choice BUT why not to see how we could get better deep shadow SNR from 1Dx.
In one of my posts I mentioned that 1Dx has a very nice feature that could be used for that.
I will not name it now (will do at SOME next post) but here is some magic - example how it works (snapshots from LR screen - noise reduction is totally disabled  )
1.  First image shows 1dX  shot at IS100  with EV=(-3)   the Data Color noise calibration chart  which was  used  for Noise Ninja NR S/W and PS plug-in. Shot  with  manual Out of Focus  to blur target texture so it would not interfere with noise
2. Second one is the same with Exposure slider in LR set to +3  - looks the same as normal shot with EV=0
3. Third  image is 200% crop  of shot with EV=0 showing normal 1Dx ISO shadow noise
4. Fourth one is 200% crop of the EV=-3 shot with LR exposure set to +3 to raise shadows to normal level.
     Level of 1Dx noise at ISO100 at  -3 EV shadows is very clearly seen
 
 And next post is  little bit of magic  - the similar EV=(-3) shoot from 1Dx  recovered in LR - 200% crop  to see the difference  in SNR for random nose with the previous shot.