ISO and long exposure issues:5Dmkiii

[Aglet]

Thanks JRista, that's a good demo of what can be done with some really good processing tools and methods.

 

[jonjt]

I am still curious how you intend to subtract your darks. It is pretty critical that be done on the non-debayered raw, otherwise you could end up with artifacts around hot pixels, which can sometimes be worse than the hot pixel themselves (i.e. funky ringing around the hot pixels, as hot pixels blend into their neighboring pixels during demosaicing, but with modern evaluative demosaicing algorithms, how they blend in the darks is not guaranteed to be the same as how they blend in the lights.)

If you subtract the darks before debayering, then you should be fine with running DeNoise. I recommend running it last, as a wide range of processing steps can enhance noise. If you reduce noise mid-processing, then subsequent processing can increase your noise. Doing noise reduction at the end, you know how much noise you really have. Just keep in mind, DeNoise DOES blur detail. You can run it really mildly, and not have a very visible impact to your detail, but to run it more heavily (i.e. on par with what I did using PixInsight), your going to lose some detail.

That's the benefit of PixInsight. Everything almost all the tools are directly affected by masking, so you can use masks to attenuate and control the impact of NR, prevent it from affecting highlights, etc. Topaz filters are theoretically supposed to work the same way, but I've never been able to gain the same kind of control over NR with DeNoise as I have with PixInsight. DeNoise is trying to control the masking and everything for you, and how they combine their masks with any other masks you may have applied is a black box.

I wanted to use DSS for the subtraction of the master darks but, your comment about the noise performance of PixInsight has me wondering if the cost is justified. I also have not been able to master the mask capabilities of Denoise so, there might be an additional value add there.

Thanks for the comments.

 

[jrista]

I wanted to use DSS for the subtraction of the master darks but, your comment about the noise performance of PixInsight has me wondering if the cost is justified. I also have not been able to master the mask capabilities of Denoise so, there might be an additional value add there.

Thanks for the comments.

Glad to be of help. If you do get PixInsight. It is an extremely powerful program, and it's built with an object-oriented UI. It's a rather different UI, so it won't look or behave like Photoshop or other common image editors. However do give it a chance. There is the 45 day free trial period....I strongly encourage you to take advantage of it.

A lot of things in PI are done by dragging instances of tools to instances of images to "apply" that tool. It seems unusual and inverted at first (at least, so I hear from other astrophotographers), but once you get the hang of it, it's a very fast and flexible way to work with your images. Also, previews are the most critical feature. Using previews, both static previews you can draw onto any image, as well as the real time preview, you can learn a lot of PixInsight's tools via trial-and-error.

Learning PI's tools by trial and error is key to using it successfully. If you look around the web, you'll hear that PI has a steep learning curve. Taken as a whole, as a single monolithic application like Photoshop, this is true. It's about as complicated as Photoshop, and to learn it in it's entirety, it would take as much effort as it takes to learn Photoshop. The nice thing about PI is everything is done with independent, isolated tools, possibly with masks. If you take each tool as a whole little app to be learned in and of itself, learning PI becomes a much less daunting task.

For someone who does night photography, I think PI can certainly bring some new powerful ways to produce quality images to the table. In the end, I expect you will use PI for some things (calibration with darks, noise reduction), and PS for everything else (all the more artsy aspects of image processing...PI can export 16-bit TIFF, which you can then just bring into PS.) Both programs are complimentary to each other, use them both together, leveraging each one's strengths to get the most out of your imaging.

Also, you are always free to ask me for help when you need it.

 

[jonjt]

Thanks for all the help!

 

[johnnycash]

The noise you are seeing is noise caused by dark current. I downloaded all of your images, and checked them out in PixInsight, an advanced astrophotography processing tool.


As far as I can tell, you do not have any many pixels. Stuck pixels tend to be significantly brighter than the background noise in a dark frame all the time regardless of exposure time or thermal status. I did see a very small handful of such pixels...values around 3500 ADU when the background noise averaged around 513 ADU (ISO 800 dark).


The thing about dark current is it is an additional signal added to your image signal. As a signal, it includes it's own random (gaussian) noise. In a 450 second exposure at higher ambient temperatures, you could easily accumulate a dark current signal in the range of several hundred ADU. If your dark current was 300 ADU, it's noise would be SQRT(300) ADU, or 17.32. That is additional random noise, on top of read noise and photon shot noise, and separate from the increase in hot pixels (which are different from stuck pixels, as they change with exposure time and thermal factors.)


None of this is unusual. The use of long exposure noise reduction (LENR) will take another 450s exposure after the image exposure, with the shutter closed, and subtract that from the image frame. That will usually remove hot pixels and stuck pixels, however it will actually also usually increase random noise, making the image noisier (even if the characteristic is better).


If you wish to get the best results possible with your night imaging, you could take the astrophotography approach. Generate a master dark frame, and subtract that yourself, rather than using LENR. A master dark frame is generated by taking a bunch of frames, say 25 or 36, and stacking them together with an averaging algorithm. This single master dark could then be used to subtract the dark current and it's noise from your nighttime images. It can be reused, so long as the temperatures of your images is within a few degrees of your master dark, so you waste less time on-scene waiting for LENR to take an additional exposure after each and every frame.

By averaging 25 dark frames together, you reduce the random noise by a factor of 5. Average 36, and you reduce it by a factor of 6. Average 100, you reduce it by a factor of 10. The lower the random noise in your master dark, the lower your random noise will be in your "calibrated" image frames. This will give you the best noise characteristics possible in your images, removing the hot and stuck pixels, the bias signal, as well as any dark signal offset.

Hi, I have checked your website and your astrophotography is amazing!!
http://jonrista.com/photography/astrophotography/deep-sky-nebula-and-galaxies/

 

[privatebydesign]

I have been meaning to post to this thread for weeks! Hope it isn't completely buried.

Anyway, I also downloaded the RAW file and dark frame and in my opinion, whilst there is obviously a pixel level improvement from the Pixinsight, I just used Photoshop and Lightroom and think they did a very good job, certainly good enough for even the largest prints, at making the image a good one.

The work that I did is very rudimentary and I am happy to elaborate if anybody is interested, but was simply process in LR, open as Smart Object, then open the dark frame as a Smart Object and bring it into the previous file as a layer, then choose blending mode 'subtract', play with a few settings and you are done.

Here are my crops as close to John's as possible. To be sure there is pixel level detail still clearly observable in the image, indeed it is a very sharp image, the handrails on the top cables and the poles on the top of the further tower are all pixel level details.

Hopefully the forum algorithms won't mess with the quality too much.

 

[privatebydesign]

Here are some 100% crops from my LR + PS version showing great detail, sharpness and dark tonality.