Brightening/Correcting Underexposed Images

[helpful]

Canon community:

I would like your help with some mathematical/imaging research that I am doing. I am the developer for an unreleased algorithm for correcting underexposure from any type of image. I developed it on the side about nine years ago, when I was in the initial stages of some image compression research. The algorithm uses mappings from complex analysis (not "complex" meaning "complicated," but meaning the use of imaginary numbers) to create a seamless tone curve for any color, brightening or darkening it somewhat independently of how much information is available in the historgram. The RAW/JPEG bit depth doesn't matter, but the compression ratio is somewhat important.

The algorithm does NOT work miracles, and it is very "RAW" in that no post processing is done, so sometimes it produces hot pixels (that's why compression ratio is important, because anything that wasn't true data in the original file gets exaggerated when mapped using complex variables).

The algorithm does not differentiate between "shadow" or "highlight" brightening; it simply works by maintaining the exact same color hue, and adding color-independent brightness to it. The biggest benefit I believe is that colors are extremely pure and precisely maintained throughout the whole process. It's kind of like showing a slide--just brighten the projector to see very dim details and increase the dynamic range.

I am looking for some photos (small resolution, because that's not important; around 720px for the maximum dimension) to experiment with that aren't my own photos. I understand my own algorithm too well, and I feel like I can cook the results too easily if I use my own photos.

I am attaching an example of a photo that is two stops underexposed, and then the corrected version. (Note: the first image is actually at a greater than 100% magnification--I accidentally had my Matlab window too large when I took the screenshot.)

Thanks everyone.

 

[helpful]

No takers?

Let me give you another example. The first image shows a photo underexposed by more than three stops. The data provided to my image color adjustment algorithm was JPEG data--no RAW file allowed.

The file was "pushed" back to the correct exposure. If you look closely you can see where there is some posterization due to the original color having limited bit depth / dynamic range (all the data was jammed into just 10-20% of the color channels available in JPEG). But I think that the colors are extremely beautiful (in the sense of being accurate to both the original photo and the original scene), which is the main goal of my tone adjustment formula.

As far as I know this is a different process based on mathematics which differ from current Photoshop tools. I have compared results, and while similar for small adjustments, the technique I am proposing starts to produce "better" results. Better in this case means much more color accuracy and fidelity when "pushing" or "pulling" by large amounts.

Sorry that I can't share the mechanism for doing this... right now one has to have math software which can do image processing as well as advanced math functions, just in order to apply my code to their images.

I am looking for some underexposed OR overexposed images which are not completely hopeless (obviously, I can't turn one photon of data in a black image into a crowd watching Queen Elizabeth's Diamond Jubilee).

If you post a poorly exposed photo as a response I will consider working on it. I don't think I can recover more than four stops from a JPEG image, so if the primary subject is five stops or more under/over exposed, then that probably isn't going to work.

 

[helpful]

This third post shows the absolute "maximum" pushing that is possible to do with Picasa from Google, compared to my algorithm shown in the 2nd post. Mathematically, Picasa has an excellent procedure for adding fill light, which is why I am using it as a comparison.

I am giving Picasa the exact same 3+ stop underexposed original JPEG photo that was show in the 2nd post above.

You can tell that the result is much more ugly and has less accurate color than the rendition produced by my software. Plus I think my software had actually pushed the image further than Picasa did (and my software can go even further).

If anybody wants to take the dark image from the 2nd post and apply any corrections to it that you know of and see if you can obtain something better than the result that I did in the 2nd post, I would be happy to see whether my software is up to the best that you all can do.

 

[helpful]

OK, to simplify this for everyone, rather than asking you all to upload random images, I am just going to give you an underexposed starting image from my files, and anyone who wants to can push it / brighten it. I will show my results when I get some responses.

The original image is attached:

Update: to make it more fun, I am also attaching an overexposed image for you to work with, if you would rather try to use your image editing skills in reverse:

Your goal is to try correct the underexposure in the basketball photo, and to correct the overexposure in the softball photo.

I will give my results after a while. These are extremely badly exposed photos, so this is a worthy challenge!

 

[briansquibb]

Are you planning to implement this as a plugin? with sliders?

 

[helpful]

Are you planning to implement this as a plugin? with sliders?

Yes, that would be my ultimate goal.

 

[briansquibb]

Are you planning to implement this as a plugin? with sliders?

Yes, that would be my ultimate goal.

Why would I play with the jpg when I normally do this work on the RAW file?

 

[wickidwombat]

do you do this on raw files too or only jpg images?
what about the pattern read noise in canon sensors?
does this get rid of that issue when pulling up shadows excessively?

 

[helpful]

Brian, I don't have the RAW file. More could be done from a RAW original, so I picked photos that were challenging.

And for ww's question, I can only do this on jpg images for now because I am still stuck with Matlab R2009a. My university has chosen not to pay for new software right now, and since I'm not a student Mathworks would charge me thousands for it. I don't think that Matlab 2009 would support any of the new RAW image formats. So actually I couldn't do this with RAW files, simply because I couldn't read them into Matlab.

My area of knowledge about semiconductors is in manufacturing quality control, analyzing the doping profile and things like that. I understand the basics about read noise, but nothing more specific about Canon sensors than what is in a textbook about any type of sensor.

When shadows are so dark that they are a single color value rather than range of values, then my method fails. It can only work when there is still a palette of color. It works by preserving those colors throughout the process of brightening. Of course, that means each color channel is brightened separately/independently by a different amount. Theoretically, someone could get the same result just by pushing up the three color channels one by one and then somehow using their eyes to get the exact color match. But the program I coded with Matlab does it automatically. (And it's all based on pure math, so I'm really not sure if it's exactly the same as pushing or pulling the three color channels with a slider in Photoshop.)

 

[wickidwombat]

hmmm how about 5Dmk2 raw?
failing that I could convert to jpg some that had a flash fail to fire resulting in under exposure to jpg and email them to you.
Its very interesting, PM me an email address and i'll get a specific one over to you, I'll also send you a "corrected" one from Lightroom (it didnt turn out very well) that will help as a point of reference for you i guess

is that the sort of thing you are chasing?

 

[NotABunny]

Have you tried Ligthroom 4 with the flowers image? (They have a free demo.)

 

[helpful]

hmmm how about 5Dmk2 raw?
failing that I could convert to jpg some that had a flash fail to fire resulting in under exposure to jpg and email them to you.
Its very interesting, PM me an email address and i'll get a specific one over to you, I'll also send you a "corrected" one from Lightroom (it didnt turn out very well) that will help as a point of reference for you i guess

is that the sort of thing you are chasing?

Yes, that is what I am looking for if I understand you correctly. Something that would have been a great photograph with good focus, good range of bright to dark... except that it was accidentally underexposed or overexposed by at least one stop or more.

I am not trying to rescue blurry small photos of school plays where one person's face is a blob of light about 150 feet away and everything else is black.

The examples that I posted are basically what I am looking for. (Note that nothing in the photos was completely blown out to white, or completely lost to black. There was still a dim range of color, which is what the formula needs in order to work.)

I will PM you later today.

 

[helpful]

Recovery Results: Basketball Photograph

Picasa results first and then "color scale" results second. Both processes were given the same input file, the JPEG image that was posted in this thread yesterday.

This isn't going to be beautiful for either method: the starting photograph was terribly deficient in data.

What are your impressions? Does the new method work any better than Picasa?

 

[helpful]

Recovery Results: Softball Photograph

The Picasa results are shown first. I did the best I could. I even added a tint to the photograph to replicate the color of the dirt (that was cheating) and used Picasa's HDR feature. This was the absolute best I could get it with any tools in Picasa.

The "colorscale" results are shown second. All I did was to darken the photo by a few stops. A little bit of detail was washed out because it simply wasn't there in the original photo, but overall the recovery is amazing.

What do these results look like to you? Does anyone know of an exposure correction tool that can do better? (You are working with only the data in the JPEGs posted higher up in the thread, which is used as input in these results that I am showing.)

 

[helpful]

By the way, it also works well as an HDR image processing tool by taking an L^p combination of brightened and darkened images.

See attached:

 

[helpful]

Another example of HDR:

First image is original
Second image is Picasa's best attempt at HDR
Third image is HDR applied with the new technique.

 

[Tammy]

Sorry i'm not posting an image for you to use but I just wanted to let you know that I think your method/software is truly going to be amazing! Even just for use to extract more accurate color/detail/dynamic range from your typically exposed photos.

Any idea on how long we might be looking at till it may be in product stage?

 

[NotABunny]

I don't understand why you show us photos processed with Picasa.

Here is one of your above photos recovered with 2 sliders in Lightroom 4.1: exposure +1, shadows +80

 

[helpful]

I don't understand why you show us photos processed with Picasa.

Here is one of your above photos recovered with 2 sliders in Lightroom 4.1: exposure +1, shadows +80

That's very good. I think the colors are a little better with my method. Also, I was just taking a screen shot of Matlab. I will upload the original corrected image for comparison. I have Matlab only in my office, unfortunately, and it's very slow to remote desktop to it.

Other methods can definitely push exposure. I do have Lightroom and the full suite of Photoshop + a lot more from Adobe (not CS 6 yet, however), but I used Picasa because really Google does have good software, and I am not sure how many people on this forum really have all the expensive tools that they should have. Picasa, although free, has many state of the art methods built in, and a much faster database than Lightroom. I have three computers with almost a million photos on each one from the past decade or so, and the rate of pictures needing to be stored is increasing all the time, and I couldn't survive without the speed of Picasa.

However, I think my method, based on true color curves, rather than shifting the histogram, is better. As you know, the histogram becomes choppy when doing these exposure adjustments.

I will also look at the histogram after I upload the original corrected image, and see whether it is less damaged than the histogram from Lightroom after doing the +1 exposure and +80 shadows.

Thanks for taking the time to help with my project!!

 

[helpful]

Sorry i'm not posting an image for you to use but I just wanted to let you know that I think your method/software is truly going to be amazing! Even just for use to extract more accurate color/detail/dynamic range from your typically exposed photos.

Any idea on how long we might be looking at till it may be in product stage?

The easiest form for me to let people use it would be through a web page, i.e., uploading an image and then having it processed--being able to select the relative brightness or darkness multiplier; or being able to specify how many stops of + and - exposure to recapture in the HDR processing option.

I am working right now on porting it from Matlab to a C program using libjpeg for compressing and decompressing JPEG images.

Did anyone know that JPEG actually supports all sorts of color curves, and 16-bit per channel color depth all the way up to 64 bit color depth, and that is per channel??

The 8-bit "baseline" JPEGs that everyone is using are just the tip of the ice berg.

JPEGs are truly marvelous because they erase only data that is invisible to the human eye at first (separating the image into chroma and luminance channels), and then erasing more and more data until an 8x8 pixel grid is reduced to a single data point, which is obviously visible to the human eye.

It would be a miracle if cameras started to use gamma curves with JPEG images that had 16-bit color channels plus an extra 16-bit luminance channel. There would be no limit to the processing options then, just a limit to the sensors (16-bits of luminance could store 64,000+ levels of brightness without ANY noise or affect on color accuracy). And it would all fit into 64-bit data value sizes, which is a perfect match for new 64-bit computers (actually 64-bit systems were adopted by mainstream users almost 10 years ago). And the JPEG image compression method would give people the option to store it all at 10:1 compression or greater if they wanted to, without losing any visible information.

And then there would be no more proprietary RAW image formats.