October 23, 2014, 02:47:31 AM

Author Topic: Brightening/Correcting Underexposed Images  (Read 4015 times)

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Brightening/Correcting Underexposed Images
« on: June 06, 2012, 04:55:54 PM »
Canon community:

I would like your help with some mathematical/imaging research that I am doing. I am the developer for an unreleased algorithm for correcting underexposure from any type of image. I developed it on the side about nine years ago, when I was in the initial stages of some image compression research. The algorithm uses mappings from complex analysis (not "complex" meaning "complicated," but meaning the use of imaginary numbers) to create a seamless tone curve for any color, brightening or darkening it somewhat independently of how much information is available in the historgram. The RAW/JPEG bit depth doesn't matter, but the compression ratio is somewhat important.

The algorithm does NOT work miracles, and it is very "RAW" in that no post processing is done, so sometimes it produces hot pixels (that's why compression ratio is important, because anything that wasn't true data in the original file gets exaggerated when mapped using complex variables).

The algorithm does not differentiate between "shadow" or "highlight" brightening; it simply works by maintaining the exact same color hue, and adding color-independent brightness to it. The biggest benefit I believe is that colors are extremely pure and precisely maintained throughout the whole process. It's kind of like showing a slide--just brighten the projector to see very dim details and increase the dynamic range.

I am looking for some photos (small resolution, because that's not important; around 720px for the maximum dimension) to experiment with that aren't my own photos. I understand my own algorithm too well, and I feel like I can cook the results too easily if I use my own photos.

I am attaching an example of a photo that is two stops underexposed, and then the corrected version. (Note: the first image is actually at a greater than 100% magnification--I accidentally had my Matlab window too large when I took the screenshot.)

Thanks everyone.

« Last Edit: June 07, 2012, 12:15:39 AM by helpful »
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

canon rumors FORUM

Brightening/Correcting Underexposed Images
« on: June 06, 2012, 04:55:54 PM »

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #1 on: June 06, 2012, 11:57:49 PM »
No takers?

Let me give you another example. The first image shows a photo underexposed by more than three stops. The data provided to my image color adjustment algorithm was JPEG data--no RAW file allowed.

The file was "pushed" back to the correct exposure. If you look closely you can see where there is some posterization due to the original color having limited bit depth / dynamic range (all the data was jammed into just 10-20% of the color channels available in JPEG). But I think that the colors are extremely beautiful (in the sense of being accurate to both the original photo and the original scene), which is the main goal of my tone adjustment formula.

As far as I know this is a different process based on mathematics which differ from current Photoshop tools. I have compared results, and while similar for small adjustments, the technique I am proposing starts to produce "better" results. Better in this case means much more color accuracy and fidelity when "pushing" or "pulling" by large amounts.

Sorry that I can't share the mechanism for doing this... right now one has to have math software which can do image processing as well as advanced math functions, just in order to apply my code to their images.

I am looking for some underexposed OR overexposed images which are not completely hopeless (obviously, I can't turn one photon of data in a black image into a crowd watching Queen Elizabeth's Diamond Jubilee).

If you post a poorly exposed photo as a response I will consider working on it. I don't think I can recover more than four stops from a JPEG image, so if the primary subject is five stops or more under/over exposed, then that probably isn't going to work.
« Last Edit: June 07, 2012, 12:02:45 AM by helpful »
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #2 on: June 07, 2012, 12:09:29 AM »
This third post shows the absolute "maximum" pushing that is possible to do with Picasa from Google, compared to my algorithm shown in the 2nd post. Mathematically, Picasa has an excellent procedure for adding fill light, which is why I am using it as a comparison.

I am giving Picasa the exact same 3+ stop underexposed original JPEG photo that was show in the 2nd post above.

You can tell that the result is much more ugly and has less accurate color than the rendition produced by my software. Plus I think my software had actually pushed the image further than Picasa did (and my software can go even further).

If anybody wants to take the dark image from the 2nd post and apply any corrections to it that you know of and see if you can obtain something better than the result that I did in the 2nd post, I would be happy to see whether my software is up to the best that you all can do.
« Last Edit: June 07, 2012, 12:11:41 AM by helpful »
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #3 on: June 07, 2012, 12:35:28 AM »
OK, to simplify this for everyone, rather than asking you all to upload random images, I am just going to give you an underexposed starting image from my files, and anyone who wants to can push it / brighten it. I will show my results when I get some responses.

The original image is attached:

Update: to make it more fun, I am also attaching an overexposed image for you to work with, if you would rather try to use your image editing skills in reverse:

Your goal is to try correct the underexposure in the basketball photo, and to correct the overexposure in the softball photo.

I will give my results after a while. These are extremely badly exposed photos, so this is a worthy challenge!
« Last Edit: June 07, 2012, 12:43:55 AM by helpful »
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

briansquibb

  • Guest
Re: Brightening/Correcting Underexposed Images
« Reply #4 on: June 07, 2012, 12:43:50 AM »
Are you planning to implement this as a plugin? with sliders?

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #5 on: June 07, 2012, 12:46:44 AM »
Are you planning to implement this as a plugin? with sliders?

Yes, that would be my ultimate goal.
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

briansquibb

  • Guest
Re: Brightening/Correcting Underexposed Images
« Reply #6 on: June 07, 2012, 01:03:12 AM »
Are you planning to implement this as a plugin? with sliders?

Yes, that would be my ultimate goal.

Why would I play with the jpg when I normally do this work on the RAW file?

canon rumors FORUM

Re: Brightening/Correcting Underexposed Images
« Reply #6 on: June 07, 2012, 01:03:12 AM »

wickidwombat

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4520
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #7 on: June 07, 2012, 01:12:22 AM »
do you do this on raw files too or only jpg images?
what about the pattern read noise in canon sensors?
does this get rid of that issue when pulling up shadows excessively?
APS-H Fanboy

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #8 on: June 07, 2012, 01:22:57 AM »
Brian, I don't have the RAW file. More could be done from a RAW original, so I picked photos that were challenging.

And for ww's question, I can only do this on jpg images for now because I am still stuck with Matlab R2009a. My university has chosen not to pay for new software right now, and since I'm not a student Mathworks would charge me thousands for it. I don't think that Matlab 2009 would support any of the new RAW image formats. So actually I couldn't do this with RAW files, simply because I couldn't read them into Matlab.

My area of knowledge about semiconductors is in manufacturing quality control, analyzing the doping profile and things like that. I understand the basics about read noise, but nothing more specific about Canon sensors than what is in a textbook about any type of sensor.

When shadows are so dark that they are a single color value rather than range of values, then my method fails. It can only work when there is still a palette of color. It works by preserving those colors throughout the process of brightening. Of course, that means each color channel is brightened separately/independently by a different amount. Theoretically, someone could get the same result just by pushing up the three color channels one by one and then somehow using their eyes to get the exact color match. But the program I coded with Matlab does it automatically. (And it's all based on pure math, so I'm really not sure if it's exactly the same as pushing or pulling the three color channels with a slider in Photoshop.)
« Last Edit: June 07, 2012, 01:34:55 AM by helpful »
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

wickidwombat

  • Canon EF 400mm f/2.8L IS II
  • *******
  • Posts: 4520
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #9 on: June 07, 2012, 02:40:25 AM »
hmmm how about 5Dmk2 raw?
failing that  I could convert to jpg some that had a flash fail to fire resulting in under exposure to jpg and email them to you.
Its very interesting, PM me an email address and i'll get a specific one over to you, I'll also send you a "corrected" one from Lightroom (it didnt turn out very well) that will help as a point of reference for you i guess

is that the sort of thing you are chasing?
APS-H Fanboy

NotABunny

  • PowerShot G1 X II
  • ***
  • Posts: 60
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #10 on: June 07, 2012, 10:42:08 AM »
Have you tried Ligthroom 4 with the flowers image? (They have a free demo.)

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #11 on: June 07, 2012, 11:50:35 AM »
hmmm how about 5Dmk2 raw?
failing that  I could convert to jpg some that had a flash fail to fire resulting in under exposure to jpg and email them to you.
Its very interesting, PM me an email address and i'll get a specific one over to you, I'll also send you a "corrected" one from Lightroom (it didnt turn out very well) that will help as a point of reference for you i guess

is that the sort of thing you are chasing?

Yes, that is what I am looking for if I understand you correctly. Something that would have been a great photograph with good focus, good range of bright to dark... except that it was accidentally underexposed or overexposed by at least one stop or more.

I am not trying to rescue blurry small photos of school plays where one person's face is a blob of light about 150 feet away and everything else is black.

The examples that I posted are basically what I am looking for. (Note that nothing in the photos was completely blown out to white, or completely lost to black. There was still a dim range of color, which is what the formula needs in order to work.)

I will PM you later today.
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #12 on: June 07, 2012, 02:54:30 PM »
Recovery Results: Basketball Photograph

Picasa results first and then "color scale" results second. Both processes were given the same input file, the JPEG image that was posted in this thread yesterday.

This isn't going to be beautiful for either method: the starting photograph was terribly deficient in data.

What are your impressions? Does the new method work any better than Picasa?
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

canon rumors FORUM

Re: Brightening/Correcting Underexposed Images
« Reply #12 on: June 07, 2012, 02:54:30 PM »

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #13 on: June 07, 2012, 02:59:17 PM »
Recovery Results: Softball Photograph

The Picasa results are shown first. I did the best I could. I even added a tint to the photograph to replicate the color of the dirt (that was cheating) and used Picasa's HDR feature. This was the absolute best I could get it with any tools in Picasa.

The "colorscale" results are shown second. All I did was to darken the photo by a few stops. A little bit of detail was washed out because it simply wasn't there in the original photo, but overall the recovery is amazing.

What do these results look like to you? Does anyone know of an exposure correction tool that can do better? (You are working with only the data in the JPEGs posted higher up in the thread, which is used as input in these results that I am showing.)
5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

helpful

  • EOS M2
  • ****
  • Posts: 212
  • Ecclesiastes 3:11
    • View Profile
Re: Brightening/Correcting Underexposed Images
« Reply #14 on: June 07, 2012, 05:32:37 PM »
By the way, it also works well as an HDR image processing tool by taking an L^p combination of brightened and darkened images.

See attached:

5DIII, 5DII, 7D x5, 6D, T2i, T3, 1D X, 10-22mm, 16-35mm II, 18-55mm II, 18-135mm IS x2, 70-200mm f/2.8L II, 24mm f/1.4L II, 50mm f/1.4, 50mm 1/1.8 II x2, 85mm f/1.8 x2, 100mm f/2 x2, 135mm f/2L x2, 200mm f/2.8L II x2, 1.4X III, 2.0X II, 60mm f/2.8 Macro, etc. only had room to list a few Canon items

canon rumors FORUM

Re: Brightening/Correcting Underexposed Images
« Reply #14 on: June 07, 2012, 05:32:37 PM »