Can more be done with the way sensors deliver us RAW info?

[Jettatore]

I'm thinking one likely avenue for future digital camera upgrades is in the way RAW data is captured and/or returned to the user.

For example, as a sensor remains active, light photons hit diodes on the sensor and build up an electrical charge. I wonder if it's possible in the future for the end user of a '1/50 or longer exposure' shot for example to get a series (or matrix) of various images (binary numbers) recorded to the sensor and be able to layer and manipulate each series in the matrix individually.

I could see this allowing us to to visualize the evolution of the captured image as it collects on the sensor, and in editing, allow us to pick and choose the most appropriate selection or combination of selections for any given area of the final image.

I could also see changes in the way RAW data is captured and delivered eventually leading to the ability to set/alter ISO speed after the fact (at least to some degree) in RAW editing.

One other improvement this could lead to potentially, is image stabilization after the shot is taken. I know there are filters that help with this already but I am thinking if it were possible to separate out exposures at various timings of the image capturing process and then layer them in post that this would be an advantage for post image stabilization.

I will admit I know too little about the back end of our current RAW files structure and as well, I know too little about the current processes of exposure on digital imaging sensors, but I know just enough to allow my imagination to run wild, which is why I am posting.

 

[Loswr]

Reading out a well while it's still collecting photons is problematic...

 

[Neeneko]

I don't see why not.

To a degree this is how exposure stacking already behaves... so combine exposure stacking with high speed video capture and you could probably get what you are asking for.

It would probably be a boon to HDR photographers.. single capture with the whole range of exposure data in it.

 

[Neeneko]

Hmmm, you mean like you do with video?

Or similar to what might happen with 14fps on the 1DX?

Hrm... there is that low cost RAW video camera that came out a while ago.... I wonder how many FPS it actually has.... I know I often do exposures int he 30s-90s range, which would kinda give the effect the OP is asking about.

 

[Jettatore]

If you like, imagine that a sensor is made up of measuring cups that hold rain water and the only way to find out how much rain is in the cup is to open a trapdoor at the bottom and let all of the water out and see how much was collected.

Ok, lets go with that. If (technically) there were some new developments or inventions which allowed for continuous readouts of a photo-diode site over time (or if not continuous, just more rapid emptying/resetting and counting of said sites over time), it could perhaps allow for a sort of 'evolutionary slideshow' of the captured/converted data as it developed over time. By separating out the values in chunks of time, it could perhaps allow us some additional flexibility in adjusting exposure in shadows/highlights, etc. etc. while in editing, etc, etc. Modify the ISO after the fact, etc. etc., Modify the shutter speed after the fact, etc. etc.

 

[rpt]

I'm thinking one likely avenue for future digital camera upgrades is in the way RAW data is captured and/or returned to the user.

For example, as a sensor remains active, light photons hit diodes on the sensor and build up an electrical charge. I wonder if it's possible in the future for the end user of a '1/50 or longer exposure' shot for example to get a series (or matrix) of various images (binary numbers) recorded to the sensor and be able to layer and manipulate each series in the matrix individually.

I could see this allowing us to to visualize the evolution of the captured image as it collects on the sensor, and in editing, allow us to pick and choose the most appropriate selection or combination of selections for any given area of the final image.

I could also see changes in the way RAW data is captured and delivered eventually leading to the ability to set/alter ISO speed after the fact (at least to some degree) in RAW editing.

One other improvement this could lead to potentially, is image stabilization after the shot is taken. I know there are filters that help with this already but I am thinking if it were possible to separate out exposures at various timings of the image capturing process and then layer them in post that this would be an advantage for post image stabilization.

I will admit I know too little about the back end of our current RAW files structure and as well, I know too little about the current processes of exposure on digital imaging sensors, but I know just enough to allow my imagination to run wild, which is why I am posting.

Thinking aloud... Feel free to tell me I did not understand you right...

So say you want 8 exposures as your series, and you (I would) want to support native 25600 ISO, I think you would like to store 8 exposures of 1/400 each. Now since you would be adding the pixel value of each of the corresponding pixels from each of the 8 frames, the ISO it needs to support natively will probably be 25600 x 8 = 204800.
Now the 1DX blokes are going to want support for 51200 native, so the base ISO then goes to 409600...

Now if your number of frames increase, the base native ISO needed to support them increases too...

This will only take care of over exposure or blowout fixes. Is this what you were thinking?

 

[Orangutan]

Reading out a well while it's still collecting photons is problematic...

I've ruminated about this also; my assumption has been that there would be multiple wells per pixel, with some gate to switch to to the next well when the first is full, or at some pre-determined time. I then took the next step and wondered whether it would be possible to do some controlled attenuation of the signal so the second (of two) well fills, say, 1/8th or 1/16 as fast, thereby lifting the top-end of the DR by 3 or 4 stops.

I'm not an EE, so I have no idea if this is remotely feasible, but if someone tries to patent this I'm going to cite "prior art" based on this post.

 

[Brendon]

The main obstacle to this idea is the readout time necessary for each of the pixels at 12 bits is too long. So instead of sampling a little faster and maintaining the bit depth, sample MUCH faster but only sample 1 bit. In essence, you run the the bias up so high on the transistor that you have an avalanche photo diode that triggers if it sees a photon ('1') or does not ('0'). The problem still persists that you need one hell of a ROIC to get you to the speeds that you need but the lane width decreases significantly (by 12x). Other problem is dark and bias noise that will cause a "false 1" but it's not like you get a full 12 bits of info from a 12 bit array anyway. The sampling speeds you would need to make sure that you are not saturating your sensor are pretty high, but when you get to the sweat spot it turns out that you need MUCH MUCH less light to make a sensor work this way and you would essentially have infinite dynamic range.