What will be the standard high ISO levels 6 to 8 years from now?

[fotomancb]

I'm certainly not an engineer. But in real, I can't believe my eye terms, the ISO settings will increase greatly with the advances of the D1X.

We are a lab. We get to play and compare at all sizes on photo paper up to 20x30. The quality at 32K, 40K and even 51K are absolutely amazing!

Under Little League lights, shooting action 3 weeks ago, I shot test shots at 16K,32K, 51K 105K and 210K. All the lessons we've been taught and learned the past 50-60 years as far as grain/noise will be thrown out the window. I've never ever seen quality on enlargements at 8x10 on all of the above ISO's. Only 210 (H2) was not usable for our action. But it wasn't needed. I was able to shoot at 1/800th @ 3.2 and the key to all of this at 2/3rd's of a stop overexposed at 16,000 ISO. It finally allowed me after 16 years of shooting at this same park and never being able to at night get any exposure on batters eyes under their batting helmet. To get any light in the past it was 6400 ISO, 200-250th @2.8. But 250th doesn't cut it when a 14 year old swings a bat. Meaning you get head and hand movement which translates to soft photos to parents. Can you imagine 800 ASA film, pushed 2 stops to 3200 just a few years ago? Which by the way is what 210K reminds me of!!!!

I did another test. Because I thought maybe I was blowing this out of proportion in my giddiness. I shot with flash inside our office with flash of a yellow spray can sitting on a shelf with a few other items. Nothing special. I shot at 400/800/1600. We made 8x10's. The following week we had a meeting with about 15 of our photographers who shoot in our business with us. They were shown the prints. EACH thought that the 1600 was the nicest shot! Yep 1600 ISO!

Now last week I shot 2 high school football games. I've shot 7 Super Bowls in my career. You haven't been able to shoot HS at night like a regular NFL game. Until now. I couldn't wait to shoot the 400mm f/2.8 with the 2x extender, giving me a 800mm f5.6. When the ambient light went down, I was at 32K, 800th at between 2.8-3.5, TV mode. Again the key is to be able to overexpose enough to see eyes inside the helmets. It worked. Not only did it work, it worked extremely well. We made a 20x30 poster of a kid going up to grab to make a catch. Just awesome at that size. No cropping needed. He was baout 30-35 yards away. Sharpness and noise held even at 32K ISO. Proof is in the pudding! You've got to see it to believe it.

Hope this was helpful. Thank you Canon! You do owe us one for the 1D Mark III debacle!

 

[pedro]

Well, A Usable ISO 51,200 for Color Images.

On the 5D3 I can use 51,200 as my limit for high-speed B&W. ISO 25,600 is my limit for Color Images.

You are right. Color looks way to strange. But converted to BW it just looks decent. The tad of noise stands for a 21st century "filmdays" effect 8)

 

[jrista]

To the OP, if we take a look at Sensorgen.info data on quantum efficiency, we see it increases about 7-8% for every additional stop of ISO. If we follow that trend, assuming Canon can keep increasing Q.E. by about 8% per stop of ISO, and if they can maintain the rate of Q.E. improvement of 16% per 4.5-year cycle...then every 4.5 years we should see two more ISO levels. That would put the next generation at a native ISO 102400 on a sensor with about 57% Q.E. for the 5D IV, and if current statistics hold true, a native ISO of 204800 with about 60% Q.E. for the 1D XI.

Now, the real question is how usable that ISO will be. I am not sure a sustained linear increase of 8% per stop is really going to make an ISO 204800 usable. It'll be better than a digital boost ISO, but dynamic range is going to severely suffer and noise will still be horrendous. I would suspect that we won't see usable ISO 204800 until we are in the 80% Q.E. range, which would greatly improve S/N ratios even in the shadows. I don't know if we can really achieve that level of efficiency in a consumer-grade device, though...to date, its always required far more rigorous constraints on manufacturing quality, and usually requires some kind of active/thermoelectric cooling. Sony Exmor technology, combined with a backilluminated sensor and...really for honest usability...ungodly fast lenses...might make it a possibility. Either way...that would put usable ISO 204800 a good 8-10 years away at the earliest, or about two to two and a half product cycles.

You are being awfully generous there about actual progress made over those years if you are talking about middle gray SNR (they have cleaned up banding and ugly noise character and such a lot, which at times, can you lead to a larger usable increase in stops than the change in middle gray SNR alone would imply though, however even in that regard they have now made pretty huge strides so it will be hard to continue that trend now) which is leading to changes in stops being paired with changes in Q.E. that don't make any logical sense.

One thing with an almost ISOless sensor like Exmor you can capture using extreme under exposure at low ISO and then lift and get nearly as clean an image as shooting at a higher ISO but shooting low ISO with large underexposure means you still have the top end not chopped off, raising ISO a stop chops off a stop each time so if you applied some fancy warped tone curve maybe you can get some more DR out of things. I don't know how easy it is to do extreme tone mapping like that, I'm sure with careful work you could make it reasonably natural to an extent to definitely help a bit.

I'm not just thinking about Canon, I'm thinking more about the industry as a whole. In that same time, the Sony/Nikon alliance has made considerable strides in noise at all levels, and their Q.E. is already up to 57%. Assuming Canon is not just a dud these days, and that they are internally innovating, within a decade (a non-trivial long period of time), and an amalgamation of technological advances across the board (whatever Canon did to achieve native 51200, Sony Exmor tech, active/thermoelectric cooling, backillumination, and who knows what other advancements) I think we can see some pretty amazing things.

I think within the next decade or two we'll encounter some of the same problems the PC CPU industry had not too long ago...where they started making gates so small they had to start REALLY innovating to keep improving, doing things like stretching silicon, switching to copper wiring, improving their optical etching technology to produce a cleaner, finer beam, etc. I don't think it will be long until were at a similar level of advancement with digital sensors...they will hit a wall of physics, probably with ISO levels, that forces them to radically reinvent and hyperinnovate to either prove the problem is a physical limitation and nothing can be done about it, or work around the problem and improve in other ways.

 

[Aglet]

Charge multiplication could become a technical factor in providing black cat in a coal cellar shots.
CM in a solid state device like an image sensor could potentially deliver multiple electrons per incoming photon.

Charge multiplication won't do much to improve overall high-sensitivity SNR or DR but could provide a beefier signal that's easier to work with and would reduce the effect of read noise at those very high sensitivities.

As it is, QE is already reasonable, some mfrs already have really low read noise and, as some have explained in previous posts (good analogy of watching raindrops on concrete), we're already playing with individual photons and electrons so high ISO images are going to look noisy unless there's a clever processing ability to smooth out that randomness for cleaner looking output. That has its limits too since DR naturally reduces as sensitivity increases, to the point of it becoming a matter of a pixel being ON or OFF.

There's a lot of areas for incremental improvement in the present image sensor paradigm but ultimately physics limits what a sensor can sense and the remainder of the improvements will come from signal processing methods.

Useful imaging will likely top out around a million ISO at some point but how "print-worthy" this will be depends on how it can be processed and there won't be much DR. Sensing abilities of multi-mega ISO could provide basic rough (B&W) images.

 

[pedro]

Fiddled a bit more with my 5D3
Here are two samples
51k, some NR
http://www.flickr.com/photos/guatitamasluz/7925036148/#in/photostream
102k, NR: Brightness 11, Chroma: 20, darkened a bit
http://www.flickr.com/photos/guatitamasluz/7925050234/#in/photostream

B/W at these ISOs works quite well.

All I was looking for. Plenty of camera.

If we get some 25kish 51k some 6 years down, that'd be awesome! 8)

 

[TAF]

One need not be a certified optical engineer with a Ph.D to understand the concepts. Its pretty basic, and boils down to total light per time interval.

LARGE SNIP

Thank you very much for one of the most concise and detailed explanations of how the imaging process works at the sensor level!

 

[jrista]

Pedro's black and white shots bring another factor to light (sorry for the pun). I've been thinking about overall sensor efficiency so far, but we also have to factor in per-pixel efficiency. So long as we are involving color in our photographs, there will be some rather significant limits on how much light we can really capture. If we think about sensors more from the angle of astrophotography CCD's, many of which cost a couple thousand dollars (for what is basically a sensor and some readout electronics...far from the advanced machinery we get in a full DSLR), such sensors are often already pushing 80% Q.E. and often require cooling. Part of the reason they achieve such high resolutions and sensitivities is they are monochromatic devices.

In a Bayer sensor, we have a color filter over each pixel, which really limits how much light each pixel can receive, and intrinsically limits the total light the sensor can record. We might be able to achieve 50-60% Q.E. today on per-color basis, but 100% Q.E. for a red pixel...which ultimately means that a red pixel can convert every single photon that strikes it into an electron...is actually still 33% of the total light incident on that pixel. We would have to drop the color filter, and preferably drop any kind of low-pass filter and as much other filtration above the sensor as possible, to really push both per-pixel as well as overall sensor Q.E. to their maximums. With say multi-layered microlenses, low-noise electronics, efficient readout electronics that introduce little of their own noise, unity gain to eliminate quantization errors, backilluminated sensor design, in a monochrome sensor...well now we are really talking. Not only are we maximizing the surface area of the sensor for optimal light sensitivity, we also maximize resolution, or alternatively allow the use of pixels with four times as much surface area as a color sensor. With pixels twice as large, we can now gather four times as much light per pixel at effectively the same resolution as we had with a color sensor, so extremely high ISO at low signal levels should be much more viable. One could also design a sensor that could either alternate color filters over the sensor to record and blend a full color image at full resolution, or even use some kind if prism to split light to three sensors simultaneously to capture full color, full resolution images for the same exact exposure.

 

[jrista]

One need not be a certified optical engineer with a Ph.D to understand the concepts. Its pretty basic, and boils down to total light per time interval.

LARGE SNIP

Thank you very much for one of the most concise and detailed explanations of how the imaging process works at the sensor level!

Welcome.

 

[LetTheRightLensIn]

To the OP, if we take a look at Sensorgen.info data on quantum efficiency, we see it increases about 7-8% for every additional stop of ISO. If we follow that trend, assuming Canon can keep increasing Q.E. by about 8% per stop of ISO, and if they can maintain the rate of Q.E. improvement of 16% per 4.5-year cycle...then every 4.5 years we should see two more ISO levels. That would put the next generation at a native ISO 102400 on a sensor with about 57% Q.E. for the 5D IV, and if current statistics hold true, a native ISO of 204800 with about 60% Q.E. for the 1D XI.

Now, the real question is how usable that ISO will be. I am not sure a sustained linear increase of 8% per stop is really going to make an ISO 204800 usable. It'll be better than a digital boost ISO, but dynamic range is going to severely suffer and noise will still be horrendous. I would suspect that we won't see usable ISO 204800 until we are in the 80% Q.E. range, which would greatly improve S/N ratios even in the shadows. I don't know if we can really achieve that level of efficiency in a consumer-grade device, though...to date, its always required far more rigorous constraints on manufacturing quality, and usually requires some kind of active/thermoelectric cooling. Sony Exmor technology, combined with a backilluminated sensor and...really for honest usability...ungodly fast lenses...might make it a possibility. Either way...that would put usable ISO 204800 a good 8-10 years away at the earliest, or about two to two and a half product cycles.

You are being awfully generous there about actual progress made over those years if you are talking about middle gray SNR (they have cleaned up banding and ugly noise character and such a lot, which at times, can you lead to a larger usable increase in stops than the change in middle gray SNR alone would imply though, however even in that regard they have now made pretty huge strides so it will be hard to continue that trend now) which is leading to changes in stops being paired with changes in Q.E. that don't make any logical sense.

One thing with an almost ISOless sensor like Exmor you can capture using extreme under exposure at low ISO and then lift and get nearly as clean an image as shooting at a higher ISO but shooting low ISO with large underexposure means you still have the top end not chopped off, raising ISO a stop chops off a stop each time so if you applied some fancy warped tone curve maybe you can get some more DR out of things. I don't know how easy it is to do extreme tone mapping like that, I'm sure with careful work you could make it reasonably natural to an extent to definitely help a bit.

I'm not just thinking about Canon, I'm thinking more about the industry as a whole. In that same time, the Sony/Nikon alliance has made considerable strides in noise at all levels, and their Q.E. is already up to 57%. Assuming Canon is not just a dud these days, and that they are internally innovating, within a decade (a non-trivial long period of time), and an amalgamation of technological advances across the board (whatever Canon did to achieve native 51200, Sony Exmor tech, active/thermoelectric cooling, backillumination, and who knows what other advancements) I think we can see some pretty amazing things.

I think within the next decade or two we'll encounter some of the same problems the PC CPU industry had not too long ago...where they started making gates so small they had to start REALLY innovating to keep improving, doing things like stretching silicon, switching to copper wiring, improving their optical etching technology to produce a cleaner, finer beam, etc. I don't think it will be long until were at a similar level of advancement with digital sensors...they will hit a wall of physics, probably with ISO levels, that forces them to radically reinvent and hyperinnovate to either prove the problem is a physical limitation and nothing can be done about it, or work around the problem and improve in other ways.

In terms of QE in capture and conversion of the photos they can't do much better, it's not possible to get 1 stop better by improving QE 8%. That's only a fraction of what is needed to do one stop better in that regard.

I guess they could improve read noise more than I was first thinking though and bump DR at high ISO up a fair amount still which would help some scenes. So while they can't ever improve high ISO all that much more for the middle and brighter tones in a captured image they probably still have more room to clean up the darker tones and make them look a good deal better. It might be tricky to improve read electronics that much but would be possible at least.

 

[LetTheRightLensIn]

Charge multiplication could become a technical factor in providing black cat in a coal cellar shots.
CM in a solid state device like an image sensor could potentially deliver multiple electrons per incoming photon.

Charge multiplication won't do much to improve overall high-sensitivity SNR or DR but could provide a beefier signal that's easier to work with and would reduce the effect of read noise at those very high sensitivities.

As it is, QE is already reasonable, some mfrs already have really low read noise and, as some have explained in previous posts (good analogy of watching raindrops on concrete), we're already playing with individual photons and electrons so high ISO images are going to look noisy unless there's a clever processing ability to smooth out that randomness for cleaner looking output. That has its limits too since DR naturally reduces as sensitivity increases, to the point of it becoming a matter of a pixel being ON or OFF.

There's a lot of areas for incremental improvement in the present image sensor paradigm but ultimately physics limits what a sensor can sense and the remainder of the improvements will come from signal processing methods.

Useful imaging will likely top out around a million ISO at some point but how "print-worthy" this will be depends on how it can be processed and there won't be much DR. Sensing abilities of multi-mega ISO could provide basic rough (B&W) images.

Yeah, I think they already have some stuff that can kick out more than one electron. So yeah there is still probably more to go in terms of making the dark parts of high iso images not look at ugly than I had said, but as you say for mid to upper tones you won't see that get much better and even lower mids will be tricky to fix up much.

They have made good strides in fixing the dark tones and still have quite some theoretical room there. ISO 3200 on 20D had any dark parts like large garbage, 5D2 had them look borderline-okish, 5D3 has them looking decent, D4 has them looking pretty decent. If you image has lots of darker areas they may be able to improve the usability some number of stops more over time. If the image is mostly mid and upper tones there is not so much room to get much better.

So i guess it will be a weird situation where many shots the high ISO might only become 1 stop better than the 5D3 but for others, in terms of usable look, maybe it could become 2.5-3.5 stops better. Already we see some of this in that for many shots the 5D3 looks only modestly better than the 5D2 at very high ISO and yet for others, the 5D2 gets so much junk in some parts, that even though main tones are only a little more noisy the overall look is ruined and the 5D3 may be 1-2 stops more usable for particular tricky shots than the 5D2.

 

[jrista]

To the OP, if we take a look at Sensorgen.info data on quantum efficiency, we see it increases about 7-8% for every additional stop of ISO. If we follow that trend, assuming Canon can keep increasing Q.E. by about 8% per stop of ISO, and if they can maintain the rate of Q.E. improvement of 16% per 4.5-year cycle...then every 4.5 years we should see two more ISO levels. That would put the next generation at a native ISO 102400 on a sensor with about 57% Q.E. for the 5D IV, and if current statistics hold true, a native ISO of 204800 with about 60% Q.E. for the 1D XI.

Now, the real question is how usable that ISO will be. I am not sure a sustained linear increase of 8% per stop is really going to make an ISO 204800 usable. It'll be better than a digital boost ISO, but dynamic range is going to severely suffer and noise will still be horrendous. I would suspect that we won't see usable ISO 204800 until we are in the 80% Q.E. range, which would greatly improve S/N ratios even in the shadows. I don't know if we can really achieve that level of efficiency in a consumer-grade device, though...to date, its always required far more rigorous constraints on manufacturing quality, and usually requires some kind of active/thermoelectric cooling. Sony Exmor technology, combined with a backilluminated sensor and...really for honest usability...ungodly fast lenses...might make it a possibility. Either way...that would put usable ISO 204800 a good 8-10 years away at the earliest, or about two to two and a half product cycles.

You are being awfully generous there about actual progress made over those years if you are talking about middle gray SNR (they have cleaned up banding and ugly noise character and such a lot, which at times, can you lead to a larger usable increase in stops than the change in middle gray SNR alone would imply though, however even in that regard they have now made pretty huge strides so it will be hard to continue that trend now) which is leading to changes in stops being paired with changes in Q.E. that don't make any logical sense.

One thing with an almost ISOless sensor like Exmor you can capture using extreme under exposure at low ISO and then lift and get nearly as clean an image as shooting at a higher ISO but shooting low ISO with large underexposure means you still have the top end not chopped off, raising ISO a stop chops off a stop each time so if you applied some fancy warped tone curve maybe you can get some more DR out of things. I don't know how easy it is to do extreme tone mapping like that, I'm sure with careful work you could make it reasonably natural to an extent to definitely help a bit.

I'm not just thinking about Canon, I'm thinking more about the industry as a whole. In that same time, the Sony/Nikon alliance has made considerable strides in noise at all levels, and their Q.E. is already up to 57%. Assuming Canon is not just a dud these days, and that they are internally innovating, within a decade (a non-trivial long period of time), and an amalgamation of technological advances across the board (whatever Canon did to achieve native 51200, Sony Exmor tech, active/thermoelectric cooling, backillumination, and who knows what other advancements) I think we can see some pretty amazing things.

I think within the next decade or two we'll encounter some of the same problems the PC CPU industry had not too long ago...where they started making gates so small they had to start REALLY innovating to keep improving, doing things like stretching silicon, switching to copper wiring, improving their optical etching technology to produce a cleaner, finer beam, etc. I don't think it will be long until were at a similar level of advancement with digital sensors...they will hit a wall of physics, probably with ISO levels, that forces them to radically reinvent and hyperinnovate to either prove the problem is a physical limitation and nothing can be done about it, or work around the problem and improve in other ways.

In terms of QE in capture and conversion of the photos they can't do much better, it's not possible to get 1 stop better by improving QE 8%. That's only a fraction of what is needed to do one stop better in that regard.

I guess they could improve read noise more than I was first thinking though and bump DR at high ISO up a fair amount still which would help some scenes. So while they can't ever improve high ISO all that much more for the middle and brighter tones in a captured image they probably still have more room to clean up the darker tones and make them look a good deal better. It might be tricky to improve read electronics that much but would be possible at least.

Well, according to DxO data (which is what Sensorgen.info is and data I believe you revere), Canon has achieved approximately 1-stop of native ISO improvement per approximately 8% Q.E. improvement. Read noise only affects most digital sensors at low ISO. At ISO 100 its usually very high, at ISO 200 its high, and at ISO 400 it is pretty low. Beyond that, "read" noise is minimal to barely registering, and for Canon is as good as read noise in any Exmor sensor at any ISO. Read noise is not a factor at ISO above 400, the very very vast majority of noise at those ISO's comes from photon shot noise, more so at a lower image signal level than a higher one. When your at ISO's as high as 12800 and beyond, assuming your saturating (and if not, you should be using a higher ISO!!), improvements in read noise will only have a minimal impact the overall results. They are already only a couple electrons worth, and with cooling, that couple electrons could drop to a fraction of an electron (well below the levels currently achieved by Sony Exmor, which achieves the same 2-3 electrons worth that Canon sensors do at ISO above 400.)

At extremely high ISO, what really matters is signal-to-noise ratio, S/N. In this respect, I think we are on the same page...what really matters is boosting the S/N for low signal. If you are already at a rough floor for electronic noise, which so far demonstrated seems to be 2-3 electrons worth unless you use extreme cooling (i.e. -80°C), then it doesn't really matter what your dynamic range is (the ratio of saturation luminance to average noise signal). The only way to improve that, in a given short time interval, is to increase the number of incident photons that actually reach the sensor into electrons representing image signal. Thats quantum efficiency. Larger pixels, backilluminated pixels, monochromatic pixels all improve the ratio of incident photons that actually get converted into electrons. And that helps improve S/N at low signal in particular. If we can capture one photon out of 2.5 right now, and convert it into an electron, and we have a gain at high ISO of some small fraction...say 0.4, then we convert each converted photon into 2.5 digital levels. A non-unity gain will always introduce quantization noise (which is a small factor of noise, but one that can be eliminated with care). We'll be lacking tonal fidelity and detail definition. What if we could capture 2 out of those 2.5 photons? Or 3 out of every 4 photons? Or 5 out of every 6? Quantum efficiency at high ISO is all about improving LOW SIGNAL to NOISE ratio, since all you have is low signal. A higher Q.E., in the face of minimal room-temperature read noise, is the only thing that is going to improve the quality of high ISO.

If we throw in cooling and reduce consumer-grade sensor tech to sub-freezing temperatures, that will certainly help as well. It will not only reduce dark current in the circuit by a significant degree, but it will improve the conversion efficiency of the photodiode as well (which seems to peak at -80°c), meaning fewer photons will convert to heat instead of an electron. That also helps boost Q.E. The dynamic range achieved by Sony Exmor is only a great feat at low ISO settings. The only reason it does anything to dynamic range is it reduced read noise at ISO 100 and 200 from many electrons to the high-ISO norm. Instead of wasting the extra bits their 14-bit image processing engine offered, they stopped wasting and put them to good use. But the waste didn't exist in the first place at high ISO, so evoking Sony Exmor as the solution to the high ISO problem won't take us very far. We need to stop wasting incident photons, and put more of those incident photons to good use just like we stopped introducing too many non-signal electrons.

 

[qwerty]

Going from a quantum efficiency of ~50% (where we are today) to 100% yields a 3 dB reduction in noise at a given ISO (measured by standard deviation) for a given image size (1). Alternatively you could bump the ISO by one stop and have the same amount of noise. So, to answer the OP, raw files will get at most one more stop at high ISO, or about the difference between the 5D classic and the 5D III. This is the decrease in noise coming out of the sensor. Image processing will also give a benefit; I can not even begin to guess how much. Any image processing geeks out there who can chime in on that?

As was mentioned before, read noise and full well capacity are not the dominant contribution to noise at high ISOs. However, as a sidenote if you can get read noise low enough (full well capacity high enough) you have a sensor where ISO is basically meaningless (2), as you can just underexpose and push (pull) in post (as Canon does for H1 and H2) as opposed to using an amplifier. That doesn't help with noise at high ISOs, but it means setting the ISO is one less thing for the shooter to worry about.

(1) The photon shot noise SNR will scale with the inverse square root of the number of detected photons. I am also not taking into account the light blockage by the color filters; you can trade off color accuracy and SNR. I think that the sensorgen numbers for QE are such that 100% QE would result in a black & white image with a Bayer filter, but am not sure.
(2) Yeah, you still have the potential for discretization error since you have a fixed number of bits to represent the signal, so you might not be able to do away with ISO entirely.

 

[jrista]

Going from a quantum efficiency of ~50% (where we are today) to 100% yields a 3 dB reduction in noise at a given ISO (measured by standard deviation) for a given image size (1). Alternatively you could bump the ISO by one stop and have the same amount of noise. So, to answer the OP, raw files will get at most one more stop at high ISO, or about the difference between the 5D classic and the 5D III. This is the decrease in noise coming out of the sensor. Image processing will also give a benefit; I can not even begin to guess how much. Any image processing geeks out there who can chime in on that?

There are some pretty amazing things being done on the wavelet deconvolution front for noise reduction. There are papers and patents covering gaussian, poisson, and banding noise removal with wavelet deconv algorithms. The banding noise removal is some of the more amazing stuff...when images are decomposed into component wavelets, it is pretty amazing the things you can identify and subtract, with little impact to the other aspects of the image. I foresee a small revolution on the noise removal front once wavelet deconvolution denoise algorithms hit the mainstream. It might not matter a wit how much noise our images have...if we have software that can identify and/or predict the noise patterns and distribution, and eliminate it with minimal or no impact to useful image data...the world will be bliss.

(1) The photon shot noise SNR will scale with the inverse square root of the number of detected photons. I am also not taking into account the light blockage by the color filters; you can trade off color accuracy and SNR. I think that the sensorgen numbers for QE are such that 100% QE would result in a black & white image with a Bayer filter, but am not sure.

I would agree, 100% Q.E. would effectively necessitate no CFA...otherwise, each pixel is only getting about 1/3rd of the light potential at each photosite. You might be able to futz with the meaning of "Q.E."...as in redefine 100% Q.E. to mean all of the light of a certain frequency range, but that is ultimately cheating IMO.

(2) Yeah, you still have the potential for discretization error since you have a fixed number of bits to represent the signal, so you might not be able to do away with ISO entirely.

Quantization error (which I believe is the official term for discretization error) or Quantization noise, is pretty minimal. You usually get a fraction of an electron's worth of noise every few pixel reads. Given that, I think the notion of an ISO-less sensor where read noise is a sufficiently inconsequential fraction of S/N (a state which I think Sony Exmor sensors are fairly close to) would still be possible despite quantization noise.

 

[jrista]

There are some pretty amazing things being done on the wavelet deconvolution front for noise reduction. There are papers and patents covering gaussian, poisson, and banding noise removal with wavelet deconv algorithms. The banding noise removal is some of the more amazing stuff...when images are decomposed into component wavelets, it is pretty amazing the things you can identify and subtract, with little impact to the other aspects of the image. I foresee a small revolution on the noise removal front once wavelet deconvolution denoise algorithms hit the mainstream. It might not matter a wit how much noise our images have...if we have software that can identify and/or predict the noise patterns and distribution, and eliminate it with minimal or no impact to useful image data...the world will be bliss.

If noise and signal are sufficiently "separated", then it is in principle possible to reduce one without affecting the other. Wavelets seems to be related to filterbanks and block-transforms (like FFT), and I fail to see the potential for ground-breaking stuff that could not have been done with filterbanks. Wavelets could still be a good transform for doing nice things, though.

In the general case, where noise and signal occupy the same parts of the frequency spectrum, the same spatial regions, and no parametric models can decribe the noise, the signal, or the noise vs signal dependency, I think it will be very hard for any kind of transform (linear or nonlinear) to reduce visible noise significantly while affecting signal insignificantly.

-h

Take a look at one paper that covers extremely clean banding noise removal with wavelet deconvolution:

http://lib.semi.ac.cn:8080/tsh/dzzy/wsqk/spie/vol6623/662316.pdf

I've never seen any denoise tool that could deband that well. I am not sure how wavelet deconvolution might compare to a filterbank (which I assume you are referring to analog filterbanks for, say, audio processing), all I do know is that some of the papers I've read demonstrate some amazing things like the one above. I've also got bookmarks somewhere to papers that describe how to remove photon shot noise via a predictive poisson algorithm that is able to nearly eliminate the most common form of noise in just about any photograph taken at any ISO setting.

So, wavelet denoise algorithms may not sound super cool like a filter bank or FFT (although FFT's are used in the article above to produce waveforms), but people do seem to be doing super cool things with them.

 

[K-amps]

Whoever will benefit from such a high ISO...

I think there will be a significant iso gain in the future, that's because Canon cut IS from their new 24-70ii - and that lens is designed to be sold for a long time to come as it's their new "standard" zoom. And the Canon techs will have more knowledge of future possibilities than us mere mortals.

So true.

Also as to who benefits from better high ISO? All of us...

It basically boils down to faster shutter speeds to freeze action by using cleaner higher ISO's...

Jrista... great explanations of the basics... even for a technophobe like me. I have a question...


1) What if they had a prism in front of 3 sensors splitting the beam into RGB and it hitting 3 sensors each without color filters, but capturing one color each and then all 3 colors are processed into an image. The Prism will absorb some light but could this be offset by the specialty sensors operating at 70-80 QE?

or

Can they make sensors that are natively wavelength band limited... ? With this arrangement no light robbing prism is needed, just smaller sensor, 3 (One for Red one for G and one for B etc) laid next to each other using the same/ similar processing algo's....

I think people smarter than my self would have thought of this by now.

 

[jrista]

Take a look at one paper that covers extremely clean banding noise removal with wavelet deconvolution:

http://lib.semi.ac.cn:8080/tsh/dzzy/wsqk/spie/vol6623/662316.pdf

I've never seen any denoise tool that could deband that well.

I will check out the link, but I am very sceptical about any paper showing amazing image processing results. It turns out that it is a lot easier to tailor an algorithm (using tons of manual labour) for a specific set of images/noise-profiles, than to design a more general algorithm. Think of how amazing results a gifted Photoshop-girl can achieve after multiple hours cleaning up one image.

I am not sure how wavelet deconvolution might compare to a filterbank (which I assume you are referring to analog filterbanks for, say, audio processing),

I was thinking about digital filterbanks, but the principle is the same. All linear transforms essentially consists of multiply and add operations.

So, wavelet denoise algorithms may not sound super cool like a filter bank or FFT (although FFT's are used in the article above to produce waveforms), but people do seem to be doing super cool things with them.

I think it is the other way around: wavelets sounds supercool, but really are highly related to digital filterbanks, FFTs/DCTs etc.

If you want to do really amazing stuff, I think you need to do non-linear signal processing. Problem is, no-one seems to really understand it

-h

Hmm, odd that people don't understand non-linear signal processing. I mean, any polynomial would be a non-linear function. I think maybe the trick is figuring out how to apply non-linear functions in a useful way...linear is easy. Non-linear can take on an infinite variety of forms, continuous or non-continuous, etc. I think its just that non-linear processing is more complex and requires more effort to effectively apply.

 

[Dark Reality]

All this tech info is beyond me, but it sounds like many people are saying, "it's as good as it's going to get basically." But that has been said countless times in history. Someone will figure out a way to fix current problems or work around them, and I certainly think that we will be seeing major improvements in iso. Maybe a stop better per year until people just don't need it.

 

[Trovador]

In 6 to 8 years you won't even need to take the lens cap off!

 

[bdunbar79]

In 6 to 8 years you won't even need to take the lens cap off!

Unless there's a light leak issue .