Canon Dual-Scale Column-Parallel ADC Patent

[jrista]

It's been a while since I last scanned through Image Sensors World blog. Around the beginning of August, as a matter of fact. Since that time, they noted that Canon filed for a "Dual Scale" CPADC patent:

http://image-sensors-world.blogspot.com.es/2013/08/canon-files-for-dual-range-column.html

If I understand the diagrams and the patent correctly, and I am no CMOS engineer, it sounds like Canon is maybe following ML's lead in using a dual gain (i.e. Dual ISO) approach to achieving higher dynamic range. Given how long it takes to produce technology viable enough for a patent, I suspect Canon had this idea long before ML...perhaps it was simply that ML got wind of this patent, and looked for a way to achieve the same thing with current Canon sensors...either way, interesting.

The more interesting thing to me than the dial gain, though, is the CP-ADC design. I've long said that Canon needs to modernize their sensor design, get rid of the noise generators (i.e. ADCs) in their DIGIC chips, and bring all that image processing onto the same die as the rest of the sensor. This is what Sony did (although they took it a step farther and converted to a digital readout/CDS approach, whereas as far as I can tell Canon's is still analog CDS and whatnot until it is actually converted to digital), and they achieved some significant DR benefits from the move.

Anyway, personally, I'm glad to hear Canon is investigating these options. CP-ADC is something I've wanted Canon to do for a long time, happy to see they might actually do it. God only knows if/when this technology may actually find it's way into their sensors...I only hope and pray it is soon. And dual-gain to boot...which has the potential to support FAR more than 14 stops of DR. With a 16-bit CP-ADC, we might even see a full 16 stops of DR (and who knows what might come after that...20-bit, 24-bit ADC? Can't imagine the file sizes though...46mp * 24bit...phew, 1.1Gb RAW (uncompressed) data size! Canon will need a DIGIC more than four times as fast as the current DIGIC chip...)

 

[CarlTN]

Very interesting, but recently you had said that if Canon relies on dual ISO, that's only a bandaid, and might not yield enough of a DR increase, at least with the combined benefit of a lower noise floor. Obviously you meant more akin to what ML did, rather than starting from quasi-scratch, as this link hints at.

It seems to me there will be a lot of lossless compression necessary for the large RAW files (and a lot of processing power). Also though, does this not make it likely, that the 2014 1-series camera, assuming it's in the 40MP range, may not use the above process? If so, it might just "only" have 14 bit RAW capability. I too was hoping it was actually going to be 16 bit, whether it actually got much over 14 stops of "real" DR or not. That would really be something, if Canon just suddenly introduced a camera that could actually do 16 stops.

Are you planning on buying the new camera, early on?

 

[jrista]

Very interesting, but recently you had said that if Canon relies on dual ISO, that's only a bandaid, and might not yield enough of a DR increase, at least with the combined benefit of a lower noise floor. Obviously you meant more akin to what ML did, rather than starting from quasi-scratch, as this link hints at.

Using the existing downstream amplifier on half the pixels, which is what ML is doing, is a bandaid (and not ideal, as it costs you in resolution). What Canon has patented here is MUCH better...the way I would expect it to be done. Since they are reading the sensor with two different gain levels, I really don't see why there would be any reasonable limits on DR for the foreseeable future...ML is only limited to 14 stops because the ADC is 14-bit. Technically, the potential for very scalable DR is there in Canon's patent (assuming I've understood it correctly, that is.)

It seems to me there will be a lot of lossless compression necessary for the large RAW files (and a lot of processing power). Also though, does this not make it likely, that the 2014 1-series camera, assuming it's in the 40MP range, may not use the above process? If so, it might just "only" have 14 bit RAW capability. I too was hoping it was actually going to be 16 bit, whether it actually got much over 14 stops of "real" DR or not. That would really be something, if Canon just suddenly introduced a camera that could actually do 16 stops.

Are you planning on buying the new camera, early on?

Agreed, normally a RAW file will have lossless compression. Still, a gigabit of information is a lot...you can't compress the read stream, really...you have to process it all in order to compress the output file. So, while from a storage space standpoint it wouldn't be all that bad, from an image processing standpoint...you would need much faster processors.

Canon, or someone, mentioned around a year ago, maybe not quite that long, that Canon might push a bit depth increase with the Big MP camera. Who knows if that is the case, it was a CR1, but still, interesting nevertheless. I can't imagine anyone pushing bit depth until there is a definitive reason to do so. For all of DXO's claims about the Nikon D800 and D600 offering more than 14 stops of DR, they are talking about downscaled output images. The native DR of the hardware itself is still less than 14 stops...13.2 for the D800 IIRC.

That's with 3e- of read noise...which is INSANELY LOW (usually, you don't see that kind of read noise until you start peltier cooling sensors to sub-freezing temperatures). There are a few new ideas floating about regarding how to reduce read noise. There have been a number of patents and other things floating around lately about "black silicon", a structural modification of silicon that gives it an extremely low reflectivity index, which supports a natural read noise level of around 2e- and some of the best low light sensitivity known, and it is being researched for use in extreme low light security cameras that can see by starlight (which blows my mind.) Theoretically, this can greatly improve DR at what would be high ISO settings.

Canon's approach with dual scaling is potentially another way to get a lot more average dynamic range at low or high ISO out of a single read by using two separate signals with different gain and sampling (I guess) to effectively do a low ISO and high ISO read at the same time for each pixel, and blend the results together using on-die CP-ADC.

As for new cameras...all that is on hold until I can get my business started and start making some money again. I don't have any plans to purchase anything at the moment, outside of possibly a 5D III if the price is right. I certainly won't be buying a 1D MPM (megapixel monster) any time soon if it hits with a price over $5k. Besides, I like to wait and see how things settle first...I am still interested in the 7D II, and want to wait for both cameras to hit the street and demonstrate their real-world performance before I make a decision.

 

[Don Haines]

I keep wondering what is going to happen in the future with dual-pixel technology. They have the ability to read both sides of the pixel seperately, I wonder how much work it would be to set the two sides to different ISO values, read them both, and combine the values for greatly expanded DR.

This would obviously require more computing power than just reading the sensor would, but comments out of Canon about the greater computational needs of future cameras ties in with this... I am really curious to see what happens with the 7D2..... It should be dual-pixel and dual processor (Digic6 or even 6+????) so it will be able to do a lot more computing than a 70D. The next year or so could be interesting.

 

[jrista]

I keep wondering what is going to happen in the future with dual-pixel technology. They have the ability to read both sides of the pixel seperately, I wonder how much work it would be to set the two sides to different ISO values, read them both, and combine the values for greatly expanded DR.

This would obviously require more computing power than just reading the sensor would, but comments out of Canon about the greater computational needs of future cameras ties in with this... I am really curious to see what happens with the 7D2..... It should be dual-pixel and dual processor (Digic6 or even 6+????) so it will be able to do a lot more computing than a 70D. The next year or so could be interesting.

They wouldn't need to bother with the dual-pixel approach with this patent. They simply read "the pixel" (regardless of whether it is a single photodiode, or two/four binned, whatever) with two different gain levels (different ISO settings, done simultaneously on different signals). This patent offers a much better way to solve the problem without resorting to "hackish" approaches like what ML did, or like what you suggest with reading one half the pixel at one ISO and the other half at another ISO (which wouldn't be nearly as good, since each half pixel is only getting half the light, so the half-reads would already be at a disadvantage large enough to completely eliminate any gains you might make with the dual-read process in the first place.)

Even better than simply reading half pixels at different ISO settings, this patent reads each pixel twice simultanesously at different gain levels, while also bringing the ADC on-die and column-parallelizing them, allowing them to run at a lower frequency, thus reducing their potential to add downstream noise. With column-parallel ADC, they could do what Sony Exmor does...per-column read tuning to eliminate vertical banding. It also brings in the benefit of shipping image data off the sensor in an error-correctable digital form, eliminating the chance that the data picks up even further noise as it travels along a high frequency bus and through a high frequency DIGIC chip. This patent would single-handedly solve a LOT of Canon's noise problems.

The only real difference between Canon's Dual-Scale CP-ADC patent and Exmor's is that Exmor uses digital CDS and digital amplification (basically, it is an entirely digital pipeline)...I see no mention of Canon's patent referring to digital data processing on-die. There are theoretically pros and cons to both digital and analog readout, so only time will tell (assuming Canon actually IMPLEMENTS this design sometime soon) whether Canon's approach produces results that are as good as Exmor or not. Sometimes it is easier, and more accurate/precise, to apply certain kinds of processing and filtering on an analog signal rather than digital bits.

 

[Don Haines]

I keep wondering what is going to happen in the future with dual-pixel technology. They have the ability to read both sides of the pixel seperately, I wonder how much work it would be to set the two sides to different ISO values, read them both, and combine the values for greatly expanded DR.

This would obviously require more computing power than just reading the sensor would, but comments out of Canon about the greater computational needs of future cameras ties in with this... I am really curious to see what happens with the 7D2..... It should be dual-pixel and dual processor (Digic6 or even 6+????) so it will be able to do a lot more computing than a 70D. The next year or so could be interesting.

They wouldn't need to bother with the dual-pixel approach with this patent. They simply read "the pixel" (regardless of whether it is a single photodiode, or two/four binned, whatever) with two different gain levels (different ISO settings, done simultaneously on different signals). This patent offers a much better way to solve the problem without resorting to "hackish" approaches like what ML did, or like what you suggest with reading one half the pixel at one ISO and the other half at another ISO (which wouldn't be nearly as good, since each half pixel is only getting half the light, so the half-reads would already be at a disadvantage large enough to completely eliminate any gains you might make with the dual-read process in the first place.)

Even better than simply reading half pixels at different ISO settings, this patent reads each pixel twice simultanesously at different gain levels, while also bringing the ADC on-die and column-parallelizing them, allowing them to run at a lower frequency, thus reducing their potential to add downstream noise. With column-parallel ADC, they could do what Sony Exmor does...per-column read tuning to eliminate vertical banding. It also brings in the benefit of shipping image data off the sensor in an error-correctable digital form, eliminating the chance that the data picks up even further noise as it travels along a high frequency bus and through a high frequency DIGIC chip. This patent would single-handedly solve a LOT of Canon's noise problems.

The only real difference between Canon's Dual-Scale CP-ADC patent and Exmor's is that Exmor uses digital CDS and digital amplification (basically, it is an entirely digital pipeline)...I see no mention of Canon's patent referring to digital data processing on-die. There are theoretically pros and cons to both digital and analog readout, so only time will tell (assuming Canon actually IMPLEMENTS this design sometime soon) whether Canon's approach produces results that are as good as Exmor or not. Sometimes it is easier, and more accurate/precise, to apply certain kinds of processing and filtering on an analog signal rather than digital bits.

Good explanation! Now I understand.... Thanks!

 

[Loswr]

Also, Canon has a prior Foveon-like patent, and now this patent. While there is certainly a large (sometimes insurmountable) gap between patent and product, these patents belie the statements those who suggest Canon is failing to innovate in the area of sensor design (as do prototypes like the 120 MP APS-H sensor).

 

[jrista]

I keep wondering what is going to happen in the future with dual-pixel technology. They have the ability to read both sides of the pixel seperately, I wonder how much work it would be to set the two sides to different ISO values, read them both, and combine the values for greatly expanded DR.

This would obviously require more computing power than just reading the sensor would, but comments out of Canon about the greater computational needs of future cameras ties in with this... I am really curious to see what happens with the 7D2..... It should be dual-pixel and dual processor (Digic6 or even 6+????) so it will be able to do a lot more computing than a 70D. The next year or so could be interesting.

They wouldn't need to bother with the dual-pixel approach with this patent. They simply read "the pixel" (regardless of whether it is a single photodiode, or two/four binned, whatever) with two different gain levels (different ISO settings, done simultaneously on different signals). This patent offers a much better way to solve the problem without resorting to "hackish" approaches like what ML did, or like what you suggest with reading one half the pixel at one ISO and the other half at another ISO (which wouldn't be nearly as good, since each half pixel is only getting half the light, so the half-reads would already be at a disadvantage large enough to completely eliminate any gains you might make with the dual-read process in the first place.)

Even better than simply reading half pixels at different ISO settings, this patent reads each pixel twice simultanesously at different gain levels, while also bringing the ADC on-die and column-parallelizing them, allowing them to run at a lower frequency, thus reducing their potential to add downstream noise. With column-parallel ADC, they could do what Sony Exmor does...per-column read tuning to eliminate vertical banding. It also brings in the benefit of shipping image data off the sensor in an error-correctable digital form, eliminating the chance that the data picks up even further noise as it travels along a high frequency bus and through a high frequency DIGIC chip. This patent would single-handedly solve a LOT of Canon's noise problems.

The only real difference between Canon's Dual-Scale CP-ADC patent and Exmor's is that Exmor uses digital CDS and digital amplification (basically, it is an entirely digital pipeline)...I see no mention of Canon's patent referring to digital data processing on-die. There are theoretically pros and cons to both digital and analog readout, so only time will tell (assuming Canon actually IMPLEMENTS this design sometime soon) whether Canon's approach produces results that are as good as Exmor or not. Sometimes it is easier, and more accurate/precise, to apply certain kinds of processing and filtering on an analog signal rather than digital bits.

Good explanation! Now I understand.... Thanks!

I just hope it finds its way into a Canon camera body soon. The patent was filed pretty recently, so I am pretty doubtful we would see it in the likes of say the Big MP camera, or even the 7D II. If Canon employs the technology, I suspect it would be in something like a next generation 1D X or maybe the 5D IV. Kind of a bummer, thinking that far out...but then again, we don't yet know what technology Canon HAS employed in either the 7D II or Big MP camera yet!

 

[Mt Spokane Photography]

Digic 6 is coming, we would likely see such a new converter in conjunction with a new processor. I'd also bet on dual pixel technology for most of the new cameras that come out. That technology has potential for producing mirrorless bodies that are very competitive with DSLR's. Fewer moving components in a camera body means more reliability. That flapping mirror is the cause of many issues in photography, but even so, it works and nothing has matched it yet.

 

[jrista]

Digic 6 is coming, we would likely see such a new converter in conjunction with a new processor. I'd also bet on dual pixel technology for most of the new cameras that come out. That technology has potential for producing mirrorless bodies that are very competitive with DSLR's. Fewer moving components in a camera body means more reliability. That flapping mirror is the cause of many issues in photography, but even so, it works and nothing has matched it yet.

Yeah, I bet we see DPAF in all new Canon bodies as well. I wonder if/when they will start improving that (QPAF?) The thing I want to see from Canon is something REALLY compelling on the EVF front. I can't even consider mirrorless, even with its advantages, until there is one HELL of an EVF to accompany it. Outside of landscapes, I rely so heavily on the viewfinder for everything else (even astrophotography...you MUST have an OVF to find and frame sky objects).

 

[Don Haines]

Digic 6 is coming, we would likely see such a new converter in conjunction with a new processor. I'd also bet on dual pixel technology for most of the new cameras that come out. That technology has potential for producing mirrorless bodies that are very competitive with DSLR's. Fewer moving components in a camera body means more reliability. That flapping mirror is the cause of many issues in photography, but even so, it works and nothing has matched it yet.

Yeah, I bet we see DPAF in all new Canon bodies as well. I wonder if/when they will start improving that (QPAF?) The thing I want to see from Canon is something REALLY compelling on the EVF front. I can't even consider mirrorless, even with its advantages, until there is one HELL of an EVF to accompany it. Outside of landscapes, I rely so heavily on the viewfinder for everything else (even astrophotography...you MUST have an OVF to find and frame sky objects).

digic6 has been out now for about half a year is a p/s.... Perhaps we get to see dual digic6 in the 7D2...

I agree about EVFs... They are the future, but not quite ready yet. There are a few nice ones starting to appear that are getting close, but they are not there yet.

 

[LetTheRightLensIn]

Yeah I posted this some months ago. I could swear you even commented on it then ;D . But I think the thread got quickly turned into a mess by all those calling it a troll thread and more DRibble and all that sort of nonsense and perhaps everyone forgot the basis of that thread.

We can just hope it is ready for the next main bodies.

 

[jrista]

Yeah I posted this some months ago. I could swear you even commented on it then ;D . But I think the thread got quickly turned into a mess by all those calling it a troll thread and more DRibble and all that sort of nonsense and perhaps everyone forgot the basis of that thread.

We can just hope it is ready for the next main bodies.

It's entirely possible I DID see it...but I've had a lot going on since August, and am still trying to get a business going, so it isn't out of the question that I forgot.

The only thing, tickling the back of my brain, that I worry about is Canon's pension for announcing really KICK-ASS things that...just disappear. Like a 120 MEGApixel APS-H sensor that could rip out 9.5 frames a second. I mean, come on. I wanted that years ago...I only want it even MORE so now. Why the hell isn't it in the Canon EOS Whoop-ASS Ds X yet?!??!?!?!!?!?!?!!!!!!! ;P

 

[jrista]

Digic 6 is coming, we would likely see such a new converter in conjunction with a new processor. I'd also bet on dual pixel technology for most of the new cameras that come out. That technology has potential for producing mirrorless bodies that are very competitive with DSLR's. Fewer moving components in a camera body means more reliability. That flapping mirror is the cause of many issues in photography, but even so, it works and nothing has matched it yet.

Yeah, I bet we see DPAF in all new Canon bodies as well. I wonder if/when they will start improving that (QPAF?) The thing I want to see from Canon is something REALLY compelling on the EVF front. I can't even consider mirrorless, even with its advantages, until there is one HELL of an EVF to accompany it. Outside of landscapes, I rely so heavily on the viewfinder for everything else (even astrophotography...you MUST have an OVF to find and frame sky objects).

digic6 has been out now for about half a year is a p/s.... Perhaps we get to see dual digic6 in the 7D2...

I agree about EVFs... They are the future, but not quite ready yet. There are a few nice ones starting to appear that are getting close, but they are not there yet.

If Canon does indeed move the ADC onto the sensor die, they would have to design a new DIGIC to pair it with, since the ADC currently lives inside the DIGIC chip up through DIGIC 6. So, if say the 7D II got this new DS/CP-ADC design...I would then expect it to have a DIGIC 7 paired with it. I would also expect that it would only need one DIGIC...the only reason the 7D, 1D IV and 1D X have had multiple DIGIC chips was to increase the number of ADC channels...with ADC on the sensor die, so long as the DIGIC 7's raw processing power was sufficient, you wouldn't even need two.

 

[CarlTN]

Very interesting, but recently you had said that if Canon relies on dual ISO, that's only a bandaid, and might not yield enough of a DR increase, at least with the combined benefit of a lower noise floor. Obviously you meant more akin to what ML did, rather than starting from quasi-scratch, as this link hints at.

Using the existing downstream amplifier on half the pixels, which is what ML is doing, is a bandaid (and not ideal, as it costs you in resolution). What Canon has patented here is MUCH better...the way I would expect it to be done. Since they are reading the sensor with two different gain levels, I really don't see why there would be any reasonable limits on DR for the foreseeable future...ML is only limited to 14 stops because the ADC is 14-bit. Technically, the potential for very scalable DR is there in Canon's patent (assuming I've understood it correctly, that is.)

It seems to me there will be a lot of lossless compression necessary for the large RAW files (and a lot of processing power). Also though, does this not make it likely, that the 2014 1-series camera, assuming it's in the 40MP range, may not use the above process? If so, it might just "only" have 14 bit RAW capability. I too was hoping it was actually going to be 16 bit, whether it actually got much over 14 stops of "real" DR or not. That would really be something, if Canon just suddenly introduced a camera that could actually do 16 stops.

Are you planning on buying the new camera, early on?

Agreed, normally a RAW file will have lossless compression. Still, a gigabit of information is a lot...you can't compress the read stream, really...you have to process it all in order to compress the output file. So, while from a storage space standpoint it wouldn't be all that bad, from an image processing standpoint...you would need much faster processors.

Canon, or someone, mentioned around a year ago, maybe not quite that long, that Canon might push a bit depth increase with the Big MP camera. Who knows if that is the case, it was a CR1, but still, interesting nevertheless. I can't imagine anyone pushing bit depth until there is a definitive reason to do so. For all of DXO's claims about the Nikon D800 and D600 offering more than 14 stops of DR, they are talking about downscaled output images. The native DR of the hardware itself is still less than 14 stops...13.2 for the D800 IIRC.

That's with 3e- of read noise...which is INSANELY LOW (usually, you don't see that kind of read noise until you start peltier cooling sensors to sub-freezing temperatures). There are a few new ideas floating about regarding how to reduce read noise. There have been a number of patents and other things floating around lately about "black silicon", a structural modification of silicon that gives it an extremely low reflectivity index, which supports a natural read noise level of around 2e- and some of the best low light sensitivity known, and it is being researched for use in extreme low light security cameras that can see by starlight (which blows my mind.) Theoretically, this can greatly improve DR at what would be high ISO settings.

Canon's approach with dual scaling is potentially another way to get a lot more average dynamic range at low or high ISO out of a single read by using two separate signals with different gain and sampling (I guess) to effectively do a low ISO and high ISO read at the same time for each pixel, and blend the results together using on-die CP-ADC.

As for new cameras...all that is on hold until I can get my business started and start making some money again. I don't have any plans to purchase anything at the moment, outside of possibly a 5D III if the price is right. I certainly won't be buying a 1D MPM (megapixel monster) any time soon if it hits with a price over $5k. Besides, I like to wait and see how things settle first...I am still interested in the 7D II, and want to wait for both cameras to hit the street and demonstrate their real-world performance before I make a decision.

Very informative points, thank you. And I think it was you who first mentioned "black silicon" on here earlier this year. I recall trying to read more about it, probably a link you posted. I think I read something on Wikipedia about it as well, for what little that is worth.

Thanks for pointing out that the compression would be useless during the read and processing stage. I knew that but hadn't even considered it...I was just thinking of the large files being written to a storage media of some kind. It almost seems like the high processing power is more achievable than the speed required to write and store the files, say while at 5 frames a second or more. You would need large internal buffer capacity. I suppose some kind of wireless technique could be used to write very large files quickly to an external computer, or watch phone or something...haha! I guess it would all get designed to work, if the need for really large files came to the fore...or rather when it does.

 

[CarlTN]

Also, Canon has a prior Foveon-like patent, and now this patent. While there is certainly a large (sometimes insurmountable) gap between patent and product, these patents belie the statements those who suggest Canon is failing to innovate in the area of sensor design (as do prototypes like the 120 MP APS-H sensor).

But was that 120MP aps-h sensor ever tested? What process was used to produce the 120MP sensor? All I've seen on here is how Canon's process hasn't gotten small enough, but something must have been small to make that sensor.

As you might know I'm a bit of a fan of the foveon technique, so if Canon actually produces one for sale, maybe it will perform really well. It does seem almost plausible to me that the 60 or 75MP sensors that have been rumored, would use the technique...Because at this point, is a bayer RGB array with that many photo sites really viable on a 36mm wide sensor? Seems like that would descend into the noise levels of compact point and shoot sensors...

 

[jrista]

Also, Canon has a prior Foveon-like patent, and now this patent. While there is certainly a large (sometimes insurmountable) gap between patent and product, these patents belie the statements those who suggest Canon is failing to innovate in the area of sensor design (as do prototypes like the 120 MP APS-H sensor).

But was that 120MP aps-h sensor ever tested? What process was used to produce the 120MP sensor? All I've seen on here is how Canon's process hasn't gotten small enough, but something must have been small to make that sensor.

As you might know I'm a bit of a fan of the foveon technique, so if Canon actually produces one for sale, maybe it will perform really well. It does seem almost plausible to me that the 60 or 75MP sensors that have been rumored, would use the technique...Because at this point, is a bayer RGB array with that many photo sites really viable on a 36mm wide sensor? Seems like that would descend into the noise levels of compact point and shoot sensors...

It was most certainly tested. That was the entire point of the press release now so many years ago...that they had successfully fabricated AND tested a 120mp APS-H sensor that was capable of 9.5 frames per second. It was an amazing feat. As for process, it would have had to have been done on their small form factor fab, as the pixels would have been only 2µm in size (too small for a 500nm process to effectively create, especially with the added logic for column-parallel readout (which the press release did mention.) Canon does have the capacity to fabricate larger sensors in multiple exposures on the fab that is dedicated to their smaller parts. It isn't particularly efficient, but that doesn't matter when you are only creating a few prototype parts for testing.

 

[jrista]

Very interesting, but recently you had said that if Canon relies on dual ISO, that's only a bandaid, and might not yield enough of a DR increase, at least with the combined benefit of a lower noise floor. Obviously you meant more akin to what ML did, rather than starting from quasi-scratch, as this link hints at.

Using the existing downstream amplifier on half the pixels, which is what ML is doing, is a bandaid (and not ideal, as it costs you in resolution). What Canon has patented here is MUCH better...the way I would expect it to be done. Since they are reading the sensor with two different gain levels, I really don't see why there would be any reasonable limits on DR for the foreseeable future...ML is only limited to 14 stops because the ADC is 14-bit. Technically, the potential for very scalable DR is there in Canon's patent (assuming I've understood it correctly, that is.)

It seems to me there will be a lot of lossless compression necessary for the large RAW files (and a lot of processing power). Also though, does this not make it likely, that the 2014 1-series camera, assuming it's in the 40MP range, may not use the above process? If so, it might just "only" have 14 bit RAW capability. I too was hoping it was actually going to be 16 bit, whether it actually got much over 14 stops of "real" DR or not. That would really be something, if Canon just suddenly introduced a camera that could actually do 16 stops.

Are you planning on buying the new camera, early on?

Agreed, normally a RAW file will have lossless compression. Still, a gigabit of information is a lot...you can't compress the read stream, really...you have to process it all in order to compress the output file. So, while from a storage space standpoint it wouldn't be all that bad, from an image processing standpoint...you would need much faster processors.

Canon, or someone, mentioned around a year ago, maybe not quite that long, that Canon might push a bit depth increase with the Big MP camera. Who knows if that is the case, it was a CR1, but still, interesting nevertheless. I can't imagine anyone pushing bit depth until there is a definitive reason to do so. For all of DXO's claims about the Nikon D800 and D600 offering more than 14 stops of DR, they are talking about downscaled output images. The native DR of the hardware itself is still less than 14 stops...13.2 for the D800 IIRC.

That's with 3e- of read noise...which is INSANELY LOW (usually, you don't see that kind of read noise until you start peltier cooling sensors to sub-freezing temperatures). There are a few new ideas floating about regarding how to reduce read noise. There have been a number of patents and other things floating around lately about "black silicon", a structural modification of silicon that gives it an extremely low reflectivity index, which supports a natural read noise level of around 2e- and some of the best low light sensitivity known, and it is being researched for use in extreme low light security cameras that can see by starlight (which blows my mind.) Theoretically, this can greatly improve DR at what would be high ISO settings.

Canon's approach with dual scaling is potentially another way to get a lot more average dynamic range at low or high ISO out of a single read by using two separate signals with different gain and sampling (I guess) to effectively do a low ISO and high ISO read at the same time for each pixel, and blend the results together using on-die CP-ADC.

As for new cameras...all that is on hold until I can get my business started and start making some money again. I don't have any plans to purchase anything at the moment, outside of possibly a 5D III if the price is right. I certainly won't be buying a 1D MPM (megapixel monster) any time soon if it hits with a price over $5k. Besides, I like to wait and see how things settle first...I am still interested in the 7D II, and want to wait for both cameras to hit the street and demonstrate their real-world performance before I make a decision.

Very informative points, thank you. And I think it was you who first mentioned "black silicon" on here earlier this year. I recall trying to read more about it, probably a link you posted. I think I read something on Wikipedia about it as well, for what little that is worth.

Thanks for pointing out that the compression would be useless during the read and processing stage. I knew that but hadn't even considered it...I was just thinking of the large files being written to a storage media of some kind. It almost seems like the high processing power is more achievable than the speed required to write and store the files, say while at 5 frames a second or more. You would need large internal buffer capacity. I suppose some kind of wireless technique could be used to write very large files quickly to an external computer, or watch phone or something...haha! I guess it would all get designed to work, if the need for really large files came to the fore...or rather when it does.

When it comes to the processing power required to process the image on the sensor, it has to be uncompressed data. But not only that, it has to be uncompressed data PLUS overhead...there is always a certain amount of overhead, additional data, additional processing to combat one problem or another, etc. So while the data size may be a gigaBIT (about 140mb per image), the actual total amount of data read is going to be larger, maybe closer to 160m per image. If one wanted a high readout rate...say 9.5 fps, then the total throughput rate would need to be 1.6gigaBYTE per second!

 

[CarlTN]

Very interesting, but recently you had said that if Canon relies on dual ISO, that's only a bandaid, and might not yield enough of a DR increase, at least with the combined benefit of a lower noise floor. Obviously you meant more akin to what ML did, rather than starting from quasi-scratch, as this link hints at.

Using the existing downstream amplifier on half the pixels, which is what ML is doing, is a bandaid (and not ideal, as it costs you in resolution). What Canon has patented here is MUCH better...the way I would expect it to be done. Since they are reading the sensor with two different gain levels, I really don't see why there would be any reasonable limits on DR for the foreseeable future...ML is only limited to 14 stops because the ADC is 14-bit. Technically, the potential for very scalable DR is there in Canon's patent (assuming I've understood it correctly, that is.)

It seems to me there will be a lot of lossless compression necessary for the large RAW files (and a lot of processing power). Also though, does this not make it likely, that the 2014 1-series camera, assuming it's in the 40MP range, may not use the above process? If so, it might just "only" have 14 bit RAW capability. I too was hoping it was actually going to be 16 bit, whether it actually got much over 14 stops of "real" DR or not. That would really be something, if Canon just suddenly introduced a camera that could actually do 16 stops.

Are you planning on buying the new camera, early on?

Agreed, normally a RAW file will have lossless compression. Still, a gigabit of information is a lot...you can't compress the read stream, really...you have to process it all in order to compress the output file. So, while from a storage space standpoint it wouldn't be all that bad, from an image processing standpoint...you would need much faster processors.

Canon, or someone, mentioned around a year ago, maybe not quite that long, that Canon might push a bit depth increase with the Big MP camera. Who knows if that is the case, it was a CR1, but still, interesting nevertheless. I can't imagine anyone pushing bit depth until there is a definitive reason to do so. For all of DXO's claims about the Nikon D800 and D600 offering more than 14 stops of DR, they are talking about downscaled output images. The native DR of the hardware itself is still less than 14 stops...13.2 for the D800 IIRC.

That's with 3e- of read noise...which is INSANELY LOW (usually, you don't see that kind of read noise until you start peltier cooling sensors to sub-freezing temperatures). There are a few new ideas floating about regarding how to reduce read noise. There have been a number of patents and other things floating around lately about "black silicon", a structural modification of silicon that gives it an extremely low reflectivity index, which supports a natural read noise level of around 2e- and some of the best low light sensitivity known, and it is being researched for use in extreme low light security cameras that can see by starlight (which blows my mind.) Theoretically, this can greatly improve DR at what would be high ISO settings.

Canon's approach with dual scaling is potentially another way to get a lot more average dynamic range at low or high ISO out of a single read by using two separate signals with different gain and sampling (I guess) to effectively do a low ISO and high ISO read at the same time for each pixel, and blend the results together using on-die CP-ADC.

As for new cameras...all that is on hold until I can get my business started and start making some money again. I don't have any plans to purchase anything at the moment, outside of possibly a 5D III if the price is right. I certainly won't be buying a 1D MPM (megapixel monster) any time soon if it hits with a price over $5k. Besides, I like to wait and see how things settle first...I am still interested in the 7D II, and want to wait for both cameras to hit the street and demonstrate their real-world performance before I make a decision.

Very informative points, thank you. And I think it was you who first mentioned "black silicon" on here earlier this year. I recall trying to read more about it, probably a link you posted. I think I read something on Wikipedia about it as well, for what little that is worth.

Thanks for pointing out that the compression would be useless during the read and processing stage. I knew that but hadn't even considered it...I was just thinking of the large files being written to a storage media of some kind. It almost seems like the high processing power is more achievable than the speed required to write and store the files, say while at 5 frames a second or more. You would need large internal buffer capacity. I suppose some kind of wireless technique could be used to write very large files quickly to an external computer, or watch phone or something...haha! I guess it would all get designed to work, if the need for really large files came to the fore...or rather when it does.

When it comes to the processing power required to process the image on the sensor, it has to be uncompressed data. But not only that, it has to be uncompressed data PLUS overhead...there is always a certain amount of overhead, additional data, additional processing to combat one problem or another, etc. So while the data size may be a gigaBIT (about 140mb per image), the actual total amount of data read is going to be larger, maybe closer to 160m per image. If one wanted a high readout rate...say 9.5 fps, then the total throughput rate would need to be 1.6gigaBYTE per second!

And what type of device or computer is currently capable of that kind of throughput?

 

[jrista]

Very interesting, but recently you had said that if Canon relies on dual ISO, that's only a bandaid, and might not yield enough of a DR increase, at least with the combined benefit of a lower noise floor. Obviously you meant more akin to what ML did, rather than starting from quasi-scratch, as this link hints at.

Using the existing downstream amplifier on half the pixels, which is what ML is doing, is a bandaid (and not ideal, as it costs you in resolution). What Canon has patented here is MUCH better...the way I would expect it to be done. Since they are reading the sensor with two different gain levels, I really don't see why there would be any reasonable limits on DR for the foreseeable future...ML is only limited to 14 stops because the ADC is 14-bit. Technically, the potential for very scalable DR is there in Canon's patent (assuming I've understood it correctly, that is.)

It seems to me there will be a lot of lossless compression necessary for the large RAW files (and a lot of processing power). Also though, does this not make it likely, that the 2014 1-series camera, assuming it's in the 40MP range, may not use the above process? If so, it might just "only" have 14 bit RAW capability. I too was hoping it was actually going to be 16 bit, whether it actually got much over 14 stops of "real" DR or not. That would really be something, if Canon just suddenly introduced a camera that could actually do 16 stops.

Are you planning on buying the new camera, early on?

Agreed, normally a RAW file will have lossless compression. Still, a gigabit of information is a lot...you can't compress the read stream, really...you have to process it all in order to compress the output file. So, while from a storage space standpoint it wouldn't be all that bad, from an image processing standpoint...you would need much faster processors.

Canon, or someone, mentioned around a year ago, maybe not quite that long, that Canon might push a bit depth increase with the Big MP camera. Who knows if that is the case, it was a CR1, but still, interesting nevertheless. I can't imagine anyone pushing bit depth until there is a definitive reason to do so. For all of DXO's claims about the Nikon D800 and D600 offering more than 14 stops of DR, they are talking about downscaled output images. The native DR of the hardware itself is still less than 14 stops...13.2 for the D800 IIRC.

That's with 3e- of read noise...which is INSANELY LOW (usually, you don't see that kind of read noise until you start peltier cooling sensors to sub-freezing temperatures). There are a few new ideas floating about regarding how to reduce read noise. There have been a number of patents and other things floating around lately about "black silicon", a structural modification of silicon that gives it an extremely low reflectivity index, which supports a natural read noise level of around 2e- and some of the best low light sensitivity known, and it is being researched for use in extreme low light security cameras that can see by starlight (which blows my mind.) Theoretically, this can greatly improve DR at what would be high ISO settings.

Canon's approach with dual scaling is potentially another way to get a lot more average dynamic range at low or high ISO out of a single read by using two separate signals with different gain and sampling (I guess) to effectively do a low ISO and high ISO read at the same time for each pixel, and blend the results together using on-die CP-ADC.

As for new cameras...all that is on hold until I can get my business started and start making some money again. I don't have any plans to purchase anything at the moment, outside of possibly a 5D III if the price is right. I certainly won't be buying a 1D MPM (megapixel monster) any time soon if it hits with a price over $5k. Besides, I like to wait and see how things settle first...I am still interested in the 7D II, and want to wait for both cameras to hit the street and demonstrate their real-world performance before I make a decision.

Very informative points, thank you. And I think it was you who first mentioned "black silicon" on here earlier this year. I recall trying to read more about it, probably a link you posted. I think I read something on Wikipedia about it as well, for what little that is worth.

Thanks for pointing out that the compression would be useless during the read and processing stage. I knew that but hadn't even considered it...I was just thinking of the large files being written to a storage media of some kind. It almost seems like the high processing power is more achievable than the speed required to write and store the files, say while at 5 frames a second or more. You would need large internal buffer capacity. I suppose some kind of wireless technique could be used to write very large files quickly to an external computer, or watch phone or something...haha! I guess it would all get designed to work, if the need for really large files came to the fore...or rather when it does.

When it comes to the processing power required to process the image on the sensor, it has to be uncompressed data. But not only that, it has to be uncompressed data PLUS overhead...there is always a certain amount of overhead, additional data, additional processing to combat one problem or another, etc. So while the data size may be a gigaBIT (about 140mb per image), the actual total amount of data read is going to be larger, maybe closer to 160m per image. If one wanted a high readout rate...say 9.5 fps, then the total throughput rate would need to be 1.6gigaBYTE per second!

And what type of device or computer is currently capable of that kind of throughput?

The original SATA standard was capable of 1.5Gbit/s, SATA2 was capable of 3.0Gbit/s, and SATA3 is currently capable of 6.0Gbit/s. That would be one of the SLOWEST data transfer rates for modern computing devices. A modern CPU is capable of around 192Gbit/s data throughput on the CPU itself and along its primary buses. A modern GPU is capable of even higher transfer rates in order to process graphics at up to 144 times per second (on 144Hz computer screens), meaning several hundred million pixels at least to the tune of trillions of operations per second requiring data throughputs of hundreds of billions of bits.

In order to handle 120 or 144 frames per second on modern high framerate gaming and 3D screens at 2560x1600 or even 3840x2160 (4k) with 10-bit precision, you would need at least 11,943,936,000bit/s throughput rate from video card to screen. (This is, BTW, the next generation of hardware, already trickling onto the market...high end gaming and graphics computing hardware, running on next generation GPUs and on early 4k SuperHD screens, using interfaces like Thunderbolt, which so happens to operate via a single channel at 10Gbit/s for v1, and 20Gbit/s for v2 via "aggregate" channels.)

General computing currently is capable of very significant data throughput. With the next generation of GPU's paving the way for high performance 4k 3D gaming (and even multi-screen 3D gaming, at that!), the average desktop of 2015 and beyond should be able to handle 150mp image files as easily as they handle 20/30/50/80mp image files from DSLR and MFD cameras today.

Assuming a 120mp camera operates at 9.5fps for 16-bit image frames (just speculating), since Canon has at least demonstrated a sensor like that. The raw per-frame bit size at 16-bit is 1,920,000,000 bits (1.92Gigabit), divide by 8 for bytes, which comes to 240,000,000MB (240Megabyte). Multiply by 9.5 frames per second, and you have a total data throughput of 2.28GB (2.28Gigabyte) per second, or 18.2Gbit/s. A single Thunderbolt v2 aggregate channel would be sufficient to handle that kind of data throughput, and be capable of transferring a full 120mb RAW image onto a computer in around 1 second...assuming you had comparable memory card technology that could keep up (which certainly doesn't seem unlikely given the rate at which memory card speed is improving.)

The real question is, will onboard graphics processors, DSPs (or rather computing packages, as they are today...usually a DSP stacked with a general purpose processor like ARM and usually a specialized ultra high speed memory buffer), will be able to reach the necessary data throughput rates. As a matter of fact, they already operate at fairly decent speeds. A single 120mp frame is 240MB. With a pair of DIGIC5+, you would be able to process 2 120mp frames per second. The DIGIC5+ chip was about seven times faster than it's predecessor, DIGIC4. If we assume a similar jump for the next DIGIC part, it would be capable of processing 3.36GB/s, more than the necessary 2.28GB/s to process 120mp at 9.5 frames per second, and quite probably enough to handle around 11 frames per second (and still have room for the necessary overhead.)

Given that release cycles for interchangeable lens cameras is usually on the order of several years, we probably wouldn't see next generation memory card performance until 1D X Next and 5D Next ca. 2016 or 2017. Sadly, at least historically, DSLRs have lagged even farther behind in data transfer standards support, so it could very likely be that we don't see comparable interface support in DSLRs and other interchangeable lens parts until 2019/2020. :\ Which means, instead of being able to transfer our giant 100mp+ images in about one second each, we will still have to slog through imports at about a quarter the speed our desktop computer technology is capable of...but were all used to that already. ;P (Which, BTW, is one of the key reasons I believe desktop computers are a LONG way from being dead...they are still the pinnacle of computing technology, and no matter how popular ultra-portable tablets and convertibles are, I think most people still have and use a desktop computer with a trusty old keyboard and mouse for their truly critical work. Tablets and convertibles and phablets and phones simply augment our computing repertoire.)