August 20, 2014, 12:53:46 AM

Author Topic: Canon Dual-Scale Column-Parallel ADC Patent  (Read 7106 times)

roguewave

  • PowerShot G1 X II
  • ***
  • Posts: 56
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #30 on: December 20, 2013, 02:27:08 PM »
I keep wondering what is going to happen in the future with dual-pixel technology. They have the ability to read both sides of the pixel seperately, I wonder how much work it would be to set the two sides to different ISO values, read them both, and combine the values for greatly expanded DR.

This would obviously require more computing power than just reading the sensor would, but comments out of Canon about the greater computational needs of future cameras ties in with this... I am really curious to see what happens with the 7D2..... It should be dual-pixel and dual processor (Digic6 or even 6+????) so it will be able to do a lot more computing than a 70D. The next year or so could be interesting.

They wouldn't need to bother with the dual-pixel approach with this patent. They simply read "the pixel" (regardless of whether it is a single photodiode, or two/four binned, whatever) with two different gain levels (different ISO settings, done simultaneously on different signals). This patent offers a much better way to solve the problem without resorting to "hackish" approaches like what ML did, or like what you suggest with reading one half the pixel at one ISO and the other half at another ISO (which wouldn't be nearly as good, since each half pixel is only getting half the light, so the half-reads would already be at a disadvantage large enough to completely eliminate any gains you might make with the dual-read process in the first place.)

Even better than simply reading half pixels at different ISO settings, this patent reads each pixel twice simultanesously at different gain levels, while also bringing the ADC on-die and column-parallelizing them, allowing them to run at a lower frequency, thus reducing their potential to add downstream noise. With column-parallel ADC, they could do what Sony Exmor does...per-column read tuning to eliminate vertical banding. It also brings in the benefit of shipping image data off the sensor in an error-correctable digital form, eliminating the chance that the data picks up even further noise as it travels along a high frequency bus and through a high frequency DIGIC chip. This patent would single-handedly solve a LOT of Canon's noise problems.

The only real difference between Canon's Dual-Scale CP-ADC patent and Exmor's is that Exmor uses digital CDS and digital amplification (basically, it is an entirely digital pipeline)...I see no mention of Canon's patent referring to digital data processing on-die. There are theoretically pros and cons to both digital and analog readout, so only time will tell (assuming Canon actually IMPLEMENTS this design sometime soon) whether Canon's approach produces results that are as good as Exmor or not. Sometimes it is easier, and more accurate/precise, to apply certain kinds of processing and filtering on an analog signal rather than digital bits.

Thank you for the explanation! I got the gist of it, but I still don't understand the basics: how is it possible to read the same pixel twice simultaneously? I thought you can't eat your cake and have it too :)? I mean, wouldn't the signal become weaker if you split it?

canon rumors FORUM

Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #30 on: December 20, 2013, 02:27:08 PM »

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3935
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #31 on: December 20, 2013, 02:31:50 PM »
I keep wondering what is going to happen in the future with dual-pixel technology. They have the ability to read both sides of the pixel seperately, I wonder how much work it would be to set the two sides to different ISO values, read them both, and combine the values for greatly expanded DR.

This would obviously require more computing power than just reading the sensor would, but comments out of Canon about the greater computational needs of future cameras ties in with this... I am really curious to see what happens with the 7D2..... It should be dual-pixel and dual processor (Digic6 or even 6+????) so it will be able to do a lot more computing than a 70D. The next year or so could be interesting.

They wouldn't need to bother with the dual-pixel approach with this patent. They simply read "the pixel" (regardless of whether it is a single photodiode, or two/four binned, whatever) with two different gain levels (different ISO settings, done simultaneously on different signals). This patent offers a much better way to solve the problem without resorting to "hackish" approaches like what ML did, or like what you suggest with reading one half the pixel at one ISO and the other half at another ISO (which wouldn't be nearly as good, since each half pixel is only getting half the light, so the half-reads would already be at a disadvantage large enough to completely eliminate any gains you might make with the dual-read process in the first place.)

Even better than simply reading half pixels at different ISO settings, this patent reads each pixel twice simultanesously at different gain levels, while also bringing the ADC on-die and column-parallelizing them, allowing them to run at a lower frequency, thus reducing their potential to add downstream noise. With column-parallel ADC, they could do what Sony Exmor does...per-column read tuning to eliminate vertical banding. It also brings in the benefit of shipping image data off the sensor in an error-correctable digital form, eliminating the chance that the data picks up even further noise as it travels along a high frequency bus and through a high frequency DIGIC chip. This patent would single-handedly solve a LOT of Canon's noise problems.

The only real difference between Canon's Dual-Scale CP-ADC patent and Exmor's is that Exmor uses digital CDS and digital amplification (basically, it is an entirely digital pipeline)...I see no mention of Canon's patent referring to digital data processing on-die. There are theoretically pros and cons to both digital and analog readout, so only time will tell (assuming Canon actually IMPLEMENTS this design sometime soon) whether Canon's approach produces results that are as good as Exmor or not. Sometimes it is easier, and more accurate/precise, to apply certain kinds of processing and filtering on an analog signal rather than digital bits.

Thank you for the explanation! I got the gist of it, but I still don't understand the basics: how is it possible to read the same pixel twice simultaneously? I thought you can't eat your cake and have it too :)? I mean, wouldn't the signal become weaker if you split it?

They aren't splitting it. I am not 100% exactly certain what they are doing, but from what I do understand, when a pixel is read, it is amplified twice, and the results of those different amplifications are transferred to the CP-ADC units simultaneously (on different channels). Same source pixel, two separate but full power signals, which are then blended together at conversion time. It is basically the same thing ML did, only with the appropriate dedicated hardware fabricated right into the sensor to do it right.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

roguewave

  • PowerShot G1 X II
  • ***
  • Posts: 56
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #32 on: December 20, 2013, 03:22:35 PM »
They aren't splitting it. I am not 100% exactly certain what they are doing, but from what I do understand, when a pixel is read, it is amplified twice, and the results of those different amplifications are transferred to the CP-ADC units simultaneously (on different channels). Same source pixel, two separate but full power signals, which are then blended together at conversion time. It is basically the same thing ML did, only with the appropriate dedicated hardware fabricated right into the sensor to do it right.

I assume there is something clever somewhere in the implementation. HDR has been around for a while, even before ML. It's hard to believe nobody thought earlier about pushing the process into the sensor instead of software.

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3935
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #33 on: December 20, 2013, 03:30:19 PM »
They aren't splitting it. I am not 100% exactly certain what they are doing, but from what I do understand, when a pixel is read, it is amplified twice, and the results of those different amplifications are transferred to the CP-ADC units simultaneously (on different channels). Same source pixel, two separate but full power signals, which are then blended together at conversion time. It is basically the same thing ML did, only with the appropriate dedicated hardware fabricated right into the sensor to do it right.

I assume there is something clever somewhere in the implementation. HDR has been around for a while, even before ML. It's hard to believe nobody thought earlier about pushing the process into the sensor instead of software.

I wouldn't call it HDR. HDR is a very misused term as it is. In its proper form, a High Dynamic Range image is an image with an EXCESSIVBLY HIGH dynamic range, stored as 32-bit floating point numbers with extremely fine precision and a dynamic range that could potentially equal thousands of stops (i.e. it can represent numbers from a couple billion down to billionths.)

HDR as it is commonly (mis)used simply refers to the mapping of tones into a limited dynamic range from a source file that might have slightly higher dynamic range. What Canon is doing isn't exactly HDR...it is a specialized read process that will allow them to better utilize the dynamic range they already have access to, but which is otherwise being diminished by read noise.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

Lawliet

  • 7D
  • *****
  • Posts: 353
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #34 on: December 20, 2013, 03:40:56 PM »

Thank you for the explanation! I got the gist of it, but I still don't understand the basics: how is it possible to read the same pixel twice simultaneously? I thought you can't eat your cake and have it too :)? I mean, wouldn't the signal become weaker if you split it?

The patent looks like a ramp ADC - they don't take the electrons out to count them, but use a voltage comparison. The unknown pile of e- on the right, you measure how long you have to add charge on the left side until both are equal(or the known one grows larger then the unknown). In theory nothing stops you from using multiple heaps that grow at different rates. You just have to keep crosstalk, external influences and such under control.

roguewave

  • PowerShot G1 X II
  • ***
  • Posts: 56
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #35 on: December 20, 2013, 03:52:06 PM »
I wouldn't call it HDR. HDR is a very misused term as it is. In its proper form, a High Dynamic Range image is an image with an EXCESSIVBLY HIGH dynamic range, stored as 32-bit floating point numbers with extremely fine precision and a dynamic range that could potentially equal thousands of stops (i.e. it can represent numbers from a couple billion down to billionths.)

HDR as it is commonly (mis)used simply refers to the mapping of tones into a limited dynamic range from a source file that might have slightly higher dynamic range. What Canon is doing isn't exactly HDR...it is a specialized read process that will allow them to better utilize the dynamic range they already have access to, but which is otherwise being diminished by read noise.

I didn't mean to call this process HDR in the strict sense - you're right, it is a misused term.

Regardless of the exact meaning, I was thinking of the common understanding of HDR along the lines of:
...HDR compensates for this loss of detail by capturing multiple photographs at different exposure levels and combining them to produce a photograph representative of a broader tonal range... (wikipedia)

I could be wrong, but isn't that the same idea? Creating the equivalent of two different exposures by applying two different gain levels and then combining them. The difference is pushing it onto the sensor rather than post-processing in software, so there is no need to take multiple shots at different exposure.

roguewave

  • PowerShot G1 X II
  • ***
  • Posts: 56
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #36 on: December 20, 2013, 03:57:37 PM »
The patent looks like a ramp ADC - they don't take the electrons out to count them, but use a voltage comparison. The unknown pile of e- on the right, you measure how long you have to add charge on the left side until both are equal(or the known one grows larger then the unknown). In theory nothing stops you from using multiple heaps that grow at different rates. You just have to keep crosstalk, external influences and such under control.

Thank you, I think I got it!

canon rumors FORUM

Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #36 on: December 20, 2013, 03:57:37 PM »

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3935
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #37 on: December 20, 2013, 04:25:25 PM »
I wouldn't call it HDR. HDR is a very misused term as it is. In its proper form, a High Dynamic Range image is an image with an EXCESSIVBLY HIGH dynamic range, stored as 32-bit floating point numbers with extremely fine precision and a dynamic range that could potentially equal thousands of stops (i.e. it can represent numbers from a couple billion down to billionths.)

HDR as it is commonly (mis)used simply refers to the mapping of tones into a limited dynamic range from a source file that might have slightly higher dynamic range. What Canon is doing isn't exactly HDR...it is a specialized read process that will allow them to better utilize the dynamic range they already have access to, but which is otherwise being diminished by read noise.

I didn't mean to call this process HDR in the strict sense - you're right, it is a misused term.

Regardless of the exact meaning, I was thinking of the common understanding of HDR along the lines of:
...HDR compensates for this loss of detail by capturing multiple photographs at different exposure levels and combining them to produce a photograph representative of a broader tonal range... (wikipedia)

I could be wrong, but isn't that the same idea? Creating the equivalent of two different exposures by applying two different gain levels and then combining them. The difference is pushing it onto the sensor rather than post-processing in software, so there is no need to take multiple shots at different exposure.

Yeah, pretty much. I don't know exactly how they get the two reference signals, but in the end, the gain isn't huge. Canon sensors currently get around 11.5 stops on average. This could allow them to get ~13.5 stops on average unless they move to an ADC with a higher bit depth. If they do move beyond 14-bit ADC, then it would definitely be a lot more line hardware HDR (imagine 15.5 stops or around there for a 16-bit ADC.)
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3935
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #38 on: December 20, 2013, 04:49:54 PM »

Thank you for the explanation! I got the gist of it, but I still don't understand the basics: how is it possible to read the same pixel twice simultaneously? I thought you can't eat your cake and have it too :)? I mean, wouldn't the signal become weaker if you split it?

The patent looks like a ramp ADC - they don't take the electrons out to count them, but use a voltage comparison. The unknown pile of e- on the right, you measure how long you have to add charge on the left side until both are equal(or the known one grows larger then the unknown). In theory nothing stops you from using multiple heaps that grow at different rates. You just have to keep crosstalk, external influences and such under control.

Yeah, that sounds very much like what they are describing. Here is the actual abstract from the patent:

ABSTRACT

An image sensor comprises plural sets of a unit pixel outputting a pixel signal based on an electric charge generated through photoelectric conversion and a conversion unit converting the pixel signal into a digital signal. A reference signal source generates reference signals and supplies the generated reference signals to the conversion unit through signal lines. The conversion unit of each set comprises a comparator which compares the level of the reference signal with that of the pixel signal, a count circuit which counts a clock based on the comparison processing, a selection circuit which selects among the signal lines, a signal line to be selectively connected to the input of the comparator, and a switch which selectively connects the selected signal line to the input of the comparator, and selectively connects a load to an unselected one of the signal lines.


I am still not entirely certain I understand what the purpose of this is. I read embodiments three and four, and the summary in the last section of each always refers to increasing the accuracy of ADC. I am interpreting that to mean less noise, but I am not sure how much less noise. Here is the summary from embodiment thre:

[0086] As described above, by selecting, from two reference signals which have been offset from each other and have different voltage ramp gradients, a reference signal to be compared with a significant signal (not sure what this significant signal is -jrista), it is possible to shorten the conversion period as compared with a case in which one reference signal is used to perform A/D conversion. At this time, the load of the reference signal line is used for comparison processing, and the load variations of the reference signal line depending on the pixel signal are suppressed, thereby enabling to prevent the accuracy of A/D conversion from decreasing.

From what I understand about Canon noise in their current setup, the high frequency ADC in their DIGIC chips is a significant source of banding noise. I've assumed that this patent, by increasing the accuracy if ADC, would reduce that noise, thereby allowing a gain in DR. Based on what I read in embodiment three, I am not really sure whether that is the case or not.

I hadn't read much farther than that before, but reading into embodiments five, six, and seven, they start talking about inverting one of the reference signals during reset read which is applied to the reference signal during normal read. That sounds especially like what Sony does with Exmor for Digital CDS...but they don't actually call it that. They also state that an analog CDS is still supported, but not necessarily required.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

Lawliet

  • 7D
  • *****
  • Posts: 353
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #39 on: December 20, 2013, 07:26:12 PM »

I am still not entirely certain I understand what the purpose of this is.
Quote

It has two implications.  At ~64K e full well capacity and 16k/14bit resulution you get away with 4e steps for sampling. In the same timeframe you can sample the low quartille at 1e Resolution, cutting the shadow noise by a good margin. Engineering details have impact on the actual numbers...
The other one: the comperators change the signal sent from the sensor to the Digics from charge to time. Some stray electrons pushed around by environmental EM are less likely to change the value read.

From what I understand about Canon noise in their current setup, the high frequency ADC in their DIGIC chips is a significant source of banding noise. I've assumed that this patent, by increasing the accuracy if ADC, would reduce that noise, thereby allowing a gain in DR.
One prong gets us closer to the single electron counting/ISO-less readout, at least for pulling up shadows. No benefit for recovering blown out Highlights or spreading the midtones. We need some margins for whining I guess. ;)
The other should shift the main cause of banding from analog amplification and transmission to accuracy of Timing. The latter is much easier to handle, something at the clock rate of parts of current CPUs would do the trick. Good news: "normal" clock instabilities cause only a marginal drift of effective ISO, but at the same rate for all channels.

Question to be answered: how noise free are the comperators and their infrastructure?

horshack

  • PowerShot G1 X II
  • ***
  • Posts: 55
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #40 on: December 20, 2013, 07:29:44 PM »
Some of this patent seems to overlap what Emil Martinec came up with back in 2008:

http://www.dpreview.com/forums/post/28750076

LetTheRightLensIn

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3431
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #41 on: December 20, 2013, 08:56:02 PM »
Some of this patent seems to overlap what Emil Martinec came up with back in 2008:

http://www.dpreview.com/forums/post/28750076


yeah I thought of that as soon as I first saw this patent

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3935
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #42 on: December 20, 2013, 09:23:05 PM »
Some of this patent seems to overlap what Emil Martinec came up with back in 2008:

http://www.dpreview.com/forums/post/28750076


yeah I thought of that as soon as I first saw this patent


Seems similar, at least if I've read the patent correctly.
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

canon rumors FORUM

Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #42 on: December 20, 2013, 09:23:05 PM »

Aglet

  • 1D Mark IV
  • ******
  • Posts: 928
    • View Profile
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #43 on: December 21, 2013, 05:08:39 PM »
Today, in the photo world the tech gurus are predicting the death of the DSLR and saying the future will be mirrorless interchangeable lens cameras. But, to me, these seem like laptops. Too big to be truly portable, overpriced and with too many compromises to truly replace a DSLR.

I strongly suspect that in five years, the tech gurus will have moved on to the next big thing. Mirrorless will have run its course and the DSLR will still be plugging away because the form factor that has worked for 75 years remains the best form factor for its purpose.

I don't think you're correct on this ML prediction.
If you've tried a Olympus EM1 you'll see just how responsive and useful a good ML-EVF system can be and the tech's got some legs yet.

I don't think ML will "run its course." It will become an alternative to the traditional SLR type camera body.  Each will have their pros and cons and appeal to different consumer segments.

All the MFT cameras and Fuji's higher end bodies are proving they're very capable already.  It will only be a matter of another generation or 2 before they will likely outperform even the best DSLRs for shooting speed, both AF and fps.
Higher frame-rate EVFs with higher resolution will surely arrive though they're already adequate to rival optical VFs for functionality.  Battery drain will be improved, extending their operating duration.  These advances may even arrive from the traditional SLR mfrs first.  I'm sure they can see the foreshadowing such technology is having on them. PentNikCan have already ventured into ML categories, not very successfully, but they've gotten their toes wet and will have to continue and may even have to get competitive, at least in their own way, within the next couple years.

The future WILL carry on WITH mirrorless cameras.  The end-of-times for glass-flappers is nigh.
And I welcome the advantages it will bring.

jrista

  • Canon EF 300mm f/2.8L IS II
  • *******
  • Posts: 3935
  • POTATO
    • View Profile
    • Nature Photography
Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #44 on: December 21, 2013, 09:26:30 PM »
UPDATE

So, I've read through most of the patent's embodiments now. I am not sure that this is actually similar to what ML did. I do think it has to do with noise reduction, however it achieves it in a different way. From what I understand, this patent uses two signals of different ADC "precision", in comparison with the source pixel signal, and a set of circuitry to increase ADC speed, increase ADC accuracy, while maintaining a constant load level despite changing voltages.

The "constant load level" is what intrigued me the most. I believe it is a varying load level in Canon's current ADCs that leads to a bulk of their read noise. When load varies in an electrical circuit, it creates oscillations..."noise", a good example of which would be that electrical buzz in a DC circuit. If you can maintain a constant load, your noise level will drop considerably.

So, while this might not be as interesting as a Magic Lantern-style dual ISO read, I think it would still have the same effective result: Less read noise, more dynamic range at low ISO. The use of reference signals at different voltage ramps is simply to provide a secondary source for comparison with the actual pixel clock, and the option to select the more accurate signal...it really doesn't have anything to do with Dual ISO. I suspect that if Canon ever does pursue Dual ISO, the patent would probably refer more directly to such a mechanism...this patent only really directly referred to high and low precision ADC, constant load, and higher ADC accuracy...none of which really seemed to indicate ISO to me.
« Last Edit: December 21, 2013, 09:58:50 PM by jrista »
My Photography
Current Gear: Canon 5D III | Canon 7D | Canon EF 600mm f/4 L IS II | EF 100-400mm f/4.5-5.6 L IS | EF 16-35mm f/2.8 L | EF 100mm f/2.8 Macro | 50mm f/1.4
New Gear List: SBIG STT-8300M | Canon EF 300mm f/2.8 L II

canon rumors FORUM

Re: Canon Dual-Scale Column-Parallel ADC Patent
« Reply #44 on: December 21, 2013, 09:26:30 PM »