June 19, 2018, 08:29:44 PM

Author Topic: Patent: Canon 5 Layer UV, IR, RGB Sensor  (Read 31635 times)

jrista

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 5319
  • EOL
    • Nature Photography
Re: Patent: Canon 5 Layer UV, IR, RGB Sensor
« Reply #60 on: June 30, 2014, 11:49:45 AM »
Pixel size doesn't matter for low light performance. Total sensor area and quantum efficiency matter. It doesn't matter how finely you divide the light your receiving and converting into free charge. If you increase the amount of light your receiving (more total sensor area) and increase the rate of incident photon strikes to electron conversions, then you have better high ISO performance. It wouldn't matter if you had 10mp, 50mp, 120mp, or 500mp.

I get this, but I've wondered whether there might be some truth to the myth, though not in the way many people imagine.  While I accept that your explanation is true, it applies when using identical tech throughout the sensor.  I've wondered whether it's disproportionately more expensive to make high-density sensors and whether some compromises would be made to keep the costs of the higher MP sensors within reason.   The practical result would be that higher MP had worse low-light performance, but only because it's not identical sensor tech.

There has certainly been a LOT of research into making smaller sensors (which pretty much always have smaller pixels) more sensitive to light. That research undoubtedly has cost billions. That said, most of the research into making better small pixels has been done to make ultra tiny sensors viable...the kinds of 1/3" down to around 1/8" sized sensors found in small compact cameras, tablets, phablets, phones, and every other device that uses a microscopic sensor. Each of those sensors is usually a tiny fraction of the cost of one APS-C or FF sensor, though, despite having considerably smaller pixels (between 1 to 2 microns these days, with a new generation of sub-micron pixel sensors coming very soon.)

<snip>

You've written about all that before and, again, I don't disagree with any of it.  I may not have made my point very clearly:  I'm not talking about R&D, but about actual production costs.  I presume that P&S sensors can tolerate a higher pixel defect rate than SLR-quality sensors, so yield is pretty high for those sensors.  I assume that keeping the defect rate down in order to get a reasonable yield is easier (hence cheaper) with recent, but not leading edge, technology.  It's my non-expert understanding that there are many refinements that occur to get a beautiful new design to produce a high yield.  I'm assuming that this problem is increased for smaller pitch pixels and the needed smaller circuitry.  Of course, once you get those production problems worked out the yield is comparable.

Regarding your very specific point, I'm honestly not sure. I don't know that the costs of fabricating a sensor with any of the modern pixel sizes is more expensive just because of the pixel sizes. Sensor transistors are pretty large these days...I mean, 180nm, 90nm? CPUs are using 22nm, with 14nm on the way. Yield certainly becomes a bigger issue with smaller pixels, however even sensors with pixels in the 1-2 micron range are still much more capable of handling defects than a processor. Based on ChipWorks analysis on multiple chips from camera manufacturers across the board, it is not uncommon for companies to farm out help for designing and fabricating their DSPs like DIGIC or EXPEED. On the other hand, Canon has continued to fabricate their own sensors.

Why hasn't Canon moved to a smaller process yet? Building a fab is a monster investment. That's hundreds of millions to a couple billion dollars just for one, depending on how small you need to fabricate transistors. I'm more inclined to assume Canon put that off simply because it's a gargantuan up-front outlay that they haven't critically needed until now. Up through the 5D III, I don't think Canon's use of an old 500nm fabrication process was hurting them. Now? I think that there is certainly a perception that Canon is really starting to lag behind the competition from a low-level technical standpoint, and spending a billion on a new high tech fab that can produce sensors with huge photodiodes and tiny transistors on large wafers is probably a worthwhile expenditure, both from a perceptual and technological standpoint. Would they do it because making sensors with smaller pixels is too expensive? They already make sensors with smaller pixels on a 180nm process using copper interconnects. They have been for years, and that is their cheaper fab (I think it's already on 300mm wafers). I think that there are increased costs when building any sensor, with any size pixels, when your just working out the kinks in a particular process. Whether the pixels are big or small, I think moving to a more modern, advanced fab capable of creating more sensors on larger wafers with smaller transistors will actually, in the long term, be a huge cost saver, even (and maybe particularly when) they move to much smaller pixels.

canon rumors FORUM

Re: Patent: Canon 5 Layer UV, IR, RGB Sensor
« Reply #60 on: June 30, 2014, 11:49:45 AM »

jrista

  • Canon EF 400mm f/2.8L IS II
  • *********
  • Posts: 5319
  • EOL
    • Nature Photography
Re: Patent: Canon 5 Layer UV, IR, RGB Sensor
« Reply #61 on: June 30, 2014, 12:21:25 PM »
Hi,
   Got one question to ask: Does readout noise increase when resolution increase?

   Have a nice day.

In practice, no. Actually, in practice, it often seems to be the opposite, particularly in the case of Canon. Canon sensors have historically seen a drop in read noise as pixel size shrinks. The 1D X has nearly 39e- read noise. The 5D III has 33.1e-, while the 7D and it's other 18mp siblings have about 8e- read noise. Sony Exmor sensors tend to have around 2.7e- to 3.3e- read noise at all ISO settings regardless of pixel size.

Read noise is not specifically dependent upon pixel size when it comes to many CCD astro imagers as well. Some older generation KAF (what were originally Kodak, and now TrueSense Imaging sensors) CCD sensors, like the KAF-11000 series, which have 9┬Ám pixels, used to have very high read noise...as much as around 40e-. Today, newer cameras with those same sensors have about 10e- read noise. Similarly, older KAF-8300 CCD cameras used to have 25-30e- read noise. Today, they often have as little as 7e- read noise.

Read noise is often a term used to describe the conglomeration of potential electronic sources of noise in a sensor. In more specific terms, read noise is the noise introduced by the readout electronics. Dark current is probably better kept separate (and in CCD cameras, read noise and dark current usually are specified independently). Dark current is a variable type of noise...it accumulates over time, and the rate of accumulation is ultimately dependent upon temperature. Dark current is a form of noise that is intrinsic to all electronics, including sensors, and is therefor the one type of noise that may specifically be caused by sensor electronics.

Read noise, on the other hand, is usually introduced by "downstream" electronics. When the sensor signal is shipped over a bus and through processing electronics, such as amplifiers (in the case of CCDs) and ADC units, that's where it can pick up a lot of noise. Higher frequency components tend to add more noise, and with the exception of a very few sensors, most use high frequency downstream ADC units in external components that are one of the primary sources of read noise, and one of the primary reasons that dynamic range often falls off and flattens out at lower ISO.

CCD cameras often resort to alternate readout speeds as a means of lowering read noise. Some CCD cameras might take several seconds to read out one frame, and one frame of significantly lower resolution (maybe as little as a few megapixels) compared to your average DSLR sensor. This very low readout rate can reduce read noise by a significant degree. When it comes to astrophotography, that is often not a big deal. You are usually going to be doing other things while readout is occurring anyway, such as dithering (basically, moving the mount ever so slightly to offset the star positions, which helps greatly in reducing random noise once you stack). In the case of DSLRs, where high frame rate, often as high as 10fps or so, is a critical feature, then high frequency components are to be expected, and so is an increase in read noise.

An increase in parallelism, or the number of units dedicated to high workload tasks, such as reading out pixels and converting them to digital units, is one way of reducing the need for high frequency output. Canon has also filed a patent for dual-scale ADC units, where they have the ability to switch to a slower readout rate, and thus a lower operating frequency, when a high readout rate is unnecessary. This could lead to even lower noise images for types of photography that are inherently "slow"...such as landscapes. The reduced noise leads to higher dynamic range, and everyone is happy.

So no...there really isn't any reason to link read noise with pixel size in general, and especially not with smaller pixel sizes. Read noise is a much more complex beast than that, and multiple things factor into determining how much read noise a given camera has.

Michael Clark

  • EOS Rebel SL2
  • ***
  • Posts: 87
  • Now we see through a glass, darkly...
Re: Patent: Canon 5 Layer UV, IR, RGB Sensor
« Reply #62 on: March 06, 2018, 11:00:28 PM »
Pixel size doesn't matter for low light performance. Total sensor area and quantum efficiency matter. It doesn't matter how finely you divide the light your receiving and converting into free charge. If you increase the amount of light your receiving (more total sensor area) and increase the rate of incident photon strikes to electron conversions, then you have better high ISO performance. It wouldn't matter if you had 10mp, 50mp, 120mp, or 500mp.

I get this, but I've wondered whether there might be some truth to the myth, though not in the way many people imagine.  While I accept that your explanation is true, it applies when using identical tech throughout the sensor.  I've wondered whether it's disproportionately more expensive to make high-density sensors and whether some compromises would be made to keep the costs of the higher MP sensors within reason.   The practical result would be that higher MP had worse low-light performance, but only because it's not identical sensor tech.

There has certainly been a LOT of research into making smaller sensors (which pretty much always have smaller pixels) more sensitive to light. That research undoubtedly has cost billions. That said, most of the research into making better small pixels has been done to make ultra tiny sensors viable...the kinds of 1/3" down to around 1/8" sized sensors found in small compact cameras, tablets, phablets, phones, and every other device that uses a microscopic sensor. Each of those sensors is usually a tiny fraction of the cost of one APS-C or FF sensor, though, despite having considerably smaller pixels (between 1 to 2 microns these days, with a new generation of sub-micron pixel sensors coming very soon.)

The reason those sensors have problems with noise, again, isn't because of the small pixels...its the small sensor area. They are WAY smaller than even an APS-C. A couple orders of magnitude smaller at least, if not many more. To have enough pixels to be useful on such small sensors, the pixels themselves have to be tiny. That doesn't increase noise...all it means is that the sensor is "resolving" and/or "exhibiting" noise at a higher frequency. Blend a 2x2 matrix of pixels together with a median algorithm, and you would have the same noise as a sensor with pixels twice as large (linearly, 4x as much area...again, assuming similar tech, however within a given generation of cameras, sensor tech is usually very similar). These tiny sensors in tiny cameras in all the tiny devices we have these days perform so well because they actually use significantly better technology that what is found in our DSLRs. These tiny cameras employ some cutting edge science to increase their light gathering capacity, increase photodiode surface area, increase quantum efficiency, use per-pixel memories to increase charge capacity, etc. If a full-frame DSLR had the same kind of technology as a 1/8" sensor, we would have something like a 864mp 15fps ISO 1.6 million megapixel MONSTER that used color splitting (rather than color filtration) with at least 24 stops of dynamic range thanks to multi-bucket memories, digital readouts, black silicon (basically silicon that uses the same general technology as nanocoated lens elements to eliminate reflection), and a host of other advancements. A full-frame sensor in a DSLR that used the same technology as the microsensor used in the upcoming iPhone or Android would be utterly mind blowing. (Not to mention space guzzling...we would need a new kind of storage technology to handle 2.7Gb per RAW. :))

BTW, when I talk about noise in this context, I am pretty much referring to random sources of noise. That is primarily photon shot noise, as well as a bit of random noise from dark current and the random component of read noise. Pattern noise, which is always due to the electronics, is a different story. That is a matter of specific technological construction, materials, and sensor design. Pattern noise is usually buried very deeply within the signal, though, and unless your lifting your shadows by many stops, it is usually a non-factor. Photon shot noise and dark current are really the big ones. In normal photography, dark current is pretty much inconsequential, as CDS takes care of it (in astrophotography, dark current can be your worst enemy, as it accumulates with time....ugh...)

Its this difference in noise frequency...all noise frequency, particularly random noise frequency, where image normalization matters (LTRLI will like this). Dynamic range is talked about a lot, however it's usually talked about in the context of editing latitide: "How many stops can I lift my shadows?" That is certainly a factor of dynamic range, and clearly the one that everyone cares about today. Increasing dynamic range in such a way that you gain editing latitude means reducing read noise such that the original RAW, unscaled or anything like that, has less noise in the shadows, thereby increasing the usable range of bit depth in the RAW image. Dynamic range is also affected by other sources of noise than just the pattern read noise, however. All random sources of noise affect it as well, though, and that includes random noise introduced during read as well as the primary source of random noise, photon shot noise.

In order to compare noise of cameras with different size sensors, one must normalize their outputs. Scale them to the same size. It really doesn't matter if you scale up or down, however scaling down to a common target is usually the approach taken. Assuming you downsampled the images from a number of cameras all with different sensor sizes, but all with the same pixel count, to the same image size, say an image with 2000 pixels on the long side, you'll find that the larger the sensor, the lower the noise. If we instead had a set of cameras where the larger sensors had fewer pixels and smaller sensors had more pixels, again we would still see that the larger sensor had less noise...however we would also find that the smaller sensors had more detail. The thing about detail is, especially when there is a lot of it, it tends to drown out noise. This is a perceptual matter...the noise of the smaller sensors with smaller pixels is still higher, statistically speaking (i.e. if it was measured), however that higher level of noise would be more readily recognized when it occurs in smooth areas, gradients and solid areas (i.e. background boke).

The perceptual factor is difficult to nail down, it's highly subjective, but it does play a role in whether we as humans THINK one camera is noisier than another. This is actually one of the big problems with the 7D. It still has a very high resolution sensor...it's pixels are still a lot smaller than those of the 5D III, 6D, and most other DSLRs on the market with the exception of less than a handful (i.e. the 70D, a couple Nikon APS-C cameras). The reason the 7D is perceived as noisy is because it has a tendency to be a bit soft. It's got a "strong" AA filter (personally, I think it's just right for the job it was designed to do, but it does blur more than a lot of AA filters on newer cameras these days), and that strong AA filter eliminates a certain amount of high frequency detail...high frequency detail that would otherwise drown out noise. (The other problem is that the 7D doesn't actually gather as much light as newer counterparts, even including some of the lower end Rebels that ended up with the same sensor...the 7D can only gather a charge of about 20ke- per pixel, vs. say the 70D, which gathers nearly 27ke- per pixel...per SMALLER pixel, which indicates the 70D is gathering almost 50% more light than the 7D within the same sensor area).  The 7D isn't necessarily much noisier than its counterparts and competitors...it just SEEMS noisier because it's a bit softer, and that softer detail has a harder time drowning out noise with meaningful information. I also think, in practice, that the 7D's noise is more difficult to clean up, as photon shot noise isn't "crisp" and just per-pixel...it kind of "bleeds" into multiple pixels (probably because of the AA filter).

Anyway, when it comes to sensors of the same size, the biggest differences are usually quantum efficiency and read noise (and, for some applications, dark current). The Sony Exmor, for example, is a superior sensor in all three of those categories. It has quite a bit more Q.E. than any Canon sensor (by as much as 15%), it has significantly lower read noise, and it actually also has less dark current (which only really matters for longer exposures.) Full frame Exmors are still the same area as the sensors in the 5D III and 1D X, but they gather a lot more light, and they introduce far less noise into the deep shadows. That's the only real difference. Assuming one created an exposure where the lowest pixel level was well above the read noise floor...you would find little of significant difference between cameras with these sensors that actually had anything to do with the sensor (you would find differences, but if you really looked into the reasons for those differences, I am willing to bet good money you would find the AF system, metering system, frame rate, and ability of the photographer to work quickly with the camera to change settings, find their subject, focus it, etc. as the key factors driving the differences in IQ.

I had an increasingly tough time with my 7D getting it to focus consistently...using the 5D III is EFFORTLESS...it practically works itself, and when I need to do anything, it's like it knows my mind. It's that factor right there, the ability to expend little effort using a camera to get good results, that makes Canon king of the DSLR. Canon is at the pinnacle of DSLR design. Their current generation of cameras are truly exquisite when it comes to making it easy, making it effortless, for the photographer to be a photographer, instead of a camera operator. I put off the 5D III for a good long while, largely because I wanted to see what the 7D II turned out to be. I rather regret that decision now, as even if the 7D II turns out to be phenomenal, and is just as effortless to use as the 5D III or 1D X...I spent an extra year hassling around with the 7D when I didn't really have to.

If you want low noise, go with a bigger frame, regardless of pixel count or size. If you want more detail, go with a smaller frame and more pixels. That's all that should really go into the decision making of whether to get a FF camera or an APS-C camera. Once you've picked one of those two things, then it's time to figure out what of all the other features will best serve your needs...and in my experience, it's all those other factors that are WAY, WAY more important. "Effortless"....that should really be Canon's new ad campaign. That's what Canon's current cameras do for you...they make photography effortless. I couldn't really give a crap about the minutia IQ when I can just point and shoot and the camera just does what I need it to.

Pixel size does affect full well capacity, which directly affects dynamic range.
It is one of the commonest of mistakes to consider that the limit of our power of perception is also the limit of all there is to perceive. - C. W. Leadbeater

5D3 7D2 5D2 EF70-200/2.8 II EF24-70/2.8 EF 135/2 EF50/1.4 EF100/2 EF24-105/4 EF17-40/4

canon rumors FORUM

Re: Patent: Canon 5 Layer UV, IR, RGB Sensor
« Reply #62 on: March 06, 2018, 11:00:28 PM »