Multilayer Sensors are Coming From Canon [CR2]

[jrista]

Guys, is it possible to get 16-bit images with FF DSLRs in theory? Would it give any real life benefit vs. 14-bit?

You won't benefit from a higher bit depth if your full well capacity in electrons is less than the maximum digital unit supported by the bit depth. With 14 bits, you can represent digital units from 0 through 2^14, or 16,384. With 16 bits, you can represent digital units from 0 through 2^16, or 65,536.

Most APS-C sensors don't have enough full-well electron charge to really benefit from 16-bit ADCs. Canon sensors top out at around 26,000e-. That's more than the 16k supported by 14 bits of data, but not enough more to warrant 65k. It may even be beneficial to "oversample" electrons, at base ISO, relative to the output bit depth. If you had ~32ke- FWC, you would effectively convert every two electrons into one digital unit. That's good dynamic range (oh please, don't let that be misinterpreted! ) A couple Nikon APS-C cameras have 30-40ke- FWC.

Full frame sensors, at least with current pixel sizes, gather a lot more charge per pixel than APS-C sensors. Most are over 55ke-, including Canon's older FF 1D series cameras. The 5D II had nearly 65ke- exactly, and even the old 5DC had over 55ke-. The 5D III, 6D, 1D X all have FWCs over 65ke-. The D800 (and A7r) have 45ke- FWC, which is on the lower side, but the D810 cranks it up to nearly 80ke- at ISO 64. The A7s has a whopping 155ke- FWC at ISO 100.

I'd say that most FF cameras could benefit from a 16-bit ADC. Even Canon cameras, which still have high read noise, can benefit. You won't see an editing latitude increase on a Canon camera (not with current read noise levels anyway), however overall, you should still see improved tonal grading. Convert 65, 80, 150 thousand electrons into 16k digital units, and your needlessly limiting your tonal range. Convert 65, 80, 150 thousand electrons into 65k digital units, and you greatly expand your tonal range...that should mean smoother gradients, softer shadow falloff (until you hit the read noise floor), etc.

So, assuming you have the electron charge capacity in each pixel to support it, you could benefit from 16-bit ADC. I don't think many APS-C sensors currently would really benefit. I think most FF sensors could benefit, especially those with really high FWC counts...the 1D X, the 6D, the A7s, the D810.

 

[Etienne]

Multi-layer, 16 bit, super charged bosons, or whatever. They could have jellyfish tentacles inside for all I care. It's a new tech, and when it hits the street we'll see if it redefines "awesome" or not.
What I do like, is that the leapfrog game of competition appears to be on, and Canon, as we all should know, is not sleeping while Sony pumps out new tech.

 

[BozillaNZ]

I'd say that most FF cameras could benefit from a 16-bit ADC. Even Canon cameras, which still have high read noise, can benefit.

You won't benefit anything by sampling the noise floor 4 times more precise. I could as well read the sensor at 12-bit and fill the lower 4-bits with random noise generator and get a "smoother tonal gradient".

 

[wtlloyd]

Thanks! That's gonna be some good readin'!

Here is one of the layered sensor patents from a few years ago (2011):

http://translate.google.com/translate?hl=en&sl=ja&u=http://egami.blog.so-net.ne.jp/2013-05-22&prev=/search%3Fq%3Dhttp://egami.blog.so-net.ne.jp/2013-05-22%26client%3Dfirefox-a%26hs%3DC0u%26rls%3Dorg.mozilla:en-USfficial

This one seems to apply the nanocoating concept to the red layer. Nanocoating uses nanoscopic scale spikes of differing sizes on a reflective surface to produce a non-abrupt transition layer. Reflections occur at abrupt transistions in refractive index, so by creating a non-abrupt transition layer, you can nearly eliminate reflections entirely. This is different from standard multicoating, which still allows reflections to occur, it just cancels them out via wave interference.

Here is another patent from 2012:

http://translate.google.com/translate?hl=en&sl=ja&u=http://egami.blog.so-net.ne.jp/2013-05-22&prev=/search%3Fq%3Dhttp://egami.blog.so-net.ne.jp/2013-05-22%26client%3Dfirefox-a%26hs%3DC0u%26rls%3Dorg.mozilla:en-USfficial

This is another sensitivity increasing patent. This apparently uses dielectric antireflective layers underneath the preceeding layer to reduce ghosting reflections. Not sure if this is intended to be used in conjunction with the nanocoating of the red layer or not...it seems to explicitly call out the blue and green layers (which are higher up than the red layer).

Canon also has their more recent patent for the five-layer sensor with UV and IR layers:

http://translate.google.com/translate?sl=auto&tl=en&u=http://egami.blog.so-net.ne.jp/2014-06-27

This patent is interesting, because it seems to depict a multi-layered BSI design, at least based on the diagram of the sensor (all the transistors are on the back side...that alone would be HUGE for layered sensor sensitivity...if you look at the ChipWorks electron micrographs of current Foveon designs, the transistors take up a huge amount of die space, as Foveon is still an FSI design...which is probably the biggest reason that sensor suffers so in low light.)

It was discovered some time ago that infrared light diffuses and reflects back subcutaneously in human skin. It can be used to greatly reduce the appearance of skin blemishes (I found a page a while back that shows that most skin features effectively disappear when you shoot full infrared). I'm not sure what UV light does for skin...apparently Canon found something useful with UV light.

Anyway, wtlloyd, there's some reference material. That's what Canon's got for multi-layered sensors.

 

[jrista]

I'd say that most FF cameras could benefit from a 16-bit ADC. Even Canon cameras, which still have high read noise, can benefit.

You won't benefit anything by sampling the noise floor 4 times more precise. I could as well read the sensor at 12-bit and fill the lower 4-bits with random noise generator and get a "smoother tonal gradient".

Your only thinking about the bottom range of the signal. Once your above the read noise floor, it's clean signal limited only by photon shot noise. You very much do still benefit from higher bit depth in that (very vast) range of signal.

Think about it. If read noise is 35e-, and the maximum signal strength is 68,000e-. If your doing 16-bit conversion, then your gain is 1.0376e-/ADU . That's almost unity gain...at ISO 100! Unity gain is what you want. With 14-bit conversion, your gain is 4.15e-/ADU. So, with 14-bit, your read noise turns into 8-9 tonal levels. With 16-bit, your read noise turns into 33-34 tonal levels. That's the bottom of the signal, though. For 16-bit, you still have 65502 levels for all the signal detail above the noise floor. Any gradients in the image at tones 35 through 65535 are going to be smoother with 16-bit conversion than with 14-bit conversion.

You don't gain in editing latitude...you would still have the same amount of dynamic range...but you do gain in tonal fidelity.

 

[V8Beast]

I'd say that most FF cameras could benefit from a 16-bit ADC. Even Canon cameras, which still have high read noise, can benefit. You won't see an editing latitude increase on a Canon camera (not with current read noise levels anyway), however overall, you should still see improved tonal grading. Convert 65, 80, 150 thousand electrons into 16k digital units, and your needlessly limiting your tonal range. Convert 65, 80, 150 thousand electrons into 65k digital units, and you greatly expand your tonal range...that should mean smoother gradients, softer shadow falloff (until you hit the read noise floor), etc.

Very interesting info. Other than higher bit ADC, what other methods can be used to improve tonal range?

I'm often shooting product images of engine or suspension parts that are some slightly different shade of gray, white, or black. Since I have 100 percent of the control over lighting, in these scenarios I'm far more interested in improvements in tonal range than dynamic range, although dynamic range is still a nice convenience.

Similar products shots I've seen captured with medium format gear absolutely kicks the $hit out of anything captured on 35mm sensors in terms of super fine tonal gradations. It makes me envious, but I don't shoot nearly enough product gigs like this to warrant investing in medium format.

 

[BozillaNZ]

Think about it. If read noise is 35e-, and the maximum signal strength is 68,000e-. If your doing 16-bit conversion, then your gain is 1.0376e-/ADU . That's almost unity gain...at ISO 100! Unity gain is what you want. With 14-bit conversion, your gain is 4.15e-/ADU. So, with 14-bit, your read noise turns into 8-9 tonal levels. With 16-bit, your read noise turns into 33-34 tonal levels. That's the bottom of the signal, though. For 16-bit, you still have 65502 levels for all the signal detail above the noise floor. Any gradients in the image at tones 35 through 65535 are going to be smoother with 16-bit conversion than with 14-bit conversion.

The noise floor does not only 'exist' in the lower-end of the signal range, it exists through-out the range event to the clip point.

Let's do a math exercise:

Here's a stream of signal:
0 10 100 1000 10000 100000

If my sensor has noise floor of 3-bits (0~8), even if I sample it using full precision, I got this:
3 12 107 1004 10002 100005, my lowest 3 bits are drown in noise, even for the highlight.

If I sample it with 1/8 precision (chops off lower 3 bits), I get:
0 1 13 125 1250 12500

Then recreate the signal by multiplying the sample signal by 8 times:
0 8 104 1000 10000 100000

And mix it with random number generator for 3-bits:
5 10 109 1001 10007 100004

The results of high and low precision sampling only fluctuates within noise floor, so are essentially the same.

The conclusion? If you are sampling more precision than your SNR, you are just sampling noise more precisely, which is still noise, and is same as if you don't sample as precise, then add noise in post.

Apply this to your example, when you have 35e- noise floor, your tonal range is not 65535-35 = 65500, but rather 65535/35 = 1872 levels (~10.8 stops), because the bottom 5 (!) bits are unstable noise.

 

[jrista]

Think about it. If read noise is 35e-, and the maximum signal strength is 68,000e-. If your doing 16-bit conversion, then your gain is 1.0376e-/ADU . That's almost unity gain...at ISO 100! Unity gain is what you want. With 14-bit conversion, your gain is 4.15e-/ADU. So, with 14-bit, your read noise turns into 8-9 tonal levels. With 16-bit, your read noise turns into 33-34 tonal levels. That's the bottom of the signal, though. For 16-bit, you still have 65502 levels for all the signal detail above the noise floor. Any gradients in the image at tones 35 through 65535 are going to be smoother with 16-bit conversion than with 14-bit conversion.

The noise floor does not only 'exist' in the lower-end of the signal range, it exists through-out the range event to the clip point.

Let's do a math exercise:

Here's a stream of signal:
0 10 100 1000 10000 100000

If my sensor has noise floor of 3-bits (0~8), even if I sample it using full precision, I got this:
3 12 107 1004 10002 100005, my lowest 3 bits are drown in noise, even for the highlight.

If I sample it with 1/8 precision (chops off lower 3 bits), I get:
0 1 13 125 1250 12500

Then recreate the signal by multiplying the sample signal by 8 times:
0 8 104 1000 10000 100000

And mix it with random number generator for 3-bits:
5 10 109 1001 10007 100004

The results of high and low precision sampling only fluctuates within noise floor, so are essentially the same.

The conclusion? If you are sampling more precision than your SNR, you are just sampling noise more precisely, which is still noise, and is same as if you don't sample as precise, then add noise in post.

Apply this to your example, when you have 35e- noise floor, your tonal range is not 65535-35 = 65500, but rather 65535/35 = 1872 levels (~10.8 stops), because the bottom 5 (!) bits are unstable noise.

I don't disagree, however I would still take a finer sampling of noise over a coarser sampling of noise. I mean, if your sensor has an analog range up through 68,000e-, with 35e- noise, that's how much noise can fluctuate up to. You might have a bunch of pixels where the correct signal value is for a midtone gray at 34,000e-. With noise, you are going to randomly fluctuate up to 35e- around that value, so you might have:

34,004
34,035
34,009
34,014
34,010
34,020
34,012
34,000

If you sample these analog values at 14-bit, you get the following:

8193
8200
8194
8195
8194
8196
8194
8192

If you sample them at 16-bit, you get the following:

35283
35315
35288
35293
35289
35299
35291
35279

If you multiply the 14-bit samples by four to put them into the same numeric range as the 16-bit sampling:

32772
32800
32776
32780
32776
32784
32776
32768

I'd rather take the finer and more random sampling that a 16-bit ADC allows, than the often repetitive sampling that 14-bit ADC does. It's that repetitive sampling with fewer bits that can lead to posterization in smooth gradients (like sky), lower SNR regions (shadows), etc. I'll take a finer, more random sampling of noise any day...despite the fact that it's still just sampling noise.

 

[BozillaNZ]

The kicker is, you can take the 14-bit samples,scale up to 16-bit, then add 2-bit randomly generated noise and get the indistinguishable (same) result. You can't get something for nothing really.

I' doing a lot of raw data manipulation recently, Canon 12-bit Raw files has the range of 127-3850. Seems pretty low isn't it? But the tonal range and dr of this model (1Ds 2) in DXO is still not much different than newer 14-bit sensors.

 

[dgatwood]

I don't expect this sensor to be used in a high speed body due to the low sensor read-out speed of a stacked sensor. That would also explain why the 7Dii was just replaced with existing sensor tech. I could see a totally new body announced as a niche for landscape or other areas that require greater detail without the need for high FPS. Video would also not work very well with this sensor due to the low read-out speed.

Readout speed is almost infinitely parallelizable, as is image compression/encoding. Performance is almost entirely a question of how much hardware they decide to throw at it. If they're feeling particularly nuts, they could use a two-sided silicon wafer, put all the circuitry on the back side with vias, and use a per-pixel ADC, as in this design....

 

[Lawliet]

Readout speed is almost infinitely parallelizable, as is image compression/encoding. Performance is almost entirely a question of how much hardware they decide to throw at it. If they're feeling particularly nuts, they could use a two-sided silicon wafer, put all the circuitry on the back side with vias, and use a per-pixel ADC, as in this design....

For starters one could take a look at the NX1 - with the action trigger mode reading and evaluating all of those 28MP at quite impressive 240 fps. Enough power to read 8K video at 120fps for slow motion, for example. Or 36m full RGB pixels faster then the 1Dx is now. Well, at 12bit, still not bad compared to 14bit with the lower parts filled with a blend of random&pattern noise.
Not that a 1DsIII-successor would need such framerates...most of the time the strobes would be the limiting factor anyway.

 

[ChristopherMarkPerez]

Interesting. I hope this comes to market.

It could explain why Canon has been to quiet on the topic of offering a Bigger Is Better sensor while Sony has ben extending the old Bayer technologies. New designs, new processes and, very likely, new the development of new tooling has to have cost Canon a boat-load of money.

Could it be that some of these sensors (escaped from the lab as demo-units) were the ones shown around NY recently (as rumored on this site)?

 

[sigint]

As a old Sigma cams user, i can tell you that if this rumor is real you can only expect :


- High end bulky camera from top of the line (It needs tripple memory buffer, multicore specially designed GPU and much bigger battery)

- Full frame max 7Mpix sensor, so it will be equivalent of 35Mpix in Bayer sensors, but in real life tests it will generate ~20-25Mpix IQ

- 5 layers will generate so much data to process that it will be very slow camera, max 7-8 FPS (5 layers are used to run potential patent lawsuit from Sigma, but it is uncertain because Sigma can't patent physical properties of silicon which are used in multilayered sensors, but only methods of image data processing.)

- Canon can use only in minimal way IR and UV layers, this potentialy can lower data stream and speed image processing in camera.

- ISO range from 100 to 6400, in color useful to 1800, in monochrome to 6400, and it will blow IQ of Foveon sensors, if Canon can get it right to ISO 1800 in color.

- Multilayered sensors needs telecentric lenses, to get away many problems with sharpes of image in corners.

- Very good IR cut filter to get away green corners problem (3 layers in sensor), but if 5 layers are used then special method to manage this problem which was a big problem for Sigma users in many years.

- And finally biggest problem - newly designed high speed RAW developer, even Sigma can manage this till now, it takes ages to process files from new sensor even on very new computers.

If Canon will hit the market with this multilayer sensor, then lots of user will be forced to buy new very expensive high end computers which could manage RAWs in reasonable time and not frustrate users in workflow (one Quattro Raw file could develop even up to 60 seconds), which is currently main factor of escapes from Sigma camp.

Look how long it takes to develop only one RAW file from Sigma Dp2 Quattro, you can get taste of future Canon multilayer cam workflow, and keep in mind that was APS-C size sensor, and what hell it will be with full frame files.

http://www.youtube.com/watch?list=UUqpOf_Nl5F4tjwlxOVS6h8A&v=o7ktvDUyTyU&feature=player_detailpage#t=603

Newest Foveon sensor it's a terrible mistake, there is almost half a year after Quattro introduction and Sigma can't get noise level right even at base ISO100, just look at pictures there is so distinctive noise in blues and light parts of the images that even older sensors not only Merrill but even 1.7 crop Foveon sensors can get better results. Theres is obvious engineering mistake in Quattro Foveon sensor to use blue layer in 4 times resolution which can't be corected in softwere, even Sigma pro photographers speaking about it. So Sigma is in blind alley with new sensor, and sales are dropping, Sigma users are reluctant to buy Quattro, instead they buy cheap Merrill or skip Quattro and want to wait next 2-3 years for new generation of Foveon sensors and in the mean time they are also buying Fujifilm MILCs (Sigma don't have MILC & DP Zoom in offer, Sigma DSLRs are not in that league in IQ as DP series).

Sigint

 

[lintoni]

Interesting. I hope this comes to market.

It could explain why Canon has been to quiet on the topic of offering a Bigger Is Better sensor while Sony has ben extending the old Bayer technologies. New designs, new processes and, very likely, new the development of new tooling has to have cost Canon a boat-load of money.

Could it be that some of these sensors (escaped from the lab as demo-units) were the ones shown around NY recently (as rumored on this site)?

http://www.canonrumors.com/2014/05/new-full-frame-camera-in-testing-cr1-2/

"much" better colour accuracy and detail

That certainly sounds like something to expect from a Foveon type sensor.

 

[ChristopherMarkPerez]

So... would a multi-layer sensor favor cameras with mirror-boxes as a way of getting the light down the layered tunnel at the edges? Might existing SLR lens designs work better on such a sensor?

;D ;D ;D

...
- Multilayered sensors needs telecentric lenses, to get away many problems with sharpes of image in corners...

 

[danski0224]

The file sizes from a Sigma DP2 Merrill are much larger than even a Canon 5DIII, and the Sigma DP2 is "15 MP".

The X3F files (RAW) are about 45 MB each at full resolution. Nothing but the Sigma software reads them, as far as I know.

They also take a bit of time to even open up on a computer once the Sigma ProPhoto software kicks in. I don't have a USB3 reader or the fastest SD card though.

Writing the image files in-camera to the SD card also takes a while. It is certainly 2 or 3 seconds from pressing the shutter to being able to review the image on the camera screen, which is an eternity compared to current Canon cameras. It is entirely comparable to the Canon 1Ds MK 1 though.

For color, ISO 100 or 200 is pretty much it. Monochrome is something else.

Battery life is abysmal on the DP2 Merrill cameras. Upside is the batteries are inexpensive.

I have not used the "pro" Sigma Foveon cameras.

Canon would have an enormous amount of work to make the Sigma DP2/DP3 process acceptably close to even 5DII operational standards, exclusive of high ISO (5DII high ISO- 1DX high ISO would be amazing).

When you get a keeper with a Merrill, it's a good one though.

 

[Destin]

Just throwing it out there..... could it be multiple layers for highlights and shadows rather than RGB?

 

[Woody]

As a old Sigma cams user, i can tell you that if this rumor is real you can only expect...

Newest Foveon sensor it's a terrible mistake, there is almost half a year after Quattro introduction and Sigma can't get noise level right even at base ISO100, just look at pictures there is so distinctive noise in blues and light parts of the images that even older sensors not only Merrill but even 1.7 crop Foveon sensors can get better results.

This is all very amusing to me. When Sigma Foveon cameras first hit the market, users were literally RAVING about them. They were putting down Bayer sensors in the market, especially the CMOS sensors from Canon. Of course, all problems associated with Foven sensors were either disputed, denied or dismissed.

Now, the skeletons are coming out of the closet. I am wondering if this may also be true for current users of other competing brands... you know who you are. ;D

Anyway, like what I said earlier, if this mutlilayer sensor technology indeed comes to pass, we can be sure most of the apparent drawbacks will have been ironed out already. Didn't folks make the same dire prognostications when Canon first announced their implementation of CMOS instead of CCD sensors? Sigma may not be able to resolve the accompanying issues of multilayer sensors, but this does not imply Canon and Sony (which also has patents on multilayer sensors) do not have other clever tricks up their sleeves.

 

[memoriaphoto]

Just throwing it out there..... could it be multiple layers for highlights and shadows rather than RGB?

Interesting thought. Something like the legendary (?) Fuji S5 Pro. I guess that would dramtically increase the DR performance.

Personally though, I hope for a more "true to life" color approach out of cam with film-like quality. The colorpalette of older Canons was/is more pleasing imho. Lately Canon has focused too much on high ISO performance with thinner CFA:s which has required more color-work in post for us VERY RARE daylight shooters ;-)

 

[jeffa4444]

A pro in a particular line of photography has already alluded to a 45MP camera. What will be interesting is whether it will seek its way into the Cinema line of the C300 / C500 as Canon has cheaper competition from Sony with the FS-7 which trumps them in many ways that the FS-700 never did. The C300 was annouced late 2011 and didnt have the same competition it now has in Black Magic, AJA, Panasonic & Sony.