Sony revolutionizes the sensor world? Active Pixel Color Sampling sensor (APCS).

[Jon_D]

http://www.sonyalpharumors.com/sr4-hot-news-sony-revolutionizes-the-sensor-world-makes-worlds-first-active-pixel-sensor-apcs/

......
Every single pixel can take the full color info with the help of an electrified moving color filter!
How this works in detail is yet not described on that paper. But basically instead of having four pixels “RGBG” interpolated into one single pixel information (as it happens on usual Bayer sensors) every pixel on the APCS sensor grabs the full color information. There is no need of interpolation. So on paper you have 4 times the resolution and that 4,8 MP sensor can be compared (again on paper) with a 19,2 MP classic Bayer sensor used by 99% of todays cameras. This is a more advanced solution than Sigmas Foveon sensor used on DP cameras. Sigma uses three vertically ordered R-G-B layers. But also here you need to merge three pixel information into one and you also struggle with noise level because electrons get absorbed (or lost) on each layer).

Positive effects of such a sensor design are:
– 4 times bigger pixels compared to same resolution Bayer sensor. This means more electrons captured and therefore higher dynamic range and lower noise (crazy ISO possible…even crazier than those of the Sony A7s).
– No moire issues anymore. No Anti Aliasing filter needed (increases per pixel sharpness).
– That tech also allows to make crazy high megapixel sensor. For example if you keep an Sony’s APCS pixel size same as the current Sony A7r 36MP RGB pixel size sensor you could in theory make a 144 Megapixel FF APCS sensor
– Less pixels to read means also faster processing and readout.

Other key features of the new APCS sensor:
– Electronic Global shutter. No more “jello” effect on videos!
– Records 2K with 16,000 frames per second (processor doesn’t have to be interpolate between RGBG pixels and therefore can use full power to read out more frames per second!). Note that the current Arri Phantom Flex shoots 4K at 1,000 frames per second.
......

canon is trying to catch up with the latest sony sensor generation and sony seem to have the next big hit ready already.

i guess as with BSI we will see this in smartphones first and maybe 2016 in more photographer oriented cameras.

or maybe it´s a VIDEO sensor. the sensor is little big for a smartphone.
but i guess the technology will make it into still cameras at some point.

anyway... personally i can´t wait to get rid of the bayer sensor and it´s shortcomings.

 

[Loswr]

It's far from a new idea in the sensor world. For many years my Zeiss microscope cameras have done something similar. In that case they're moving the sensor under the Bayer filter instead of moving the filter over the sensor, but the end result is the same – no color interpolation needed. Note that the resolution increase is color resolution only, not a true increase in spatial resolution.

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision. For most microscopy, specimens are fixed and speed isn't an issue. For moving subjects it is, so the movement of the Bayer filter will need to be fast enough to support motion-stopping speeds for use in dSLR/MILC (since each image is really the sum of three successive images). Note that Sony touted the advantages of 'sensor shift' in-body image stabilization (IBIS), but they don't offer it in full frame sensors, most likely because the larger sensor can't be translated rapidly/precisely enough.

So, while interesting, this technology is neither conceptually new, nor is it all that and a bag of chips.

 

[Coldhands]

Pure snake oil at this point. Not a single iota of information on how this magical "electrified moving colour filter" works? Get back to me with some science. Edit: Looks like Neuro has the goods.

And honestly, the writer's technical understanding makes Tony Northrup look like a genius. Bayer image sensors interpolate four pixels into one? Seriously? Saying this 4.8 MP sensor is equivalent to a 19.2 MP Bayer sensor is simply wrong.

And no more AA filter? Doubt it. The reason most sensors have an AA filter now is to deal with rendering of detail near the sensor's Nyquist frequency, and that doesn't change by getting rid of the CFA.

144 MP full frame? I shouldn't even need to bother pointing out how absurd this claim is.

Maybe there is a kernel of truth in here somewhere, but it's buried under a pile of breathless speculation.

 

[Jon_D]

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision.

but who says there is something mechanical moved?
could be switched electronically.

i doubt they move something physically 16000 times a second.

 

[Jon_D]

Pure snake oil at this point. Not a single iota of information on how this magical "electrified moving colour filter" works? Get back to me with some science. Edit: Looks like Neuro has the goods.

yeah well do i have to remind you what kind of forum you are visiting here? ;D
remember 10 years of talking about the 100-400mm II??

And honestly, the writer's technical understanding makes Tony Northrup look like a genius. Bayer image sensors interpolate four pixels into one? Seriously? Saying this 4.8 MP sensor is equivalent to a 19.2 MP Bayer sensor is simply wrong.

And no more AA filter? Doubt it. The reason most sensors have an AA filter now is to deal with rendering of detail near the sensor's Nyquist frequency, and that doesn't change by getting rid of the CFA.

144 MP full frame? I shouldn't even need to bother pointing out how absurd this claim is.

Maybe there is a kernel of truth in here somewhere, but it's buried under a pile of breathless speculation.

he cleary says "on paper" and that it´s theoretically.

sure he sounds a bit carried away by the prospects...

 

[Coldhands]

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision.

but who says there is something mechanical moved?
could be switched electronically.

i doubt they move something physically 16000 times a second.

For reference, the dust reduction built into our sensors moves a filter at up to 50khz (50,000 cycles per second).

 

[Coldhands]

Pure snake oil at this point. Not a single iota of information on how this magical "electrified moving colour filter" works? Get back to me with some science. Edit: Looks like Neuro has the goods.

yeah well do i have to remind you what kind of forum you are visiting here? ;D
remember 10 years of talking about the 100-400mm II??

And honestly, the writer's technical understanding makes Tony Northrup look like a genius. Bayer image sensors interpolate four pixels into one? Seriously? Saying this 4.8 MP sensor is equivalent to a 19.2 MP Bayer sensor is simply wrong.

And no more AA filter? Doubt it. The reason most sensors have an AA filter now is to deal with rendering of detail near the sensor's Nyquist frequency, and that doesn't change by getting rid of the CFA.

144 MP full frame? I shouldn't even need to bother pointing out how absurd this claim is.

Maybe there is a kernel of truth in here somewhere, but it's buried under a pile of breathless speculation.

he cleary says "on paper" and that it´s theoretically.

sure he sounds a bit carried away by the prospects...

As you rightly point out, we're all here for rumours, but I feel that if one is going to proclaim that some thing will "revolutionise the world", then it should a be accompanied by some convincing information.

I just really take issue with all the false assertions that the writer puts his SR4 stamp on.

 

[Maximilian]

I say: bring it on and we'll see what it can do.

To me, after short reading and not willing to study sensor design development I thought it was more like:
"Sony NOW has finally found a patent on its own to get into the foveon/multilayer/whatever sensor design to avoid the Bayer/XTrans/whatever patterns”. And put some icing on top, that it looks better than the others. Hooray!

We'll see if this will work in practice and if it'll be evolutionary, revolutionary or nothing at all.
Competence is always good!

 

[Loswr]

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision.

but who says there is something mechanical moved?
could be switched electronically.

Read:

Every single pixel can take the full color info with the help of an electrified moving color filter!

Sure, it's possible to use a tunable filter (my scopes use several acousto-optical tunable filters, for example). But if the article you link got something so basic wrong, the whole thing could be false.

As Coldhands points out, your suggestion that moving something the size of a sensor at 16 kHz is proven false by existing technology. But it's not trivial to move it at that speed with sufficient precision.

Like I said...not revolutionary and likely irrelevant for dSLR/MILC.

 

[Jon_D]

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision.

but who says there is something mechanical moved?
could be switched electronically.

Read:

Every single pixel can take the full color info with the help of an electrified moving color filter!

he also wrote he has no clue how it works...

i could imagine it means something like in LCD technology.

 

[Jon_D]

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision.

but who says there is something mechanical moved?
could be switched electronically.

i doubt they move something physically 16000 times a second.

For reference, the dust reduction built into our sensors moves a filter at up to 50khz (50,000 cycles per second).

and precision plays no role at all in this case.
apples and oranges.

not to mention that the dust reduction is not working all the time.
i had great doubts if this RGB filter would be a mechanical construction.

 

[rs]

Great. So each colour is sampled at a different time, as the colour filter physically moves in front of the pixel. This will be great for tripod based static scene shots, but will result in colour tearing - much like a field sequential display does. Bad news for video, bad news for stills of anything that moves as there will effectively be three exposures taken at different times for red, green and blue, and then all merged into one.

Plus the exposure time for each colour cannot be more than 1/3rd of the total exposure time, so I'd hazard a guess that the sensitivity isn't increased either by these larger pixels for the same resolution. And as the filter can't transition instantly between the colours, its less than 1/3rd of the exposure time available to each colour.

 

[Jon_D]

Great. So each colour is sampled at a different time, as the colour filter physically moves in front of the pixel. This will be great for tripod based static scene shots, but will result in colour tearing - much like a field sequential display does. Bad news for video, bad news for stills of anything that moves as there will effectively be three exposures taken at different times for red, green and blue, and then all merged into one.

Plus the exposure time for each colour cannot be more than 1/3rd of the total exposure time, so I'd hazard a guess that the sensitivity isn't increased either by these larger pixels for the same resolution. And as the filter can't transition instantly between the colours, its less than 1/3rd of the exposure time available to each colour.

did you read that the sensor is able to readout 16000 times a second?

i wonder how you will see color smearing.... especially when bayer sensors are all about "smearing colors".

 

[rs]

Great. So each colour is sampled at a different time, as the colour filter physically moves in front of the pixel. This will be great for tripod based static scene shots, but will result in colour tearing - much like a field sequential display does. Bad news for video, bad news for stills of anything that moves as there will effectively be three exposures taken at different times for red, green and blue, and then all merged into one.

Plus the exposure time for each colour cannot be more than 1/3rd of the total exposure time, so I'd hazard a guess that the sensitivity isn't increased either by these larger pixels for the same resolution. And as the filter can't transition instantly between the colours, its less than 1/3rd of the exposure time available to each colour.

did you read that the sensor is able to readout 16000 times a second?

i wonder how you will see color smearing.... especially when bayer sensors are all about "smearing colors".

A 1/8000 shutter speed would only allow time for two colours at that paltry speed. Three readouts, or 24,000 per second is needed for full colour and full on tearing at the shutter speeds most of us can use right now. If long shutter speeds could be made up of hundreds or thousands of frames stacked together, then it could be good - at slow shutter speeds. Forget about it for action until that figure closes in on millions per second, but then you're still left with low sensitivity due to shorter exposure times per colour. Not to mention the processing, and how to make something move millions of times a second.

 

[Coldhands]

There's also no guarantee that it's scalable – there's a good chance this will not make it into APS-C or larger sensors any time soon. The larger the thing being moved, the more difficult it is to move it fast enough with sufficient precision.

but who says there is something mechanical moved?
could be switched electronically.

i doubt they move something physically 16000 times a second.

For reference, the dust reduction built into our sensors moves a filter at up to 50khz (50,000 cycles per second).

and precision plays no role at all in this case.
apples and oranges.

not to mention that the dust reduction is not working all the time.
i had great doubts if this RGB filter would be a mechanical construction.

Fair point, of course, but I'm fairly certain that it would be possible to synchronise the movement of a filter array via an electro-magnetic driver (think something akin to a speaker coil) with the read-out of the sensor. On the physical side, it would actually be a relatively simple transfer-function to model (essentially a spring-mass system), especially if you compare it to something like a hard-disk drive that has to move the read head with incredible precision over a disk rotating at up to 10,000rpm.

I'm not a dedicated controls engineer, so this is about all the insight I can offer, but I think I'll place my bets on something like this.

 

[Loswr]

he also wrote he has no clue how it works...

Oh, well that makes it all ok then. Kind of like saying moving something physically at 16,000 times per second is unlikely, I guess some people don't mind writing things about which they have no clue.

 

[RLPhoto]

Hasn't Hasselblad had the multishot series for awhile now? It doesn't move the color filter but it does move the sensor to get the full color information. Still it's pretty slow, clunky and requires still subjects but the idea of moving around something to extract more information isn't new.

 

[AcutancePhotography]

I look forward to reading more about this technology. I hope Sony can get a prototype camera with this technology working so we can see some tests.

 

[Loswr]

For me the interesting part was "12bit / 16bit".

Obviously Sony are looking to move beyond 14bit raw images.

Haven't seen any Canon patents mention 16bit raw yet.

Don't be too sure. Keep in mind that this is a sensor patent. 16-bit output from the sensor doesn't mean 16-bit RAW file output from the camera. Sony already promotes that they use 16-bit in-camera processing.

 

[jasny]

For me the interesting part was "12bit / 16bit".

Obviously Sony are looking to move beyond 14bit raw images.

Haven't seen any Canon patents mention 16bit raw yet.

Don't be too sure. Keep in mind that this is a sensor patent. 16-bit output from the sensor doesn't mean 16-bit RAW file output from the camera. Sony already promotes that they use 16-bit in-camera processing.

Doesn't look like a patent. More like product description.

BTW: What kind of application for this sensor? Obviously, it looks more video then stills oriented. But, does any video cam use that format? Dont know.
Taking photo cameras into consideration… Well, probably only Canon G1 X uses 1.5 inch sensor.