I don't know how many of you followed me back on my older site, but I was always talking about Canon's SPAD (Single Photon Avalanche Diode) image sensor patent applications – so much so that I thought Canon was starting to troll me. Needless to say with the release of the MS-500 I felt a little better about things and silently asked Canon for forgiveness for all those mean thoughts in the past.
This patent application is curious and I find it interesting because it could open some exciting imaging opportunities in the future.
One of the main challenges of SPAD sensors has always been too much of a good thing is a bad thing. SPAD sensors work by actually counting the photons that hit the surface of the detector. Everything is fine until there are too many photons coming in too fast. The diode then gets completely overwhelmed and messes up its count. Think of it as a conveyor belt, the more items on the belt the harder it is to accurately count the items.
CMOS however converts photons into voltage and the light intensity is measured by the amplitude of the voltage using analog to digital convertors.
Secondly, another disadvantage of a SPAD sensor is that it takes current to put the diode into “avalanche mode” so it can start counting photons. This increased current means that the sensor itself requires more power than a traditional CMOS sensor.
Generally speaking, CMOS sensors (actually their analog to digital converters) are more accurate with more light and SPAD sensors are better when it has a chance to count every photon (low light), so it stands to reason that the combination of these two sensors may be the best thing since sliced bread. It also means the image could have an absolutely gob-smacking amount of dynamic range. A single image of the full moon showing the star field in the background? No problem.
The patent application goes into a mind-numbing amount of detail discussing how Canon would do such a thing and various means or methods that Canon could use to switch between a SPAD and a CMOS sensor based upon differences in shutter speed, video, and even power draw. Also and most interesting to me, is that with Figure 11 and related example, Canon discusses taking an image with the SPAD image sensor, and in the section where the SPAD sensor is overwhelmed with too much light (for example in this image, the sun), replacing that section of the image with corresponding section from the CMOS sensor. Perhaps even more interesting, one of the easiest to imagine use cases of a camera of this sort would be one smartphone, since you could have two image sensors side by side.
As with all patent applications, this may never end up in an actual product, but it serves as a glimpse into what Canon is researching.
Source: Japan Patent Application 2023-123474
As I said:
My constant thought since many years, of sensors future developments, has been a sensor that can vary its ISO photodiode per photodiode (so, for the very little thing I understand, varying the current to each photodiode), so you can shoot a single image with a super extended DR without needing of HDR and/or compositing, leaning on AI skills (or user inputs) of the camera.
Certainly an application firstly for mobiles, but of course could be useful also in a professional photo environment; I think, just to make a basic example, to a sun backlit situation were I can shoot the subjects with 400/800iso in full daylight (and AI recognize them, so only that part of the image gets enhanced iso) while the background gets shot at 100iso, that would be pretty cool, it's like having a key light or a fill light in front of the subject while you expose for the background. Of course the best solution is actually expose for the background and then use a panel or a strobe to fill in, so you shoot all at 100iso and the picture surely comes better already, but if you travel light, or your gear gets broken/lost, batteries dies, etc, having that kind of power in the camera could save your work. And as I said, for the mobile phones would really increase the quality SOOC.
Any idiot could have thought of that.
Although Canon probably has the only practical SPAD sensors right now.
On that note, why has no one combined 2 CMOS sensors that have different amplification?
That would create a lot of dynamic range even though it would come nowhere close to this.
The lenses could have different apertures,
Canon has Developed a 1-inch 4K Sensor With 24 Stops of Dynamic Range - Y.M.Cinema Magazine
That makes me think.
Imagine one of these and the other a SPAD sensor.
Not that I would ever need more than 24 stops of dynamic range anyway.
I still don't think anyone but Canon could build such a thing right now.