We will be releasing an upcoming series of articles on infrared photography that I've been slowly working through the last few weeks, and this amusingly popped up in the list of patent applications published. In this patent application (2024-054762), Canon is exploring using sensor pixels more sensitive to IR light to assist with color images. Their reasoning is that infrared light travels through haze and smoke better, and this would allow for better imaging performance in adverse conditions.
While this sensor would probably see fit to exist in security cameras, it could see applications in a variety of more specialized markets such as automotive sensors. What practical purpose would this have to our normal photography? Oh, absolutely zero, but it's one of the more innovative ideas I've seen on a sensor no matter how practical or capable it may be.
According to Canon's patent-speak;
Depending on the atmospheric conditions, the image of the subject may not appear as a signal value in the visible light image, and in this case, the appropriate method of patent document 1 [prior patent] is used.
Even if a correction processing technique is used, there is a possibility that a suitable correction result cannot be obtained. The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an image processing device, an imaging device, a control method, and a program that improve the visibility of a subject image in a visible light image.
The sensor itself would end up having a different color filter array from the standard Bayer format, looking something like;
Canon describes that the system would take an image of color and then follow up with the image using the IR pixels, leading to something like this:
Canon would then combine these images and use the infrared light image to further refine the color video. Canon specifically mentions that this patent application that this application can be applied to stills and video.
Could this actually be used in real life? As I mentioned, I do think that this is a security camera sensor, as imaging through smoke, environmental haze, etc seems to be a good feature to have there. But environmental conditions do happen in our regular photography as well.
As with all patent applications, this is simply a look into Canon's research.
Japan Patent Application 2024-054762
Unmodified
Red + Green
Blue
Process
Complete
Hopefully this is understandable. I didn't do any adjustments to the channels, but we can expect Canon would develop software AI algorithm to utilize the extra channel the most effective way. So, I think if you want to experiment for getting an idea how this might work, you'll want to adjust the the blue channel (or whichever one you want to pretend is the inferred channel) and keep the other two the same.
in the completed image, you can see some stronger details in the flower petals. Canon's software would know how to only use image data that gave an enhanced quality to the output - depending on some user selectable settings of what you will want enhanced.
This all just my best guess.
a better experiment could be to use a diffusion filter for two channels and take a separate photo with it removed for the third. This should be a bit closer to the idea about fog.
Have fun if you want to try!
Greetings Andreas