We will be releasing an upcoming series of articles on infrared photography that I've been slowly working through the last few weeks, and this amusingly popped up in the list of patent applications published. In this patent application (2024-054762), Canon is exploring using sensor pixels more sensitive to IR light to assist with color images. Their reasoning is that infrared light travels through haze and smoke better, and this would allow for better imaging performance in adverse conditions.

While this sensor would probably see fit to exist in security cameras, it could see applications in a variety of more specialized markets such as automotive sensors. What practical purpose would this have to our normal photography? Oh, absolutely zero, but it's one of the more innovative ideas I've seen on a sensor no matter how practical or capable it may be.

According to Canon's patent-speak;

Depending on the atmospheric conditions, the image of the subject may not appear as a signal value in the visible light image, and in this case, the appropriate method of patent document 1 [prior patent] is used.

Even if a correction processing technique is used, there is a possibility that a suitable correction result cannot be obtained. The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an image processing device, an imaging device, a control method, and a program that improve the visibility of a subject image in a visible light image.

The sensor itself would end up having a different color filter array from the standard Bayer format, looking something like;

image 12 - Canon Patent Application: Hybrid IR/Color sensor to improve Image Quality

Canon describes that the system would take an image of color and then follow up with the image using the IR pixels, leading to something like this:

image 13 728x344 - Canon Patent Application: Hybrid IR/Color sensor to improve Image Quality

Canon would then combine these images and use the infrared light image to further refine the color video. Canon specifically mentions that this patent application that this application can be applied to stills and video.

Could this actually be used in real life? As I mentioned, I do think that this is a security camera sensor, as imaging through smoke, environmental haze, etc seems to be a good feature to have there. But environmental conditions do happen in our regular photography as well.

As with all patent applications, this is simply a look into Canon's research.

Japan Patent Application 2024-054762

Go to discussion...

6 comments

  1. If you want to get a taste for what it could do, take your normal rgb image modify it to be a two channel image and change the third to grey scale, then combine the two. It should be helpful in the weather limited scenario as well as some low light scenarios. Whether (no pun intended) or not it's esthetically pleasing enough for Canon to put it into a camera most of us would buy? I'm gonna to guess it's unlikely, but you never know - they have made some astronomy bodies.
  2. If you want to get a taste for what it could do, take your normal rgb image modify it to be a two channel image and change the third to grey scale, then combine the two. It should be helpful in the weather limited scenario as well as some low light scenarios. Whether (no pun intended) or not it's esthetically pleasing enough for Canon to put it into a camera most of us would buy? I'm gonna to guess it's unlikely, but you never know - they have made some astronomy bodies.
    Would you be so kind to post an example-picture?
  3. Would you be so kind to post an example-picture?
    unmodified.jpg
    Unmodified

    red+green.jpg
    Red + Green

    blue.jpg
    Blue

    process.jpg

    Process

    complete.jpg
    Complete

    Hopefully this is understandable. I didn't do any adjustments to the channels, but we can expect Canon would develop software AI algorithm to utilize the extra channel the most effective way. So, I think if you want to experiment for getting an idea how this might work, you'll want to adjust the the blue channel (or whichever one you want to pretend is the inferred channel) and keep the other two the same.
    in the completed image, you can see some stronger details in the flower petals. Canon's software would know how to only use image data that gave an enhanced quality to the output - depending on some user selectable settings of what you will want enhanced.

    This all just my best guess.

    a better experiment could be to use a diffusion filter for two channels and take a separate photo with it removed for the third. This should be a bit closer to the idea about fog.

    Have fun if you want to try!
  4. View attachment 216056
    Unmodified

    View attachment 216057
    Red + Green

    View attachment 216058
    Blue

    View attachment 216059

    Process

    View attachment 216060
    Complete

    Hopefully this is understandable. I didn't do any adjustments to the channels, but we can expect Canon would develop software AI algorithm to utilize the extra channel the most effective way. So, I think if you want to experiment for getting an idea how this might work, you'll want to adjust the the blue channel (or whichever one you want to pretend is the inferred channel) and keep the other two the same.
    in the completed image, you can see some stronger details in the flower petals. Canon's software would know how to only use image data that gave an enhanced quality to the output - depending on some user selectable settings of what you will want enhanced.

    This all just my best guess.

    a better experiment could be to use a diffusion filter for two channels and take a separate photo with it removed for the third. This should be a bit closer to the idea about fog.

    Have fun if you want to try!
    Using the existing bayer pattern and modifying the channels in-camera, could the result be better AF as the contrast seems to be better? Or perhaps used only in low light or when focus hunting?
  5. Using the existing bayer pattern and modifying the channels in-camera, could the result be better AF as the contrast seems to be better? Or perhaps used only in low light or when focus hunting?
    Most likely, if there is any benefit to changing how the there channels are used in terms of AF, someone at Canon thought about it years ago. An inferred fourth channel should definitely have benefits in some situations.

Leave a comment

Please log in to your forum account to comment