More colors

[AJ]

Todays sensors have heaps of megapixels. More than you can shake a stick at. More than most of us need.

Now, today's sensors still have a bayer filter with a basic R,G,B pattern. RGB is what our monitors use. Newer sensors have standard green alternating with an expanded green, but it's still pretty much the same idea.

What if you created a color filter that sampled the color spectrum in a different way? You could sample a wider spectrum for richer violets and deeper reds. These two colors are weak on DSLRs, something I miss after switching from slide films like Velvia.

You could also sample the spectrum more finely. Either way you'd end up with a wider gamut. Maybe a computer monitor wouldn't show the full extent of the improvement, but a good quality print should.

So, the idea is to use these sensels to record more color information, rather than focusing in spatial information.

In the same vein, some sensels could have a darker filter on top (e.g. RGgB where the g is 3 stops less sensitive than G) for wider dynamic range.

 

[Jettatore]

I think Pentax has some unique system for seperating color, it has a seperate sensor for R, G and B I think, instead of a Bayer filter.

However, I find this useless, as you can quite easily do absolutely anything you like with color inside of a good image editing program, and easily so once you get the hang of it, -so as long as the underlying grayscale/value tonality is in full rich quality, changing the hue and saturation that it drives is infinitely modable.

 

[JerryKnight]

You should go work for Canon or Nikon and get them to do this.

Isn't there a sensor somewhere that is RGBW (+white) so that it can better measure luma independently from chroma?

And what's stopping sensors from adding additional wavelengths? Let's see a RYGBVW sensor that's 30+ megapixel, so that you could still get a decent resolution image, but with much more precise color.

 

[JerryKnight]

I think Pentax has some unique system for seperating color, it has a seperate sensor for R, G and B I think, instead of a Bayer filter.

However, I find this useless, as you can quite easily do absolutely anything you like with color inside of a good image editing program, easily.

Sure, you can make something up and fake it, but it's different if you can actually measure the color you're capturing. Otherwise, why don't we all just shoot monochrome and add our own color?

 

[Jettatore]

I think Pentax has some unique system for seperating color, it has a seperate sensor for R, G and B I think, instead of a Bayer filter.

However, I find this useless, as you can quite easily do absolutely anything you like with color inside of a good image editing program, easily.

Sure, you can make something up and fake it, but it's different if you can actually measure the color you're capturing. Otherwise, why don't we all just shoot monochrome and add our own color?

This is true (although shooting monochrome and adding your own color takes 5 minutes to an hour or so depending on the shot), but I'm not personally into photography for capturing scientifically accurate images. I'm in it for the communication aspects of visual imagery and with this in mind my views on it's usefulness might be wildly different than yours, since if I want the magentas more vivid I just move a slider. Also, I would add that the aforementioned Pentax is absurdly priced, which had some unmentioned weight in my comments.

 

[TrumpetPower!]

Ideal would be for every photosite to capture a spectral data plot (such as a spectrophotometer like the i1 Pro does), but that ain't gonna happen for a looooooooooooong time.

In the mean time, current cameras are good enough for art reproduction if you've got a carefully-designed workflow. It can all be done with target profiling, but you can even get fancy and do spectral imaging by taking multiple exposures with carefully-selected color filters and some fancy in-house software. The Smithsonian does that sort of thing, but it's serious overkill for anybody outside of those circles.

Cheers,

b&

 

[Policar]

Now, today's sensors still have a bayer filter with a basic R,G,B pattern....These two colors are weak on DSLRs, something I miss after switching from slide films like Velvia.

Velvia only has three layers....RGB.

The weak colors have very little to do with the RGB sensor, they have a lot more to do with the strength of the color filters over the photosites. Increasingly, manufacturers are using weaker and weaker color filters to get better low light, but at the cost of "pure" colors. DXOmark measures something similar with their "portrait" measurement and some MBDB do quite well here but dSLRs are worse. Look at the spectral sensitivity curves on Velvia 50--super narrow. No matter how much you process, you won't get Velvia colors from dSLRs. You can probably fake it well enough through processing, but film is still magic.

Did you shoot LF? If so, lenses are another actor. Modern dSLR lenses (particularly zooms and fast wide angles) have a lot of longitudinal chromatic abberation so colors are not very pure. Look at photos taken with a true aprochromatic lens (the Coastal optics 60mm f4). Colors are incredible, bokeh is flawless.

 

[jrista]

Silicon is naturally less sensitive to blue and violet wavelengths of light than it is to green, and it is most sensitive to red. I am not really sure that adding additional pixel types with, say, a violet filter, will really improve the quality of violet in digital photos taken with such a sensor. Current bayer CFA's use fairly broad filter ranges for blue, green, and red pixels. Blue filters allow in light down to around 380nm (very violet), while red filters allow in light up to and beyond 700nm. Green overlaps both just a tad.

Assuming you did actually involve say violet, blue, cyan, green, yellow, orange, red, deep red CFA pixels in some "advanced" sensor design. One way or another, you still need to "coalesce" that greater variety of colors into RGB image pixels in the end anyway. You would have to do some weighted mathematical averaging of blue with violet and cyan, green with cyan and yellow, and red with deep red, orange and yellow. There are a few drawbacks to such an approach, and potentially no real benefits:

- You need to be extremely careful about how you lay each color out in the CFA to ensure that color distribution is effective to capture enough luminance per pixel and still be capable of interpolating an output pixel (today most interpolation is based on 2x2 quads of bayer pixels with slightly weighted algorithms to eliminate color artifacts, zippering, etc.) Fuji's mew X Pro-1 uses 6x6 blocks of pixels to produce an output pixel, and it still only uses RGB albeit with less pattern repetition.
- You need a more complex RAW file format to acomodate the increase in various CFA pixel color types.
- You need considerably more complex interpolation algorithms to convert larger blocks of input CFA pixels into output RGB pixels.
- You need more advanced logic to convert multiple input colors into fewer output channels per output pixel with appropriate color accuracy.
- When working RAW, you need to reprocess the bayer pixels and send them through your entire pipeline of edits to render the image on screen, so more complex interpolation and color conversion will impact the performance of post-process workflows

I think more can be done by moving to a Foveon X-3 style sensor, where each color channel is stacked at a single photosite. Silicon is naturally sensitive to most of the visible wavelengths of light, and considerably more sensitive to near infrared than it is to UV. As such, it also naturally filters light the deeper it penetrates into a silicon well, making it pretty easy to capture blue at the surface, green in the middle, and red deep into the well (which is exactly what Foveon sensors do.) By eliminating an alternating pattern of RGB pixels, you improve a LOT of the characteristics of the sensor directly, and since you have full color data at every single pixel, you also have full color fidelity. Foveon sensors are fairly well known for having richer blues than bayer-type sensors, without the need for a lot of complexity. You also eliminate color moire, minimize monochrome moire, reduce noise in the blue channel, etc.

 

[JerryKnight]

This is true (although shooting monochrome and adding your own color takes 5 minutes to an hour or so depending on the shot), but I'm not personally into photography for capturing scientifically accurate images. I'm in it for the communication aspects of visual imagery and with this in mind my views on it's usefulness might be wildly different than yours, since if I want the magentas more vivid I just move a slider. Also, I would add that the aforementioned Pentax is absurdly priced, which had some unmentioned weight in my comments.

Perfectly valid opinion.

I guess my point is that there is a line somewhere between perfectly captured color and completely fabricated color. Most photographers would prefer (all else being equal) for their sensor to make accurate color measurements.

It's a lot like the dynamic range of the sensor. Sure, you can tweak the exposure in post, and the final image file will have a fixed range and color space, but it's always better to start with more information than exactly what is needed in the final image.

(The "all else being equal" part is where we run into harsh reality. Exotic sensors, if they are even made, will be far more expensive for a very long time, until they're no longer considered exotic.)

Ideal would be for every photosite to capture a spectral data plot (such as a spectrophotometer like the i1 Pro does), but that ain't gonna happen for a looooooooooooong time.

In the mean time, current cameras are good enough for art reproduction if you've got a carefully-designed workflow. It can all be done with target profiling, but you can even get fancy and do spectral imaging by taking multiple exposures with carefully-selected color filters and some fancy in-house software. The Smithsonian does that sort of thing, but it's serious overkill for anybody outside of those circles.

This is exactly what is done in astrophotography using a telescope and dedicated CCD with a set of specifically calibrated filters.

 

[JerryKnight]

Assuming you did actually involve say violet, blue, cyan, green, yellow, orange, red, deep red CFA pixels in some "advanced" sensor design. One way or another, you still need to "coalesce" that greater variety of colors into RGB image pixels in the end anyway. You would have to do some weighted mathematical averaging of blue with violet and cyan, green with cyan and yellow, and red with deep red, orange and yellow. There are a few drawbacks to such an approach, and potentially no real benefits:

There are benefits. You're starting with better information, which you then dissolve down into whatever RGB colorspace that the final image has. The same goes for higher dynamic range. What's the point, if you don't have an HDR monitor? The point is, it gives you more to work with before having to fit it into the final restrictions.

Of course, even putting the image on your monitor, it's already missing much of the information collected by the sensor, but as you move your sliders around, it could use the additional information rather than guess.

Of course, this is all a flight of fancy. The cost vs benefit would be obscenely out of proportion.

 

[jrista]

Assuming you did actually involve say violet, blue, cyan, green, yellow, orange, red, deep red CFA pixels in some "advanced" sensor design. One way or another, you still need to "coalesce" that greater variety of colors into RGB image pixels in the end anyway. You would have to do some weighted mathematical averaging of blue with violet and cyan, green with cyan and yellow, and red with deep red, orange and yellow. There are a few drawbacks to such an approach, and potentially no real benefits:

There are benefits. You're starting with better information, which you then dissolve down into whatever RGB colorspace that the final image has. The same goes for higher dynamic range. What's the point, if you don't have an HDR monitor? The point is, it gives you more to work with before having to fit it into the final restrictions.

Possibly. I'm not really sure you do start with better information. The greater variety of color filter types is going to mean your filtering out more light per pixel, over a greater area of pixels than with a standard RGB CFA. That is going to reduce your overall luminance information...so at least in one sense, your starting off with "less".

In terms of better...how do you define better in this context? With a greater variety of color filter types, you have to be more careful about exactly what range of wavelengths each filter allows to pass, and about how much overlap there is between the allowed wavelengths for each type of filter. A greater diversification of colors will probably also increase photon shot noise more than even reducing pixel size would. With less luminance to start with, the sensor is going to have to be more sensitive to compensate, so photon shot noise will likely be enhanced more so than if you have fewer color filter types. You also still have to mathematically average the "more accurate" color distribution into fewer output color channels, and there is usually a loss of accuracy when computing the values of multiple discrete inputs into a discrete output. You might be able to mitigate that by converting to floating point values rather than integer values, thereby preserving as much original precision as possible, but you still have a finite amount of precision that you can use to represent pixel values without increasing the complexity of the math such that it makes editing such a RAW file impractical.

Of course, even putting the image on your monitor, it's already missing much of the information collected by the sensor, but as you move your sliders around, it could use the additional information rather than guess.

Of course, this is all a flight of fancy. The cost vs benefit would be obscenely out of proportion.

Yup.

 

[AJ]

I started thinking about this after reading the DXO threads. It seems that nowadays people want more colors and more dynamic range, but not necessarily more spatial information (megapixels).

Yes the demosaicing would be a lot more complex. But that's why we have fast computers and smart engineers to write the algorithms.

Interesting point about Bayer filter becoming lighter to increase sensitivity, at the expense of pure colors. I did not know that!

 

[AJ]

It'd be neat if you could have a Foveon-type sensor where the first layer is white, then followed by the colors. You'd record 100% of the photons coming in, rather than screening them out with a color filter up front. B&W shooters would benefit from this too.

 

[jrista]

It'd be neat if you could have a Foveon-type sensor where the first layer is white, then followed by the colors. You'd record 100% of the photons coming in, rather than screening them out with a color filter up front. B&W shooters would benefit from this too.

Well, white is simply the presence of all visible wavelengths of light. If the "first layer" recorded white, there wouldn't be any other colors left for the BGR layers below to record. (Note that there are no color filters in a foveon sensor...the pixels themselves do the filtering simply by the way they are designed.) Technically speaking, a foveon sensor IS good for B&W shooters...you get full luminance and chrominance data at every photosite just like a monochrome sensor would. All a B&W shooter would need to do is convert in post.

 

[LetTheRightLensIn]

Todays sensors have heaps of megapixels. More than you can shake a stick at. More than most of us need.

Now, today's sensors still have a bayer filter with a basic R,G,B pattern. RGB is what our monitors use. Newer sensors have standard green alternating with an expanded green, but it's still pretty much the same idea.

What if you created a color filter that sampled the color spectrum in a different way? You could sample a wider spectrum for richer violets and deeper reds. These two colors are weak on DSLRs, something I miss after switching from slide films like Velvia.

You could also sample the spectrum more finely. Either way you'd end up with a wider gamut. Maybe a computer monitor wouldn't show the full extent of the improvement, but a good quality print should.

So, the idea is to use these sensels to record more color information, rather than focusing in spatial information.

In the same vein, some sensels could have a darker filter on top (e.g. RGgB where the g is 3 stops less sensitive than G) for wider dynamic range.

I wonder though if they don't already capture a gamut wider than any current printer or wide gamut monitor? I thought cameras had pretty huge gamuts.
I'm not really sure though.
Are you comparing images on a wide gamut monitor to your Velvia or on a standard gamut monitor?
You could be right though.

I do know that sRGB monitors take much of the blame for things. As soon as I got a wide gamut monitor, suddenly I saw deep violet/purple flowers looking radically closer to reality than they had before and realized it wasn't something I was doing wrong with processing or shooting but simply my old sRGB monitor clipping away the colors. Red roses looked way more realistic too. Intense fall foliage. Sunsets, suddenly had bright saturated bands show up that had disappeared on my old sRGB monitor.

 

[Jettatore]

Semi-related to the topic. I wrote a macro script that separates the color layer from the underlying value layer of any image for Photoshop to it's own layer, isolated from the underlying values. It's really useful for studying images and how colors work, and I also find it useful for my photo-retouching and digital painting workflows. You can get it free here: http://forums.cgsociety.org/showthread.php?p=6313023#post6313023

This is a slightly more complicated version of the above that also seperates out the high and low frequency details to separate layers. http://forums.cgsociety.org/showthread.php?p=7088384#post7088384

You have to register to download the files posted in those threads but that's free.

 

[LetTheRightLensIn]

Silicon is naturally less sensitive to blue and violet wavelengths of light than it is to green, and it is most sensitive to red. I am not really sure that adding additional pixel types with, say, a violet filter, will really improve the quality of violet in digital photos taken with such a sensor. Current bayer CFA's use fairly broad filter ranges for blue, green, and red pixels. Blue filters allow in light down to around 380nm (very violet), while red filters allow in light up to and beyond 700nm. Green overlaps both just a tad.

Assuming you did actually involve say violet, blue, cyan, green, yellow, orange, red, deep red CFA pixels in some "advanced" sensor design. One way or another, you still need to "coalesce" that greater variety of colors into RGB image pixels in the end anyway. You would have to do some weighted mathematical averaging of blue with violet and cyan, green with cyan and yellow, and red with deep red, orange and yellow. There are a few drawbacks to such an approach, and potentially no real benefits:

- You need to be extremely careful about how you lay each color out in the CFA to ensure that color distribution is effective to capture enough luminance per pixel and still be capable of interpolating an output pixel (today most interpolation is based on 2x2 quads of bayer pixels with slightly weighted algorithms to eliminate color artifacts, zippering, etc.) Fuji's mew X Pro-1 uses 6x6 blocks of pixels to produce an output pixel, and it still only uses RGB albeit with less pattern repetition.
- You need a more complex RAW file format to acomodate the increase in various CFA pixel color types.
- You need considerably more complex interpolation algorithms to convert larger blocks of input CFA pixels into output RGB pixels.
- You need more advanced logic to convert multiple input colors into fewer output channels per output pixel with appropriate color accuracy.
- When working RAW, you need to reprocess the bayer pixels and send them through your entire pipeline of edits to render the image on screen, so more complex interpolation and color conversion will impact the performance of post-process workflows

I think more can be done by moving to a Foveon X-3 style sensor, where each color channel is stacked at a single photosite. Silicon is naturally sensitive to most of the visible wavelengths of light, and considerably more sensitive to near infrared than it is to UV. As such, it also naturally filters light the deeper it penetrates into a silicon well, making it pretty easy to capture blue at the surface, green in the middle, and red deep into the well (which is exactly what Foveon sensors do.) By eliminating an alternating pattern of RGB pixels, you improve a LOT of the characteristics of the sensor directly, and since you have full color data at every single pixel, you also have full color fidelity. Foveon sensors are fairly well known for having richer blues than bayer-type sensors, without the need for a lot of complexity. You also eliminate color moire, minimize monochrome moire, reduce noise in the blue channel, etc.

I think I'd have to generally agree with much this.
Anyway it's complicated stuff.

 

[Jettatore]

I think for an idea along these lines to have a practical impact, there might need to be serious upgrades to the sensor and resolution (phase 1 Medium format++) to even begin to get to a level where there is a shift in hue/saturation that isn't at least recorded along with the value data already. Even if the hue and saturation are 'off' (aka don't match what your eye's see) that's a simple slider shift in editing. However if there are subtle variations in the hue/saturation that are not being picked up by current sensor technology there would be missing data that would not be shift-able via a slider. I'm not sure that there is any missing data capture for hue/sat, that would show up to the human eye, current HD monitors and or high-end printers. I took a test shot of a pillow with a detailed semi-photo realistic embroidery on it with my 7D and then compared it zoomed in on the back of the camera to the pillow, and my eye could not see hue or saturation shifts that were not being captured by the camera accurately. Zoomed in that far I could see each individual thread and the shading and colors were all captured so far as my eye could see.

 

[TrumpetPower!]

I took a test shot of a pillow with a detailed semi-photo realistic embroidery on it with my 7D and then compared it zoomed in on the back of the camera to the pillow, and my eye could not see hue or saturation shifts that were not being captured by the camera accurately.

If you want to know what all the fuss is about, compare the pillow to a print of the photo of the pillow instead of the back-of-the-camera preview. I guarantee you you'll see differences.

If you were doing portrait photography and your model had posed with the pillow, I can all but guarantee you that you wouldn't give a damn about those differences, so long as the model's skin looked good.

And if you took a picture of the pillow to sell it on Ebay or to email to your family, it'd also be just fine.

If you took the picture to include in a product catalog, depending on the client, it might matter or it might be "good enough."

If it were an oil painting instead of a pillow and you were making giclee prints to sell at a hundred bucks a pop for an 8" x 10", it would not be "good enough," though there're those who'll still try to pass it off as such.

And if it were an oil painting that you were recording as part of a preservation effort for a major museum, you'd be laughed out of the room.

For the overwhelming number of users for the overwhelming number of situations, modern DSLRs have unimaginably fantastic color reproduction just as they are. For the overwhelming number of the remainder of users for the overwhelming number of remaining situations, modern DSLRs can be made to function to superlative standards, if you know what you're doing. And, for a very tiny fraction of a percent of the population, modern DSLRs are a toy when it comes to color reproduction.

Me? Most of the time I'm in the first camp, but I spend a significant amount of my photographic time in the second camp. And I'm positively thrilled with my 5DII and I'm in the process of getting my 5DIII all dialed in and anxious to see how well it does. I'm pretty sure that the 5DIII, more importantly combined with some enhancements to my workflow that I'm working on at the same time, will let me actually get some results that might start to rival the stuff they do in major museums -- maybe even good enough for minor museums, which could open up some really interesting possibilities for some really fun gigs.

Cheers,

b&

 

[AJ]

I took a test shot of a pillow with a detailed semi-photo realistic embroidery on it with my 7D and then compared it zoomed in on the back of the camera to the pillow, and my eye could not see hue or saturation shifts that were not being captured by the camera accurately.

If you want to know what all the fuss is about, compare the pillow to a print of the photo of the pillow instead of the back-of-the-camera preview. I guarantee you you'll see differences.

If you were doing portrait photography and your model had posed with the pillow, I can all but guarantee you that you wouldn't give a damn about those differences, so long as the model's skin looked good.

And if you took a picture of the pillow to sell it on Ebay or to email to your family, it'd also be just fine.

If you took the picture to include in a product catalog, depending on the client, it might matter or it might be "good enough."

If it were an oil painting instead of a pillow and you were making giclee prints to sell at a hundred bucks a pop for an 8" x 10", it would not be "good enough," though there're those who'll still try to pass it off as such.

And if it were an oil painting that you were recording as part of a preservation effort for a major museum, you'd be laughed out of the room.

For the overwhelming number of users for the overwhelming number of situations, modern DSLRs have unimaginably fantastic color reproduction just as they are. For the overwhelming number of the remainder of users for the overwhelming number of remaining situations, modern DSLRs can be made to function to superlative standards, if you know what you're doing. And, for a very tiny fraction of a percent of the population, modern DSLRs are a toy when it comes to color reproduction.

Me? Most of the time I'm in the first camp, but I spend a significant amount of my photographic time in the second camp. And I'm positively thrilled with my 5DII and I'm in the process of getting my 5DIII all dialed in and anxious to see how well it does. I'm pretty sure that the 5DIII, more importantly combined with some enhancements to my workflow that I'm working on at the same time, will let me actually get some results that might start to rival the stuff they do in major museums -- maybe even good enough for minor museums, which could open up some really interesting possibilities for some really fun gigs.

Cheers,

b&

I'm satisfied with the color response most of the time, but there are times when I'd like deeper reds, and more details in those reds (blows out so easily on a DSLR). Ditto for deep violets and blues when shooting sunsets. Greens on the other hand are really well represented with RGB.

I can get close to the colors I want by shifting hues and color-balance. Contrast and saturation doesn't get you into that part of the gamut, it just tends to blow things out. That said, it's a past-processing fix of something I'd like my camera to capture in the first place.