where there’s smoke there’s forum fire
I was just reading an article on Luminous Landscape about the future of camera sensor as they increase MP, were the limit is, and the future of the Bayer filter Array type sensor. The Canon 7D has shown us the diffraction problem with higher MP at F8 and above.
Will the change to have a pattern dominated by the color blue, which is sharper than green or/and will they use binning of pixels to get better performance at both high ISO and smaller aperture
Pete
I think the Bayer filter has still got some legs left. I think a possible candidate to replace it is the reverse of what they do with those one laptop per child computer screens (edit: actually, now that I read this it looks like they went away from the prisms, but they still have the image that shows the concept—the first one on the right if you look at that link). They use a prism to split up the light into its color components just before it goes through the pixels of the screen, and the lack of a color filter lets you pass almost all the light, instead of throwing light away by absorbing it in a color filter like a standard display. Someday I'll bet they'll have microprisms that do something similar on imaging sensors—split the light before it hits three separate photodiodes for each pixel. Or maybe they could have more than three channels(?). I think it would be fun having channels for near-IR, red, orange/yellow, green, blue, and violet. You could save an image based only on the IR channel, for example.
While we're on the subject of the bayer filter, what do you guys think about a camera with a modified sensor (maybe DX to keep it inexpensive) that has no bayer filter and no AA filter to shoot black and white only? If you did this to the D300s sensor, it would give you a camera with a base ISO of around 800 with noise there comparable to the D300s at ISO 200, but it would buy you around two stops on the high-gain end, as well (compared with acceptable noise at ISO 3200 on the bayer pattern sensor, you would have acceptable noise at ISO 12800 on the non-bayer sensor). I know it's a niche market, but I'm sure a few people would buy them and it doesn't seem like it would be too hard to implement.
I'd get one Jonny... I do half my stuff in black and white anyway. I've been wishing somebody would do this for a while actually.
Does anyone feel like donating their camera to try it out (or know of someone who has)? I'll bet acetone would take off the bayer filter (unfortunately, it would probably also take off any microlenses on the sensor as well).
Unfortunately it is not enough to just remove Bayer color array filter you must change the whole camera, or at least the electronics and firmware to get monochrome readouts from the sensor.
It would look like gobbledygook until you convert to gray scale in a raw converter or photoshop, but as far as the amount of light measured by each pixel it seems to me like numbers are numbers. As long as you keep in mind that the color channels are no longer actually color channels there shouldn't be a problem. JPG shooting would be out, you're right about that.
The interpolation to RGB pixel values happens post-NEF, no? If you remove the Bayer filter (assuming the previous statement is correct) all the data you need should be there in the already known NEF format, ready for your personal reinterpretation of the values, no? The advantage (disadvantage?) this would give you would be a trade of fast base ISO for resolution, as you could use all the photosites individually instead of (as is current) combining them into pixels. (which is what the "convert to grayscale in post" procedure will continue to do)
EDIT: I would expect the screen to be worthless, though, for if Thom Hogan is correct (and I have no reason to doubt him) the images displayed are post image adjustment and post jpg processing. (See his (believe it was him) discussion on WB and clipping / histogram values on the camera display.)
EDIT 2: You would also need to crank the exposure compensation permanently if you wanted to use your in-camera meter, for even if you did continue to combine multiple photosites into pixels the lack of filters will through off the ISO calibration.
I agree with everything you said, Soap.
Regarding the sensor with prism to split the light, Nikon already has a patent on that :-)
Kodak has a patent on having a colourless sensor included with the coloured ones ie every other sensor pixel has no color on it, so you in effect have what you want(higher iso) and still keep the colour. Its going to be really fun when these patents get implemented.
To be able to get sequential monochrome readouts from the sensor (as in monochrome image file) you will have to alter the way camera reads the data.
Red, Green, Blue and second Green readouts (ordered for the specific sensor) are already there in the NEF (or any other RAW) file, interpolation just tries to reconstruct (more or less successfully) RGB levels for any single pixel on the image. The less successful interpolation is seen as moiré or blur on the image and is the reason why Hasselblad developed 4 shot method to overcome this issue.
I have seen results of the work some people have done by removing Bayer filter and it is not something to be excited about. They have tried to reconstruct monochrome image from a RAW data but there is no out of the box RAW converter that can do that, and I doubt making a NEF converter would be an easy task.
This is much easier done on D40 for example then on D300 because D300 has a couple of optical glass filters in front of Bayer color array. Best use is probably IR imaging (by putting IR pass filter) because sensor without Bayer filter is actually the only way to capture real IR light and resulting resolution and noise is not of much concern.
http://camerahacks.10.forumer.com/a/bayer-filter-removal_post3496.html
True elimination of Bayer filters for color imaging is a ways off but improvements on it are near. Sony has a patent application that alternates white and color filters. Key difference with this design is they've managed to increase well depth and therefore dynamic range. Net effect of this design should be doubling of sensitivity over standard Bayer filters.
Look at Fig. 4 in the patent to get an idea of how it works:
astro: Thanks for that.. Looks interesting .. however it is replacing one of the greens with a white. not "alternating white and colour". Still it may be better than the Kodak version. as it seem to have a method to aggregate groups of sensors and then generating pixels from the various configurations. This has been suggested in the past by someone on the internet. They even provided mathematical algorithms for this.
Lately we have been talking about sensors, color and B&W and Bayer interpolation and why do we hate it my precious, so I thought maybe you would not mind seeing my point of view on the subject.
As we all know Bayer color array looks like this:
Suppose we are interpolating image pixels 6 and 7. Simple Bayer interpolation actually goes like this:
Image pixel 6 red, green and blue components would be:
IR6 = (SR2 + SR10) / 2
IB6 = (SB5 + SB7) / 2
IG6 = SG6
And image pixel 7 red, green and blue components would be:
IR7 = (SR2 + SR4 + SR10 + SR12) / 4
IB7 = SB7
IG7 = (SG3 + SG6 + SG8 + SG11) / 4
Pretty cool ha, just by using few simple operations we can actually have color image resolution same as monochrome (B&W) one instead of 4 times smaller. Alas this works well for large uniform-colored areas but not so great otherwise. This is the result of a simple Bayer interpolation on a generic high contrast images:
Not so good. Some of the sharpest film era lenses are accused of having CA on digital cameras because of this. Actually this is masked in newer cameras by using optical low pass filters and software algorithms to blur the image.
There has been some alternative color arrays proposed for some time, this one for example uses shifted larger surface octagonal sensor pixels and color array rotated by 45 degrees with additional transparent or “white” elements:
In this example the interpolation uses chromatic information from the surrounding color pixels and luminance information from “white” pixels. Every image pixel corresponds to one “white” pixel that should give much better contrast and luminance sensitivity similar to a human eye. The additional bonus is that you get pure panchromatic (B&W for example) same resolution image in the same RAW file just by using “white” pixel readouts, and you could use this same sensor without conversion for IR photography for example just by swapping filters because “white” pixels are sensitive to all light visible or not. The drawbacks are you use twice as many pixels as Bayer for the same resolution and there are currently no fast enough algorithms for video processing. Resolution actually should not be much of a problem because with lower blur you could go for much higher pixel densities (P&S sensors have very large pixel densities today). As for video better algorithms could be developed or we could use this same color array in an improved Bayer interpolation:
Here we use the same simple Bayer interpolation with luminance adjustment acquired from the “white” pixels. In live view or video we could also use the information from the “white” pixels not only for image but, because of higher sensitivity and contrast, for much faster and more accurate metering and AF. Lab results from this setup and by using same sample images as for Bayer will give almost perfect image:
So the bottom line it is not the question is it possible to make better alternative to Bayer array, the question is will the customers will be willing to sacrifice half the resolution for the benefits they will get from it.
cool post mb!
Right, hearty. This is great, mb. I had never even thought about the CA-like artifacts before.
mb are these your diagrams? If not can you just mention where you got them from?
Actually most of it was mine ... some older some newer ...
Great mb! Just checking that's all. Thanks for the great info! I like your friendly educational style! :^)
You must log in to post.
