Every once in a while a wild idea pops into our head, a dream or a vision of something that does not yet exist, but that seems cool and (to us, anyway) compelling. "What if," we think, "this really existed? Wouldn't that be wild?" The other day I had such a daydream and I want to run it by you. It was about how images might be captured in future digital cameras.
As is, digital image capture consists of reflecting a picture of a subject out there in the real world onto a small flat sensor that breaks down the image into a flat array of pixels. The sensor records the color and brightness of each dot. Those dots are then displayed on a screen or they are printed out. As we look at the dots on the screen or on a print, our brain fills in many blanks and "sees" an actual three dimensional picture though the image itself is just two dimensional.
Now imagine, if you will, that we'd capture not only the color and brightness of a pixel, but also the distance between where it sits in the real world and its location on the sensor in the camera. That would literally add a whole new dimensions to image capturing.
What I am proposing seems entirely possible as the focusing mechanism in today's cameras already measures the distance to objects very accurately. Capturing distance data for an entire image could therefore be accomplished via a full image scan by a somewhat enhanced focusing mechanism. The scan would then record distance values for each pixel, or in the beginning perhaps for each group of pixels. That distance data would be added to a pixel, or group of pixels, as an additional 8-bit datapoint. That way, each pixel would now not only have a value for color and brightness, but also one for its distance from the camera. Eight bits would allow for 256 levels of distance, and it wouldn't add too much size to an image file. Actually, I don't think image size matters at all, given ever increasing storage capacities and computing power, and cameras might offer different degrees of distance capturing. 16-bit capturing, for example, would contain over 65,000 distance values for each pixel or group of pixels.
With that distance information now part of a picture file, image manipulation would assume an entirely new form. You could, for example, instantly sharpen a background or change its tint or lightness without the use of 2-D masks and other such cumbersome crutches. You could do that because the imaging program would simply have to read the distance of the pixel away from the camera and manipulate pixels that fall into a certain range. You could request only pixels that are between three and six feet away from the lens, or only those in the far distance. You could then apply blurring or sharpening only to certain areas. The ever-difficult manipulation of depth of field would no longer be solely a function of actual camera settings; it could also be changed and managed inside a graphics application. The possibilities for unbelievably interesting image manipulation would be endless. It would mean the beginning of a new generation of plug-ins and image manipulation software. Named after my initials, these wonderful new Comprehensive Holo Block .chb files would totally change and enrich the art of image processing!
Now imagine going even beyond that. Imagine that adding new kinds of data to common image capture files might not stop with just distance but that we'd capture other data as well. Moving elements within the picture, for example, could be identified by a sensor in the camera and their movement geometrically described, with pattern recognition algorithms filling in the blanks.
Eventually, such recognition methods could be tied into software lookup tables and databanks that could instantly call up the full data of an object and thus supplement what the camera initially sees. For example, the camera electronics would recognize a particular model car and call up all of its data. Instead of a static shot of that car, the picture would have at its disposal a full 3D rendering of the car, plus whatever other data might come in handy.
That way, one single snapshot might enable you not only to manipulate the resulting image in most interesting ways, but it could also become a moving picture, a 3D image far more versatile than an actual movie which is, after all, really only a sequence of still pictures. Taking it one step further, the databank might also add additional values to such objects, like what they look like at different times of the day, in different seasons, or at different ages. That way, a single picture could, in a way, capture time itself by being able to project a scene backward and forward in time.
The more I thought about all this, and the further I projected it into the realm of what might be possible, the more I believed that this may well be the future of digital imaging. Some day, we'll look back and wonder how primitive things had been in the days when cameras, even digital ones, were dumb and only measured color and brightness of dots on a little chip.
