Camera developed to recognize objects faster and more efficiently

Researchers at Stanford University have devised a new type of camera system that can classify images faster while being energy efficient by combining two types of computers that is designed specifically for image analysis.

By Andrew Myers, Stanford University September 8, 2018

Researchers at Stanford University have devised a new type of camera system that can classify images faster while being energy efficient. The hope is they will one day be built small enough to be embedded in the devices themselves.

The image recognition technology used in today’s autonomous cars and aerial drones as well as tomorrow’s cancer-seeking robotic medical devices, all depend on artificial intelligence (AI). These are designed to teach themselves to recognize objects such as a pedestrian crossing the street, a stopped car, or a cancer tumor.

"That autonomous car you just passed has a relatively huge, relatively slow, energy intensive computer in its trunk," said Gordon Wetzstein, an assistant professor of electrical engineering and computer science at Stanford, who directed the research.

Wetzstein and Julie Chang, a doctoral candidate in his lab, have married two types of computers into one, creating a hybrid optical-electrical computer designed specifically for image analysis.

Consumed by computation

The first layer of the researcher’s prototype camera can be thought of as an optical computer. Optical computers do not require the power-intensive mathematics of digital computing. The second layer is a traditional digital electronic computer This optical computer operates by physically preprocessing image data, filtering it in multiple ways that an electronic computer would otherwise have to do mathematically.

Since the filtering happens naturally as light passes through the custom optics, this layer operates with zero input power. This saves the hybrid system a lot of time and energy that would otherwise be consumed by computation.

"We’ve outsourced the math into the optics," Chang said.

It’s like a camera that takes multiple images of the same scene, as if each variation was taken through a specially designed filter. The images are captured optically, just like a photograph on film. Each image captured in that instant would have to be extracted mathematically with electronic computing. The result is fewer calculations, fewer calls to memory, and far less time to complete. Having leapfrogged these preprocessing steps, the remaining analysis proceeds electronically with a considerable head start.

"Millions of calculations are circumvented and it all happens at the speed of light," Wetzstein said.

Rapid decision-making

In speed and accuracy, the prototype rivals existing electronic-only computing processors that are programmed to perform the same calculations, but it provides substantial computational cost savings. While their current prototype, arranged on a lab bench, would hardly be classified as small, the researchers are confident their system can be dramatically miniaturized to the point that one day they could be embedded within the form factor of a handheld video camera or an aerial drone.

In both simulations and real-world experiments, the team used their system to successfully identify airplanes, automobiles, cats, dogs, and more within natural image settings.

"Some future version of our system would be especially useful in rapid decision-making applications, like autonomous vehicles," Wetzstein said.

Wetzstein, Chang, and their cohort at the Stanford Computational Imaging Lab are busy developing the next generation of their design. They are looking at ways to make the optical component do even more of the preprocessing, something Wetzstein describes as making it "more expressive." And, of course, there is work to be done shrinking the scale.

Stanford University

Stanford University 

– Edited by Chris Vavra, production editor, Control Engineering, CFE Media, cvavra@cfemedia.com. See more Control Engineering discrete vision and sensor stories.