4D vision’s benefits for robotic automation

Machine vision is a critical aspect for manufacturers, who are applying the technology to robots for many automation technologies. After 2D vision, 3D vision systems added depth to the camera sensor. Now, Apera AI CEO Armin Khatoonabadi wants to go one step further to 4D.

Khatoonabadi said 4D vision isn’t so much as going deeper as it is going smarter in his presentation “4D Vision & How It’s Revolutionizing Robotic Automation,” which was presented virtually at Automate Forward, an A3 online conference and exhibit during the week of March 22. The idea behind 4D vision is making robot imaging faster and more adept than 3D vision, which is too slow and not as intelligent.

While the fourth dimension is thought of as time, in this case it’s more about expanding the robot’s intelligence, which is limited by its programming. “4D is the intelligence that comes when the image is captured and continues to where the robot goes and makes decisions,” he said.

Five potential 3D vision challenges and drawbacks

While 3D vision systems have advantages and can be useful for specific applications, 3D vision does come with some major drawbacks. Some of the 3D machine vision challenges are enough to scare manufacturers and system integrators away and can include:

1. Wastefulness: The whole scene is captured and processed throughout the imaging pipeline, which leads to a tremendous amount of data. Very little of it is actually useful.
2. Missed surface details: Very little 2D information is included in the process.
3. Unneeded expense and programming time: Given the choice, Khatoonabadi said manufacturers and system integrators would rather work around 3D vision because it’s not worth the time and expense.
4. Inability to produce a high-quality point cloud, critical for better object recognition: 3D cameras aren’t always good at this if the settings are less than optimal.
5. Slowness: Because a lot of data needs to be processed, the robot may take much longer than necessary deriving the correct result.

How 4D vision works

In a way, 4D vision is a throwback because it uses 2D cameras rather than 3D. The key addition to the process is artificial intelligence (AI). Khatoonabadi explained how a scene is processed. The 2D camera captures the scene and then the AI extracts 2D information such as shadows, edges, texture and color—like a human would—while performing object recognition and pose estimation. Then a pose of the object is sent to the robot to help it better perform the task like a 3D camera would.

The information supplied to the robot is less of a strain on the system while still allowing the robot to perform its task. Using AI and more freedom with the software also allows the robot to work with more flexibility, which is to the system’s benefit.

“We use more AI. Extracts a lot of information. Doesn’t pay attention to only the edges. It pays attention to occlusion as well as textures, shadows,” he said.

Khatoonabadi showed a few short video examples of robots picking and placing metal objects quickly and accurately in a matter of seconds. These objects ranged from metal spikes to jars of skin cream.

“The system is trained on geometry and shape of picture,” Khatoonabadi said. “We have a lot of control over what it should do and what it should ignore. We can tell the system to ignore a piece of geometry, and this gives the user a great deal of control.”

A little bit of old technology is paired with new technology to put a new spin on machine vision and its potential for robot manufacturers.

Chris Vavra, web content manager, Control Engineering, CFE Media and Technology, cvavra@cfemedia.com.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.