How deep learning enables machine vision solutions

Deep learning enables machine vision solutions for applications where defects or inspection criteria cannot be easily quantified or mathematically defined. It offers machine vision designers a powerful new tool for advanced inspection, assembly, and quality assurance applications because the software essentially helps the designer develop the best possible algorithm. But until recently, creating deep learning software required traditional programming skills, preferably combined with a strong grounding in statistics and traditional machine vision system design.

Today, thanks to deep learning – enabled smart cameras, gateways and application-specific coding, deep learning for the masses may be a few clicks away.

Smart cameras get deep

The deep learning process is broken into two parts: training and inference. Training involves the optimization of a neural network by feeding it images of “good” products and “bad” products. Human operators tag the images as good or bad, and the deep learning software performs a statistical analysis to create a weighted table. The training process is very computationally intensive, prompting many designers to use high-end workstations for the training step or to leverage cloud resources.

The resulting statistical matrix from the deep learning software is the heart of the second step in deep learning: inference. Inference involves using the deep learning algorithm to solve real-world problems. This part of deep learning software can run on a much wider variety of computer platforms, ranging from high-end PCs to smart cameras. Now, rather than using TensorFlow, Caffe, or another complex deep learning data visualization program to train a deep learning algorithm, users can turn to other solutions. For example, Cognex’s newest smart camera, the In-Sight D900, makes deep learning programming as simple as creating a spreadsheet.

Cognex’s In-Sight series of vision systems pioneered the use of spreadsheet programming to simplify the design and deployment of machine vision solutions using a smart camera platform. By combining its ViDi deep learning software with the In-Sight platform, Cognex hopes to bring deep learning solutions that can perform more complex assembly verification.

“The D900 is the first deep learning machine vision system that does not require a PC to deploy,” explains Brian Benoit, Senior Manager Product Marketing at Cognex. “Users are able to train the deep learning software through the In-Sight software interface on a PC. Once the training step is complete, the job is loaded on the camera and you’re ready to deploy.”

According to Benoit, the D900 is specified for applications running up to 150-200 inspections per minute. Cognex achieved this performance by upgrading In-Sight’s internal processing power while leveraging the speed inherent to an embedded solution.

Gateway brings deep learning to every camera

Not every machine vision customer that wants to leverage deep learning will be willing to replace existing machine vision hardware with new platforms, however. To bring deep learning to existing installations while simplifying the image acquisition and tagging that is so important to deep learning success, machine vision network specialist Pleora has launched its AI Gateway. Based on the NVIDIA GPU framework, AI Gateway sits like a frame grabber between the camera and the processor or other computational engine.

With a single gateway, Pleora can provide deep learning machine vision capability for multiple “dumb” cameras, according to Jonathan Hou, Pleora’s Chief Technology Officer. “Customers may have the best deep learning algorithm, but how do they deploy it to the factory floor? Our solution allows them to keep their installed machine vision hardware base without forcing customers to choose any particular manufacturer’s deep learning solution path.”

Never stop learning

While most deep learning systems have a hard stop between the training and inference steps, so that an algorithm’s effectiveness isn’t deluded by images that haven’t been expertly tagged or graded by humans, some solutions might be improved if they never stop learning.

For example, barcode and traceability experts at Datalogic first applied deep learning technology to their image-based identification systems. “We’re heavily focused on anything around traceability, so we’ve applied deep learning to identification, code reading and some pattern recognition in general,” said Bradley Weber, product marketing manager at Datalogic. “When we look at a label on a package, obviously we want to read the barcode. But the most robust solution will also read the numbers under the barcode, to and from shipping information – any and all marks on the box. Every bit of data you collect on a package increases the chances that you’ll be able to trace the package, even if the barcode or labels are damaged.”

Weber said Datalogic’s focus has been on embedding the power of deep learning in its products so that customers do not need to access the underlying code. “When we look at who installs traceability solutions, they are not going to be a vision engineer,” he said. “So the solution needs to be simple and error-proof. For example, we will soon be releasing our Smart-VS, a vision sensor that uses artificial intelligence. Without programming, a user can push a button to teach what is a good item, push the button again to teach what is a bad item, and have the sensor will learn how to reject or sort the item.”

According to Weber, traceability applications like code reading and OCR could benefit from continuously learning about how a customer’s packaging changes over time. The system could learn new fonts for OCR – possibly even handwriting.

“Right now, processing is so cheap that we’re focused at putting computational power at the edge, which is enabling deep learning but also limiting our ability to train on the fly,” Weber said. “As 5G technologies roll out, we can imagine the momentum shifting back to the network and away from microprocessors. I suspect the back and forth between more powerful computer systems and faster networks will continue to guide how we deploy and use deep learning in machine vision applications. Anything that allows us to automate transportation and traceability – especially during these crazy times of COVID-19 – is a very good thing.”

Winn Hardin is contributing editor for AIA. This article originally appeared in Vision Online. AIA is a part of the Association for Advancing Automation (A3), a CFE Media content partner. Edited by Chris Vavra, production editor, CFE Media, cvavra@cfemedia.com.

Original content can be found at www.visiononline.org.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.