Four machine vision innovations for industrial automation

From optics and lighting to smart cameras to artificial intelligence (AI) and machine learning (ML), machine vision is growing in industrial automation and changing in many ways. Four innovations are highlighted.

By Andrew Abramson April 11, 2022
Courtesy: Grantek

 

Learning Objectives

  • Machine vision is one of the fastest-evolving segments in industrial automation.
  • Innovations include advances in web browser interfaces, artificial intelligence (AI) and lighting.
  • These advances are tied into the Industrial Internet of Things (IIoT), which integrates technologies to improve overall plant-floor intelligence.

Machine vision has consistently been one of the fastest-evolving segments in industrial automation. While the fundamentals behind machine vision have changed little over time, manufacturers continue to find creative ways to stretch these fundamentals into new and innovative products. Machine vision innovations have taken many forms, although the industry has been most influenced four machine vision technological advancements:

  1. Smart camera prevalence
  2. Optics and lighting
  3. Web browser interfaces
  4. Artificial intelligence/machine learning (AI/ML).

1. Smart camera prevalence

In industrial machine vision applications, the system will acquire an image, perform some automatic inspection or analysis on the image, and then affect a change on the system based on that inspection or analysis. The processing horsepower performing the inspection or analysis on the image could be in one of many locations: a PC on the plant floor, in the cloud, or on the camera.
With the convergence of information technology (IT) and operational technology (OT) networks along with increased awareness of industrial cybersecurity in the OT space, customers have been removing thick client PC assets from the production floor and offloading those tasks to virtual environments, either on-premise or in the cloud. Machine vision PCs are no exception.

Customers’ IT departments are requesting that PC assets are not added to the plant floor unless it’s necessary.

Using a smart camera eliminates the need for another plant floor PC, reducing security risks and IT maintenance. For an application to be suitable for a smart camera as opposed to a PC, the smart camera processor must be fast enough to keep up with the demand, a factor reliant on both the line speed and complexity of inspection.

Moore’s Law predicted a regular pattern of increases in the number of transistors in integrated circuits. The principle is still referenced today, which affects computational power available in modern integrated circuits. The machine vision industry has dependably taken advantage of this in what seems like ever-changing new camera models, especially relative to other industrial hardware.

Where a programmable logic controller (PLC), photoeye or variable frequency drive (VFD) part number can stay valid for years or decades, smart camera manufacturers are introducing new part numbers every year or two. This is because computational power has a direct impact on the ability to process images expediently, and manufacturers must leverage available modern integrated circuits or fall behind the competition.

While there are still applications where the processing power of a PC is necessary, with the growing capabilities available to smart cameras, they are absorbing more of those applications. Smart cameras can even now handle advanced applications such as 3D and artificial intelligence.

Figure 1: Machine vision detects pepperoni in a veggie pizza. Courtesy: Grantek

Figure 1: Machine vision detects pepperoni in a veggie pizza. Courtesy: Grantek

2. Machine vision optics and lighting

The adage goes “garbage in, garbage out,” and the same is true with machine vision. The processing results are only as good as the image provided. The image must present the features in a way so they can be analyzed. As a result, machine vision manufacturers have devised exciting new ways to reveal features that previously remained unseen. Outside of the ever-expanding resolution capabilities enabled by modern lenses and vision sensors, there have been other compelling advancements in optics and lighting.

Machine vision solutions for varying working distances

A common challenge for machine vision systems has always been applications where the distance between the camera and the inspection point varies. For example, this could take the form of a single line that runs multiple different sizes or heights products. Many times, the requirements for range of working distances and depth of field stretch beyond the ability of a single lens. In the past, this has been tackled through relocating the camera during changeover, multiple cameras or using lenses that could mechanically adjust their focus.

Liquid lens technology is a modern solution to these problems. Previous mechanically-focused lenses solved the problem of varying distances, but they had slow response times and faced reliability issues because of moving parts. Liquid lenses, and their lack of moving parts, have solved these problems.

Recently, cameras have become available that can conduct onboard distance measurements in addition to dynamic focus adjustment. When the product presents itself under the camera, the onboard laser distance sensor can detect the working distance and automatically adjust to the proper focus via the liquid lens. This is enormously useful for an application where product may be coming down the line with unknown height variations.

Specialized lighting for machine vision

Extracting subtle detents, etchings, or embossed characters on a surface can present challenges to vision systems. Combinations of traditional lighting could extract some features but rarely the faithful form of an indentation or raised surface. Instead of lighting from a single source, manufacturers now allow for multiple light sources targeted at the same location from different directions to alternately burst their lights and capture an image with each burst. Those images are then intelligently stitched together to form a single image which can expose those small indentations or other deviations from an even surface.

Steerable mirrors for machine vision

It may feel like a step back in time to the spinning mirror 1D raster laser scanners, but mirrors are being used for a new purpose in machine vision. Vendors are employing controlled mirrors to use and redirect camera to multiple locations over a larger area, effectively doing the work of multiple cameras. Of course, there are speed considerations, but for many applications such as pallet label barcode scanning, this can make an otherwise unreachable solution affordable.

3. Web browser interfaces for machine vision

The ability for operations and maintenance to interact with cameras is critical for prompt troubleshooting and resolution of issues. This has required proprietary hardware or software to view images or change programs or configurations.

However, newer smart camera models have changed this with a web server built into the camera itself, enabling display and configuration with only a web browser. A typical human-machine interface (HMI) on a piece of equipment with machine vision needs to interface with both the camera and a control system such as a PLC. A web interface on the camera now allows for the camera interface to be seamlessly integrated into an industry standard HMI application via a simple web browser component.

If a camera is out of position or focus, a maintenance technician can now view the camera interface from a phone, tablet, or maintenance laptop without the aid of fixed HMI at the line.

Figure 2: Optical character verification (OCV) is shown on transparent material. Courtesy: Grantek

Figure 2: Optical character verification (OCV) is shown on transparent material. Courtesy: Grantek

4. Artificial intelligence/Machine learning (AI/ML)

Over time, we have been regularly requested to inspect a product and confirm it as “good” and free of defects or foreign material. With traditional machine vision tools, this was always a challenge. How do we define what “good” and “bad” are? How can we detect a defect we aren’t anticipating and haven’t seen before?

Through a combination of tools, users may be able to make sure an area is clear of debris, or that an edge is clean, or confirm another quantifiable metric, but it requires the user know and define in advance what the pass/fail metric is.

Imagine trying to define for a vision system through quantifiable requirements what a hair might look like on top of a frozen pizza in any length, curl, color, bend or orientation on an ever-changing background. It’s a near impossible task. For the machine vision system to decide what was a hair and what was cheese used to be science fiction, but it is now a real obtainable solution for machine vision applications.

Instead of defining discrete pass/fail metrics through a combination of vision tools, we can capture some number of “good” and “bad” images, and then provide them to the machine vision system. Using these provided images and some initial supervision in the form of defect classifications, the system creates a neural network mode it will leverage for future inspections.

The concept of artificial intelligence (AI) to analyze images is not new. In fact, many of the largest technology companies offer commercial solutions for AI image analysis alongside other artificial intelligence offerings. They can leverage their computing power in the cloud to create complex neural network algorithms that can be exploited by edge devices on the plant floor. Applications for this technology extend well beyond manufacturing, but with the rise of Industry 4.0, tech giants are specifically targeting manufacturing applications.

Another difficult application is optical character verification (OCV) inspection for on-demand printing on the production line. This is ordinarily a code date the manufacturer wishes to confirm was printed legibly and accurately. On-demand printing systems, such as inkjet, laser or print-and-apply, will vary in quality, and fonts are not always consistent from print to print, which has been a challenge for machine vision systems. There is an added layer of challenge when the printing is on a clear surface with a changing background of product which can create contrast issues at the outlines of the characters. With the help of AI cameras, the neural network model can now better distinguish between the character and the background behind it.

Andrew Abramson, CAP, PMP, director of client success, Grantek. Edited by Chris Vavra, web content manager, Control Engineering, CFE Media and Technology, cvavra@cfemedia.com.

MORE ANSWERS

Keywords: machine vision, artificial intelligence

CONSIDER THIS

How can machine vision advances improve operations on your plant floor?


Andrew Abramson
Author Bio: Andrew Abramson, CAP, PMP, director of client success, Grantek.