HMI, IIoT, and AR technologies: Parts of the Industry 4.0 revolution

Industry 4.0, the next industrial revolution is underway, and human-machine interface and Industrial Internet of Things (IIoT) and augmented reality (AR) technologies are helping. Here’s how they can help your applications.

By John Clemons and Justin Robinson November 8, 2020


Learning Objectives

  • Understand how human-machine interface (HMI), Internet of Things (IoT) and augmented reality (AR) help Industry 4.0 initiatives. 
  • Define HMI, IoT, and AR and how they can help manufacturing and process control applications. 
  • Learn how technologies integrate to help improve industrial applications. 

We are well into the fourth industrial revolution, but it’s turning out to be much more of an evolution than a revolution for most people. We have new generations of technology, but little that’s truly revolutionary. The real revolution is how the different pieces of technology work together to be far greater than the sum of their parts.

Thus, it’s not any one technology all by itself that’s the revolution. It’s technologies like artificial intelligence, augmented reality, virtual reality, adaptive manufacturing, and edge and cloud computing coupled together that’s the real revolution. The benefits of these new generations of technology aren’t additive, they’re multiplicative. Together, they are far greater than any single technology by itself.

As an example of this, let’s look at three evolutionary technologies that, when coupled together, might be a small step to a real industry 4.0 revolution.

Human-machine interface, Industry 4.0

The human-machine interface, or HMI for short, has been around in various forms since the very first industrial revolution. After all, humans have always had to interface with machines one way or the other. If nothing else, just to turn the machine on or off and to get it to do what it needs to do.

Today, the HMI is thought of as software running on an industrial computer providing the interface between the human and the control system. This software is almost always highly graphical in nature, with an entire engineering discipline arising around the concept of high-performance HMI.

The concept of the software-based HMI has always been challenging. When someone takes an entire control room full of pushbuttons, switches, loop controllers, chart recorders, and other control devices and then somehow shrinks that down to a couple of 20-in. monitors, something is going to get lost in the translation.

Today, the situation is even worse. The monitor might be bigger and higher resolution, but the amount of information available for display has grown. Manufacturing plants have grown more complex. Timeframes continue to shrink. The span of control continues to grow with information now available from every square inch of the manufacturing facility. Couple that with data from the supply chain, suppliers, distributors, customers, and sales and marketing, and the amount of data that has to make it on a 30- or 40-in. hi-res monitor is staggering – and overwhelming.

Industrial Internet of Things Technology, Industry 4.0

Today, Industrial Internet of Things (IIoT) technology is leading the way in the manufacturing user experience, providing unique and innovative tools and solutions that aren’t tied much to traditional paradigms.

Most of the paradigms for user experiences in manufacturing have grown out of the older generations of the technologies. Physical pushbuttons look a certain way so that’s the way they look on the screen. Pumps look a certain way on the P&IDs so again, that’s the way they look on the screen. And so it goes as one generation of technology evolves to the next, but the look and feel remains the same.

That doesn’t have to be the case. A pushbutton, a pump, or anything can look any way they need to look. That’s the value of the software – it can be designed to look and work any way it needs to.

Some of the most advanced user experience solutions being developed with IIoT technology are based on gaming paradigms and not on traditional manufacturing user-interface paradigms.

Think of the best games and their user experiences and notice the similarities. Using simple and clean reticles. Multiple variations of heads-up displays. Minimizing the information shown and not clogging  up the screen with marginally useful data. Providing quick access to contextual and relevant data, usually through icons in the corners of the screen. Always focusing on relevant data in the right context. And maybe most importantly, providing simple and easy actions with minimal effort.

The objective of these user experiences is simple – to maximize situational awareness. The definition of situational awareness is also simple – the gap between the human’s understanding of the situation and the actual situation. The objective is getting this gap to zero. This is becoming more possible with a great user experience.

Augmented reality, Industry 4.0

Augmented reality (AR) is a user experience that combines the real world with the computer world. It’s typically a view of the real world that is augmented, or enhanced, by computer objects and information. Not only does AR combine these two worlds, it’s both real time and interactive so it becomes a powerful interface between humans and the world. It’s expanding to be more than just visual; there’s ongoing research into sound, touch, smell and even taste.

Most people believe they must physically view the real world through the typical AR glasses or goggles for AR to work. In the manufacturing world, it’s often not practical or even needed.

Remote cameras, robot cameras, co-bot cameras, drone-based cameras, cameras for remote manipulators and the like are better options in the industrial world. All can support AR.

AR allows operators to view the real world, supplement that view with computerized information that has a situational awareness gap of close to zero and make the right decisions and take the right actions.

The next step with Industry 4.0

Each of these technologies evolved out of older technologies and each has its own niche in the industry 4.0 world. They’re cool and very hi-tech, and all solve very specific problems and provide unique capabilities, but individually, they’re all just evolutionary.

They don’t have to work individually, though. They can be combined and create a new user experience that goes way beyond typical HMI solutions. They combine the best of the IIoT user experience and add AR as an integral part of the complete solution rather than an afterthought.

The combination is HMI, IIoT and AR technology working together and providing capabilities none of the three technologies can provide by themselves. When they work together, they can provide a seamless interface between the humans, the real world and the computer world.

Combining HMI, IIoT and AR technologies allows manufacturing complexity to advance because it means the human ability to deal with the complexity isn’t the limiting factor. It supports lights-on manufacturing, which allows computers to do what they do best and humans to do what they do best in a collaborative effort.

It allows manufacturing velocity to increase because humans can better harness the speed of the computers and achieve results and complete tasks faster. It supports greater productivity and lower costs because the people become sophisticated “pilots” of the manufacturing facility, controlling the entire facility with ease and confidence.

Technology combinations for Industry 4.0

We are in the fourth industrial revolution. It can be a real revolution, but we must combine the most powerful technologies to create the real revolution.

HMI, IIoT and AR are evolutionary by themselves, and help solve significant manufacturing challenges. But when the best of the best technologies are combined, they create something new and powerful, with benefits that are far greater than any single technology. That’s the revolution of the fourth industrial revolution. HMI, IIoT and AR technology working together can provide a strong next step in that revolution.

John Clemons is a senior consultant with Rockwell Automation and Maverick TechnologiesJustin Robinson is a global program manager for Rockwell Automation specializing in process industry and information solutions. Maverick Technologies is a Control Engineering content partner. Edited by Mark T. Hoske, content manager, Control Engineering, CFE Media and Technology,


KEYWORDS: Industry 4.0, HMI, IIoT, AR



Are you going to be halfway through Industry 4.0 before competitors begin Industry 5.0?

Author Bio: John Clemons and Justin Robinson, Maverick, Rockwell Automation