Tracking HMI advances

In addition to holding on to past advances, HMIs continue to evolve. Advancements include higher resolution, wider screens, multi-touch, integrated programming, cloud connectivity, higher efficiency, greater durability, and improved connectivity.

By Clark Kromenaker, Omron Automation and Safety November 29, 2015

The typical application for a human-machine interface (HMI) is anywhere a human needs to interact with a machine or process. We humans are either pushing information into a machine or process and/or pulling or reading the information back. The genesis of the HMI began when it became clear that it was less expensive to program an HMI-type display than to have an electrician wire individual push-buttons and display devices on a machine panel for every machine produced. Early HMI devices were thought of as just that: push-button replacers. They had an object toolbox to match that thought. At that time, basic push-buttons and indicator lights were the core of the configuring object toolkit.

In the world of control systems, the HMI has progressed with advances in technology and with the rest of the control-system world. The technological trend is to adopt hardware designs that follow the development of commercial computer technologies, such as PCs, laptops, and mobile devices.

There was a time when the processors, memory, and peripheral hardware used in HMI devices were fairly esoteric. Because commercial-grade hardware was relatively fragile at the time, many manufacturers selected military-grade components for the HMI and the rest of the control system. This tended to keep prices high and limit supplies. With the advent of the PC and mobile devices, the durability of hardware components has increased significantly. Not only has the durability of the components improved, but because PCs, cell phones, and tablets have driven volume manufacturing of compact durable components, prices have decreased. Here’s an interesting technology reversal: touchscreens were in use in industrial HMIs far sooner than they appeared in mobile devices (see Figure 1).

On the HMI software and firmware development front, the same tools used to develop the firmware and software that drives the hardware in the HMI has gone through similar changes. Instead of assembly code and X86-type processors, designers can use the latest reduced instruction set computing (RISC) architecture and devices like ARM or ATOM processors, akin to what is used in current mobile devices. The operating systems have evolved as well. Early HMI devices had proprietary operating systems with specific tools and limited familiarity, reusability, and function growth that come with closed systems. Today, the trend is clearly to standardize more open systems, such as Microsoft Windows Embedded 7 (WEC7), where common reusable tools can be used over generations of hardware and where design talent is much more available.

The types of microprocessors used in HMIs have often followed the lead of the PC market. Processors used in the PC market are more easily available at reasonable prices. The same is true of memory and peripheral hardware.

It wasn’t until later that more advanced tools were added. Some of the more advanced tools are graphing objects, trending objects, recipe tools, all the way to more advanced standalone programming languages, such as C++ and VB.Net. The current trend is to automate the HMI screen design process. That process is aided by templates set up with the typical tools and objects for applications that include logic, motion, vision, safety, robotics, and networks. HMIs are becoming so powerful that we get to where we are today: the possibilities of offloading some PLC functionality into the HMI to allow the PLC to do other things. 

What’s hot today?

In the absence of a paradigm-shifting development, what is hot today in HMI technology? Remote-screen viewing of an HMI on mobile devices has caught on with users as well as manufacturers. This function is seen as a way for supervisory personnel to check on what the operator is viewing as well as to collect quick information, such as machine status and production quantities, at any time. For example, a supervisor in a meeting can view the operator’s screen on his mobile device to confirm production status or how a particular process is progressing.

The ability to display media in popular formats on the HMI during runtime is a becoming a must-have capability. PDF, Xcel, Word, and video format viewers allow the HMI program to display information that can train an operator, help an operator run the machine, or solve a machine issue. For example, a machine jam can log an alarm and then display a video on how to clear the jam and how to get the machine going again.

Being able to share a design or parts of a design is becoming important with the impetus being to standardize the way the same or similar machines, or machine processes, operate within a manufacturing entity. Screens or groups of control objects can be packaged into a file that can be shared with other HMI designers such that standard ways of interfacing with a machine or process can be established that can lead to standardized operator training-even at a global level.

Wide-screen, high-resolution displays are finding their way into many new HMI models. Wide screens have the advantages of allowing more control objects to be placed on the screen and of reducing the number of screen changes, where in a particular process or operation, changing screens would be cumbersome (see Figure 2). However, there can also be the opposite problem when too many control objects are placed on one screen without much forethought. Some may argue that high-resolution displays are not needed in a manufacturing environment. However, for machine builders in a competitive market, a high-resolution HMI control panel can impart higher value to a machine. Either way, with the consumer market driving the volume of displays in the direction of high-resolution, wide screens, these may someday become the most economical and perhaps the only option.

Arguably, the majority of the current crop of programming software for HMI devices is not really programming software at all, but more of a configuration package with functionality limited to features the HMI supplier offers in the configuration package. Many HMI providers are now offering the ability to more deeply control the HMI to create custom controls and functionality with more advanced language support. VB.Net and C++ are options in some models. Using these languages allows HMI designers a more direct tie to the underlying hardware of the HMI for faster operating and sometimes more efficient, custom functionality.

Integrated development environments for the HMI, controller, and control peripheral devices are making more sense to design engineers. Separate development/configuration environments where the tags and/or other configuration data must be imported or exported from a variety of configuration or programming packages are being frowned upon. Environments with shared-tag databases and integrated configuration/programming environments are becoming more preferred. For automation suppliers who can provide the entire automation solution, the question is becoming, "Why are there so many programming packages to maintain to use your full solution?" This question becomes tough to answer for automation suppliers who partner with peripheral device vendors to present a complete automation solution as they may need to integrate their own proprietary environment into a completely different one.

Because most of us use mobile devices, the familiarity we have with those devices, the ability to enlarge or shrink an image, and the ability to switch screens with the swoosh of a finger is starting to find its way into HMI devices.

What’s ahead?

Predicting the future is a touchy topic, but here it goes. One direction may be the unbundling of features. This market, like many others, is driven by the wants and needs of the end user. Ultimately, the trend of users is to only want and pay for features that they will use. Possibly, at some point, a model could evolve where tools and features are selected for an application, and end users are charged only for what they use.

The cloud is introducing entirely new concepts into the world of industrial control. At some point, we may see some parts of the HMI and control system residing in a private or public cloud. Models may evolve where the HMI or other control hardware itself is either purchased or perhaps leased and based on the functionality needed. Those features would be downloaded to the hardware from the cloud based on unique Internet of Things (IoT) identifier (MAC ID or IP address) or serial number associated with what has been purchased or leased for the device.

The cloud may also be a way to store information that is not directly needed for machine control. Information, such as collected data; required compliance data; operator login, logout, and activity information; security records; product traceability information; predictive maintenance information; recipe information; and alarm histories could be stored in the cloud as a service. The only data that really needs to stay local to the machine would be control-centric data. The cloud could be used to minimize the information technology (IT) work a controls engineer has to deal with. Today, controls engineers straddle the controls world and IT world with responsibility for rack-mounted servers and switches in cabinets or closets close to the manufacturing area. If it is important to a manufacturing organization to keep manufacturing communications and enterprise data separate, the cloud could assist in this as well.

The cloud might also be a way to ensure that remote locations for a global manufacturing entity that should be running the same control and HMI programs are doing just that. Systems can be designed to periodically refresh their programming or do control program validity testing from a cloud-based master.

Other interesting developments are the continuing miniaturization and improved power efficiency of electronics. We may see the HMI mounted on the surface of the enclosure for the machine with only small holes in the panel for power and communications, instead of having to cut a rectangular opening in a panel to mount the HMI device. If power-over-Ethernet continues to grow, there would only need to be one electrical connection to the HMI instead of the current two connections of power and communications. Mounting the HMI on the outside surface of the enclosure would reduce heat buildup in the enclosure, make the HMI easier to install, and would also make it easier to meet hazardous location requirements because cutouts into the enclosure would be minimized. The next step beyond this may be an on-machine HMI, where the HMI no longer has an enclosure, is mounted directly to the machine wherever needed, and connects to an on-machine communications block nearby or perhaps operates wirelessly (see Figure 3). Further out, with flexible OLED displays on the way and flexible circuit boards already here, we may see HMI displays that are flexible to some degree and can mount on uneven surfaces or displays with improved durability when used in vibration-prone environments.

Concepts are being developed that use eye movements in conjunction with a display to indicate an object or function that needs to be modified. Hand gestures in an active field in front of the display could be used to make changes.

One thing we can be sure of, the ways we interface with machines will continually evolve.

Clark Kromenaker is the product marketing manager for HMI, IPC, and RFID at Omron Automation and Safety. He has more than 15 years of experience in engineering, applications, and marketing for industrial controls and high-technology products.

This article appears in the Applied Automation supplement for Control Engineering 
and Plant Engineering

– See other articles from the supplement below.