Information systems: The evolution of the HMI
From push buttons to PCs, from text-based to graphical, from CRT to LCD, the shape and role of what has come to be called “visualization” has changed dramatically over a few short years. The move from islands of automation to sophisticated control and analytics reflects unprecedented growth, and the journey is far from over.
In the beginning was the push button, and with the push button were the lights and the switches. And then hardwired devices appeared and became electronic panels, and the integrated circuit emerged. Personal computers and software programs were born and overran the display playing field, and the Internet paved the way for information to go anywhere and everywhere. Proprietary systems became "open," and the operator display that once watched one machine embraced many, within a single plant and around the world.
Evolution is real—at least where the human-machine interface (HMI) is concerned. What began as a simple, dedicated function has grown and changed immeasurably in just a few decades to become quite possibly the biggest influence in the manufacturing environment today. Thanks to an explosion of bandwidth and accessibility, HMI systems interconnect millions of pieces of data, observed John Krajewski, director of product management for HMI supervisory, Invensys. "In the past, HMIs were isolated doing basic machine control. Now collaborative systems have unleashed a new power to connect and access."
Industry historically has sought to monitor and understand its processes and operations. Its journey from on/off devices to complex, sophisticated systems for monitoring and analysis has been a dramatic and remarkable ride. What follows is a look at the passage from push button to MMI (man-machine interface) to HMI to visualization, how it has shaped the industrial process, and how it continues to impact the manufacturing world.
Back in the day...
It started with push buttons and lights, dial and panel gages, and first-out annunciators, recalled Louis Szabo, business development manager, Pepperl+Fuchs. "Even as late as 1990, many plants saw push buttons as a more economical option than an HMI panel."
But once manufacturers had the option of replacing those buttons with an electronic terminal for about $100 per installed push button, added Marco H. Wishart, business manager, electronic operator interface at Rockwell Automation, "the push buttons became extremely expensive and the terminals so much more efficient."
We have truly seen an evolution, said Emmanuel Ecochard, North America general manager, PcVue. "Not paradigm shifts maybe, but continual incremental improvements. Systems 30 years ago were hardwired and not very flexible. Now products let you build almost any application fast, and maintain it easily."
Recollections continued: In the early years, the CRT (cathode ray tube) with monochromatic screens and rudimentary text-based displays dominated, added Nathan Eisel, technical support manager, Beckhoff Automation. "Interaction was through mechanical buttons, a keyboard, or a mouse."
Operator terminals started out as add-on components to the machine, offered Marcia Gadbois, vice president, business development, InduSoft. "They augmented the operation and provided control, but the commands in those early devices were literal, the systems weren't user friendly. It was a text-based environment that had to be programmed to get any kind of visualization."
Then the PC was born and turned the HMI world upside down. The focus shifted from hardware to software, a device became a system, and the job of the HMI began to grow, and grow exponentially. “HMI systems ran on proprietary equipment at first," said Szabo. "Then interoperability became the buzzword. But the big game-changer came late in the 1990s when companies like Wonderware and Intellution introduced control software for the PC," he went on. "The late 1990s also saw the introduction of new hardware (LCDs instead of CRTs, intrinsically safe touchscreens), but when manufacturers talk about HMIs now, they talk about software. That's where and when changes really began."
The 1990s also saw the birth and proliferation of the open system. More recently, the thin client appeared, first in business systems, then in the process as manufacturing began to use the technology with PLCs and DCS and the control room began migrating onto the plant floor. "Historically, the HMI device had looked down on a machine and displayed information about it," said Scott A. Miller, business manager, visualization software, Rockwell Automation, "but soon it was looking across and up, gathering and using information to help do a better job and to do things that had never been done before."
Building bridges, breaking down walls
Obviously many developments have influenced the evolution of the HMI: processor capability, Ethernet, and the Internet. Which have had the greatest impact, and what have the changes they brought meant for manufacturing? For one, they have allowed the HMI to become intimately involved in plant operations. State-of-the-art HMI systems access data from multiple sources and reach out to remote spaces. Operators make more informed decisions.
On the hardware side, the transition from hardwired devices to the electronic terminal was instrumental in setting the stage for advanced capabilities. "The changes were incremental but essential," said Rockwell Automation's Wishart. "They took us from CRTs to LCDs to LED backlit screens and aspect ratio changes. The rapid advancement of processors also has allowed HMI devices to do so much more."
The biggest differentiator in the mind of P+F's Szabo was a toss-up between Microsoft Windows and Ethernet. "Both technologies have helped give HMI systems the flexibility to go almost anywhere and do almost anything. And thanks to connectivity, information is available in real time."
InduSoft's Gadbois cited connectivity and open systems as elements that empower the HMI and its operators, but also pointed to the refinement of the user interface as noteworthy. "Because of connectivity, walls have been broken down. We stopped being islands of automation, moved to a distributed environment, and involved more people in the decision-making process," she said. "Using the HMI to make business decisions based on large amounts of available data and deliver cost-effective solutions to the operation is changing the way we work. Information used to be specific to a machine. Now operators have a real impact on the bottom line because the HMI system shows them a complete picture, not just one machine."
Together, these changes have had one major impact: they have taken the command center outside the walls of the plant. "More and more, end users are asking for remote access capabilities," said PcVue's Ecochard. "Mobility and remote access are what are so important now, but these capabilities also raise concerns. The more mobile you are, the more vulnerable you are. As mobile applications develop, security must be strengthened."
Rockwell Automation's Scott Miller, and colleague Mike Burrows, director, market development, integrated architecture at Rockwell Automation, are quick to concur. "Security is a huge issue, as is bandwidth," said Burrows. "As industry moves to make a variety of technologies such as video and wireless work together, layers of defense are a necessity."
Mobile devices are augmenting existing capabilities, added Miller. "Many facilities are pushing data to smartphones, and that will challenge security. An operator using a mobile terminal to move around the shop floor makes sense, but a manager at the mall on his smartphone needs to be certain he doesn't mistakenly stop a production line by hitting a wrong button."
Just what, then, is an HMI?
Has a smartphone, then, become an HMI? In some respects, it is, according to Gadbois. “We are morphing the term and calling the HMI function 'visualization.' Every interface is more than an HMI. It is visualization. The term ‘HMI’ may disappear; maybe visualization will replace it, but the purpose and the function remains."
The terminology is changing. Over the past 15 years, the convergence of technologies has led to a collection of functions and capabilities now called “visualization” by many. "We almost never use the term ‘HMI’ anymore," said Wishart. “It is visualization: software and hardware running on multiple devices. The same systems, data, and integration are involved whether it's on a dedicated panel, a PC server, a smartphone, or other device. At the core it's still an HMI, although as it evolves it will take different forms."
Rockwell Automation's Miller calls visualization the blending point. However, how that blending happens will be different for each industry, application, and client, he added. "What will proliferate is the way capabilities are applied. Cutting-edge visualization systems will access information and deliver the right content to the right person. Whether an operator is in front of a bottling machine all day, every day, moving around the plant, or moving from plant to plant, the information he uses comes from the control system and the information system. The more we blend those systems, the better we can serve multiple applications. That's the way HMI technology is moving."
The definition of an HMI has definitely broadened, added Invensys' Krajewski. "The technology may evolve to where the term ‘HMI’ doesn't even apply,” he added. "Applications are integrated. Rarely is the HMI the sole mechanism an operator uses. An HMI may have interacted with one piece of equipment years ago, but now larger systems are under its control. Although operators will still have jobs to do, operations management will be their next frontier. Managing an entire operation from a single console will develop in the future."
Unprecedented past, unfettered future
HMIs have a rich and varied past, but industry anticipates even more growth. "We've come a long way," said InduSoft's Gadbois. "We thought the changes of the past were magical, but look at what's happening now. The evolution is almost scary."
Industry today uses a wide range of brilliant, full-color TFT LCD displays, noted Beckhoff's Eisel. "Touchscreen technology is commonplace. Industrially hardened control panels and panel PCs made of aluminum or steel are paired with rich HMI software graphics to create dynamic operator interfaces. Wide screens are commonplace, and full HD resolution is available from many HMI vendors."
According to Rockwell Automation's Burrows, the next evolutionary stage will find the operator interface becoming inseparable from the controller. "They are still separate engines, but moving forward, the controller and the operator interface will become one system using one database. The controller is the starting point for all data reaching the HMI. It is becoming the plant server. Visualization is more powerful when there is no computer in the middle, when it is embedded in the control environment."
Arguably one of the most significant changes affecting current HMI systems is the rise of multitouch technology. "Systems with integrated multitouch capabilities are already available," said Beckhoff's Eisel. "Displays use a projective capacitive touchscreen (PCT) to ensure industrial compatibility. Its high touchpoint density enables accurate and reliable operation for industrial users. Operation with work gloves is possible. In addition, familiar functions of smartphones and touchpads, such as zooming, scrolling, object turning, and flicks, can be transitioned to industrial applications as well."
Rockwell Automation's Burrows sees devices such Apple's iPad as "the big gorilla in the room. It sets the vision for what is possible: wireless, mobility, cloud computing, data integration," he said."I'm not suggesting the iPad be used in industry, but it does set the vision for what is possible."
The next frontier in the eyes of P+F’s Szabo is the rapid expansion of “virtualization,” a client-server HMI architecture. "The business model of a high availability server with IT support, and highly distributed thin clients deployed throughout the facility, is now the automation model right on the plant floor," said Szabo. “The benefits include hardware refresh optimization, lower risk upgrades and expansion, increased security, and lower lifecycle costs.”
Perhaps the greatest change, however, will come not from the device, but from the people. "The manager doing a weekly or monthly review has his finger on the pulse of the profitability of the organization in one way," said Invensys' Krajewski, "but it is the operator who sees things in real time on his HMI who has the better view. The modern HMI can collapse the decision-making process, taking it down to the level of the worker, the one with the greatest influence.”
Sidebar: A cultural challenge: HMIs in a changing workforce
Powerful, evolving HMI systems of the 21st century are bringing new capabilities to the plant floor and beyond. Conversely, the workforce of the 21st century is placing new demands on the HMI. And with good reason. As more data are gathered and processed in real time, the worker that interfaces with those data must make more and quicker decisions, said Mike Burrows, director, market development, integrated architecture, Rockwell Automation. "To do that adequately, that worker needs to understand those decisions. That is an educational challenge. Having more real-time information available is changing the way we work as well as increasing opportunities for the HMI."
The current manufacturing environment with its sophisticated HMIs requires much more of its workforce. Thankfully, however, modern employees are increasingly up to the cultural challenge and well-equipped to respond. Men and women entering manufacturing today have a different skill set than did workers of the past. Not only are they ready to embrace the new technologies that have evolved, they cannot imagine a workplace without them. John Krajewski, director of product management for HMI supervisory, Invensys, calls them digital natives. "They have certain expectations about the equipment they use," he said.
Up until now, HMI evolution was driven primarily by technological development, added Emmanuel Ecochard, North America general manager, PcVue."Today, it is driven more by the demands of the end user. Today's end user is familiar with technology and wants it in the workplace. In fact, today's worker wants to have the same experience in the plant as in daily life."
Similarly, manufacturers accept technology more readily than they used to, agreed Scott A. Miller, business manager, visualization software at Rockwell Automation, due largely to their consumer device experiences. "They are more comfortable with technological changes and with the rapid rate of change," he said, affirming that the demographics of the workforce are impacting--and will continue to impact--the evolution of the HMI. "A younger workforce that embraces new technology is stepping in and expecting it. User experience will be a key requirement going forward."
Marcia Gadbois, vice president, business development, InduSoft echoed the sentiment."My children think a mouse is antiquated," said Gadbois. "When they grow up, they will expect to have a rich, visual 3D environment in daily life, and that will include the workplace. The younger workforce expects to find iPads, tablets, and slates in the workplace. They look at older computers like they are dinosaurs. Today's college graduate does not remember a time without Microsoft Windows-based systems."
Today's up-and-coming workforce is steeped in consumer electronics, which, like it or not, is a major driver of industrial HMI change today. According to Nathan Eisel, technical support manager, Beckhoff Automation, "The devices that engineers interact with at home ultimately influence their expectations in HMI technology. It’s a pretty safe bet that any major interactive technologies that come along in the future will eventually make their way into HMIs for machines, manufacturing, and processing, whether it be in hardware or software."
As HMIs grow and expand in sophistication and complexity, management and workers alike will need to know how use them, in whatever form they come. If today's device-oriented culture is any indication, they will be up to the challenge.
Sidebar: The Evolution of the HMI--A Brief History
Over roughly the past 30 years, the mechanisms used by industry to communicate with its machines and devices have changed dramatically, driven by rapidly advancing technology and the needs of manufacturing facilities and the people who run them. The brief history below outlines some of the major events and developments that have influenced that evolution.
1980s--Pushbutton, lights, and dial, gauges, first-out annunciators, and tubes predominate.
This time-period was characterized by monitoring processes by drawing PFDs (process flow diagrams) on a large wall and adding gauges and indicators to represent tank level, pressure, flow, temperature, and other parameters.
1980s to mid-1990s--Proprietary MMI (man-machine interface) from DCS (distributed control system) companies used for control room use only.
PLC manufacturers used proprietary OITs (operator interface terminals) for on-machine control.
Mid-1990s–PCs enter the playing field.
PCs were introduced to manufacturing, but required real-time OS (operating systems) such as QNX. Configuration tools migrate from proprietary hardware to proprietary hardware on PC platforms. Microsoft introduces Windows OS; Windows NT enters the field.
Platforms migrate to the plant floor driven by several factors; MMI hardware becomes competitive with traditional MMI hardware.
DCS consoles migrate to Microsoft Windows; Wonderware, Intellution, and similar companies introduce game-changing, platform independent system integration software. The focus of HMIs moves from hardware to software.
Mid- to late-1990s.
Ethernet enters the market and opens the door to boundary-free connectivity.
Turn of the 21st century.
Displays suitable for hazardous areas debut;
DCS focus on HMI software and control PC specifications;
Change from passive matrix to active matrix eliminates ghosting;
Multiple touchscreen technologies develop, driven by applications, environmental conditions;
Migration from KVM (keyboard, video, mouse) extenders to thin client architecture;
LED-based LCD panels extend backlight life, increase brightness for daylight visibility;
Widescreen systems introduced, driven by consumer television technology.
Multitouch, video technologies enter the market;
Tablets, smart phones, other mobile devices assume HMI roles driven by consumer market, demands of a technology-proficient workforce.
The term "visualization" is introduced to describe the broadening and growing functions of the HMI.
The term "virtualization" is introduced to describe a client-server architecture for automation as a method of reducing the refresh rate of hardware and facilitating plant expansion.
Information for this timeline was provided by Louis Szabo, business development manager, Pepperl+Fuchs. Visit the Pepperl+Fuchs website at www.pepperl-fuchs.us.
Sidebar: Remote control, HMI/SCADA systems help renewable energy company monitor wind farm turbines
Wind energy is among the fastest-growing sources of energy in the world and a tremendous source of domestic power for the United States. Iberdrola Renewables LLC, the U.S. division of Spain's Iberdrola S.A., is one of the largest providers of wind energy in the world, and the second-largest in North America. It has been in the global energy business since 2000.
From its National Control Center in Portland, OR, the company helps utilities green up their energy portfolios. The center, staffed 24/7, helps customers manage the risks and uncertainty of the natural gas and power generation industries while fulfilling energy requirements with clean, sustainable power. Opened in 2001, it maintains, develops, builds, and operates more than 3,500 MW of wind power and other energy facilities in 20 states. Its goal is to add roughly 1,000 MW of new renewable energy each year, an amount that exceeds the capacity of any other renewable energy supplier in the country.
The center resides in a room that looks like NASA’s Mission Control. Systems analysts oversee turbines at wind farms throughout the country, monitoring the performance and efficiency of each one. Within the purview of the center is keeping an eye on approaching storms to warn technicians in the field to get to safety before harsh weather hits. Analysts there also help scientists conduct groundbreaking wildlife research at the wind farms while others help the nation’s various transmission system operators ensure grid reliability that will keep the lights on under any circumstances.
Key to the effective operation of the control center is PcVue, a powerful HMI/SCADA system developed by ARC Informatique. The suite of software and hardware products is designed for displaying, controlling, managing, and analyzing information from multiple installations. Among the functions of this ‘nerve center’ is the connection of all components at the wind farms to the central control room and their remote operation and control. Its responsibilities include the wind tower turbines, substations, meteorological stations, and bird/bat avian radar and surveillance systems for preserving wildlife.
The system allows operators to supervise the behavior of all turbines in all wind farms. By recording turbine activity at predetermined intervals, the system determines what adjustments or corrective actions need to be taken. It also records energy output, availability, and error signals. In addition, it has the capability to implement compliance requirements and control other parameters such as power factor, voltage, and reactive power production, and manage the wind farms’ contributions to network voltage and frequency control. Further, it lets operators manage power output based on real-time grid requirements.
Managing the turbines...and more
In the past, only a small number of wind turbines were transmitting energy into the grid and entry into the industry was fairly easy. Recently, however, congestion has become a issue with wind energy suppliers balancing energy production with available transmission inputs. Strict requirements have led Iberdrola to design an integrated system using curtailment via setpoints to manage its generation profile on a real-time basis.
The wind farms use various types and makes of turbines, each of which is equipped with its own supplier-installed control/HMI system. A major advantage of the PcVue HMI/SCADA control system is that it is turbine-supplier neutral. It is not tied to any one PLC vendor and thus is able to provide data reporting and analysis formats independent of turbine type. The robust, high performance HMI/SCADA system is one of the few available that is separate from the PLC, a feature that was particularly important to Iberdrola Renewables LLC because of the myriad turbines and PLCs being used.
Each wind turbine is fitted with a control box installed at the top of the tower. It contains a PLC, power converter, control boards, and I/O device. Sensors detect wind speed, wind direction, shaft rotation speed, and numerous other parameters. Data are collected and transmitted to the PLC, which then signals the control system to adjust a motorized yaw gear to direct the turbine for maximum power generation. The control box is connected to the tower base over a fiberoptic, redundant ring Ethernet-based LAN, which, in turn, connects to the remote control system that collects and manages all data; adjusts turbine settings; and enables intelligent alarm, troubleshooting, and reporting.
OPC is the primary communications protocol used by the system to transmit data from the various PLCs. PcVue’s OPC data access client and the OPC DA XML client exchange real-time data with communication servers; an OPC DA server facilitates data exchange with third-party applications. All data are routed to the control center.
Operators configure the behavior of alarms by using groups, filters, sorting, acknowledgement and masking. They also create alarm counters and associate specific actions with any alarm. Alarms can be acknowledged by operators directly from mimics and those actions can be broadcast automatically to all nodes on the network.
The system provides a single user view that allows an easy visual display and overall management of the PLC, HMI, and control systems on the turbines from the myriad systems in place, said Harm Toren, managing director, head of operations services, wind operations for Iberdrola Renewables in Portland, OR. “Because we monitor avian migration and weather in addition to controlling and managing our turbines, we needed a system that would provide a simple, easy to read GUI [graphical user interface] so that we can react at a moment’s notice.”
Keeping everything under control
The fiberoptic network and the controls that manage the company's growing business are state of the art. Through the Portland control center, which is similar to the company's Toledo, Spain facility, the company is able to access remotely the facilities throughout the country and all alarm and event conditions. Management of the multistation configurations uses the HMI/SCADA system's advanced tools to ensure consistent configuration data and deployment at all the stations. The system's centralized configuration provides the capabilities for managing and tracing application versions and changes. It also supports automatic updating. Any time a station on the network is started up, the control system automatically runs consistency checks of the application versions in use.
The control center is able to provide energy management services to any owner of a wind energy facility. U.S. installations currently produce 3,600 MW of wind power across 35 independent power plants. The Iberdrola control system maintains 2,479 wind turbines, each with 300 to 350 I/O data points, or approximately 700,000 to 850,000 I/O data points on more than 20 servers. To handle the diverse demands of the farms, the system alarms are highly configurable. Messages may be printed, viewed in alarm lists, and archived. The SCADA software integrates and connects with the wind turbines acting as a light client to the PcVue application and managing up to 2.5 million data elements. The configuration provides the operators with all required information about the turbine signals.
The distributed client-server architecture with built-in redundancy features is a fault-tolerant design that ensures continuity of data collection even in the event of a component failure. PcVue also supports dual networks for communication with field equipment and among PcVue stations. Each component and each station in the configuration has a validity status to enable operators to view conditions in real time. The client stations communicate via OPC with redundant front ends connected to the 1,000 Mbps TCP/IP Ethernet network. Each front end can handle up to 60,000 I/Os.
Supervision is done on two levels to handle the large volume of information (around 350 signals per turbine) and to facilitate operation and maintenance of the facilities. The first level provides an overview of the most relevant alarms, values, and counters to supervise the turbines in a normal situation and to detect failures that need to be corrected. The second, more detailed level is triggered on request to display selected data so that the operators can immediately and precisely diagnose any failures that have occurred and determine remedial operations. The data received can be processed as set points, historical records, alarm management, trending, and the like.
A model for future systems
Iberdrola liked the system for several reasons. "We chose the PcVue software system," said Toren, "because it had already been proven user-friendly and highly functional in Iberdrola’s Spanish operations. It proved reliable, scalable, and easy to configure."
It also minimized the risk of overlaying crucial information and helped simplify the HMI/SCADA view. The ability to create templates for contents and behavior ensured consistency of all animations in mimics. Iberdrola uses multi-level access rights and menus associated with each user to ensure that navigation within the application is tailored to the needs and permissions of each individual. This ensures better security, traceability, and control of user actions.
The system is working very well and has met all Iberdrola Renewables expectations. The company plans to use the operation as the standard model for future facilities and expects it will becomes its “out of the box” solution. The company is working towards a more scalable system to suit the next generation of renewable energy markets.“We are installing wind turbines to operate in harmony with other energy sources such as nuclear power, solar, and hydropower in a netting arrangement to optimize performance," said Toren. "We are on the cutting edge.”
Information about this application was provided by PcVue. For more about the system and the company, visit the website at www.pcvuesolutions.com. For more on Iberdrola Renewables, visit the company website at iberdrolarenewables.us.
Sidebar: Evolving HMIs: From single machine interface to enterprise visualization
In the human machine interface of the future, according to the vision of Invensys, the HMI is the intelligent center of a comprehensive enterprise visualization platform with core services built in. This broadens the scope of available information to everyone in the company--from operators to CEOs, expanding the methods in which information is conveyed, making it highly relevant and specific to each user.
The HMI at the bottom center below is the main point of access to all applications shown in the surrounding monitors. From a single vantage point, the operator can access cameras pointing to equipment and areas, trend charts, KPIs, training and user guide documentation, and real-time personnel collaborative video.
When an event occurs, such as the alarm signified below, the operator would simply click on an alarm icon and each screen would adjust instantly to display information relevant to the issue at hand.
Such timely access to information would extend and promote worker interaction and collaboration significantly, shortening time-to-decision and thus enabling a more productive, profitable, and agile company.
Jeanine Katzel is a contributing editor to Control Engineering. Reach her at email@example.com.
- If you haven’t checked HMI technology in a while, it’s changed, a lot.
- An HMI isn’t just an interface, it is a means to manage an entire operation.
- There are more ways to dive deep into your process, collaborate with others, and access critical information.
For additional information about visualization and HMI products and systems, visit:
- Events & Awards
- Magazine Archives
- Digital Reports
- Global SI Database
- Oil & Gas Engineering
- Survey Prize Winners
- CFE Edu