Information systems: Adding video to your HMI

Growing demand for a more precise, accurate eye on operations—on the plant floor and in remote locations—is fueling the use of video-enabled displays. And advances in cameras, processing power, storage capacity, and connectivity are creating the perfect storm to put this technology within reach of nearly every plant.

By Jeanine Katzel April 18, 2012

“A picture is worth a thousand words.” Years ago, the adage was common—and true. With the advent of computer-based camera systems and high-speed Internet applications, use of the phrase faded. Yet, as technology progresses and new and exciting options find their way into industrial systems, a slightly reworked version of the saying is emerging. Thanks to rapid advances in processing power, memory capacity, and camera features, the modern industrial plant floor is more and more frequently adding video applications to its operations—and its HMIs—and making a moving picture worth a million words.

Demand for video in manufacturing has increased steadily over the years as technology has improved and costs have dropped. Video for control systems is becoming commonplace, and economical enough to do the inspection tasks capacitive and inductive proximity sensors and photoelectric eyes once did. State-of-the-art software enables extremely high-speed inspections. Now the use of video is expanding further, making its way into the HMI to solve a variety of problems and reap a number of benefits.

According to Cisco’s Chris Haley, vertical solutions architect, manufacturing, video is a game-changing capability for industry that is in the early stages of mass adoption. “Over the past two years, the number of video end-points has jumped exponentially,” he said. “It is testament to how heavily people are investing in this tool. How will it develop? Well, I’d say the industry is inventing that right now as we go.”

A perfect storm of progress

Enhancements in technology and reductions in costs have combined to make video more attractive than ever before and a viable, economical option for most facilities. Haley cited two trends that are driving this growth of video in manufacturing today. “First, we are seeing a shift from task or line workers to more skilled employees,” he said. “The automation and instrumentation in industry has become more complex, creating a demand for workers who understand the manufacturing process and can respond to real-time information. And, today we are a visual culture. Younger workers, in particular, react more effectively to what they see than to words or numerical data. Video, then, provides skilled employees with the visual view they need and expect—one that can also be tied back to a central back-end architecture to become the kind of information that can boost productivity.”

Video typically is embedded into the HMI for two primary reasons:

  • Real-time monitoring and inspection activities on the plant floor and/or in remote areas
  • Historical recordkeeping and playback functions for maintenance procedures and training purposes.

Applications include transferring visual views to and from a multitude of locations and the control room and corporate offices flexibly and cost effectively. Video-enabled HMIs also provide surveillance for plant safety and security, and they act as instructional tools for maintenance and training. Perhaps most importantly, video is critical to remote monitoring. “A live feed video provides a multitude of benefits for process control automation in remote locations,” said Mark Lochhaas, product manager at Advantech. “If an alarm indicates a problem, live video lets you actually see what’s going on.”

Facilities use video for many different reasons, noted Sam Schuy, engineering manager at Maple Systems, “from machine vision for monitoring operations and product quality to safety and security. Video ensures what the manufacturer is producing is coming out as intended. In most cases, we see HMIs equipped with video so that the plant can make a visual record, either for security purposes or to capture abnormal events and alarm responses. The camera maintains an accurate visual record of what happened.”

For an open system that is computer integrated, equipped with a touchscreen, and linked to a network, adding video can be relatively simple today, added Lochhaas. “Most modern video camera systems come with a set of drivers that can be added to the application, and most large MS Windows-based HMI platforms typically include the hardware and software needed to handle video as well as other tasks.”

Changing the way we work

HMIs now do so much more than they did in the past. “HMIs have enough smarts to be able to control interfaces and displays locally and be able to connect with the network in the installation,” said Steve Motter, vice president, business development at IEE Inc. “Five years ago, processing video was a challenge, but PDAs and tablets have made video information sharing so fundamental that now video is built down into the silicon. It enables the latest displays to support these functions without a significant cost or power penalty.”

Connecting a camera to an HMI admittedly can be resource intensive. Only a few years ago, HMIs didn’t have the horsepower to capture video and do other tasks, observed Schuy. “But now processors are more powerful and HMIs have the memory to allow video systems to operate without hampering the performance of the HMI,” he added.

Although costs have unquestionably dropped, video systems are still a sizable investment, with hardware only part of the expense. A significant portion of the investment comes from the time and engineering needed to configure the HMI system to do the required tasks. The technology for video has moved forward on a number of fronts, said Richard Harwell, advanced solutions and connectivity manager at Eaton Corp. “But the adoption, thus far, has been relatively slow. The capability is there, almost standard, on most open platform HMIs, but obstacles remain on the application side.”

Motter agreed that the challenges of integrating video into HMIs are application dependent. “The end-user needs to understand the process and the implementation parameters,” he said. “What will the video monitor? How fast does the process operate and how fast does the information need to be transferred? How much bandwidth is required? What display resolution is needed?”

For Michael Sopko, embedded systems product marketing engineer at National Instruments, cultural issues also impact the acceptance and adoption of video into the HMI. “Workers who watch wide-screen flat panel TVs expect to see the same high-tech approach in the workplace,” he said. “They ask, ‘Why can’t we incorporate this into industry as well?’”

Eaton’s Harwell called these issues generational. “In most cases,” he said, “a younger workforce is more receptive to video technologies. In fact, they look for them. Using video on an HMI will grow more acceptable as the YouTube generation looks for what they know as commonplace. These technologies are changing the way we work as well as the way we live.”

Few good reasons not to do video

There are few good reasons not to add video to your systems, processes, and operations. Video is one of the best ways to present large amounts of information quickly and give an operator the ability to make a decision. Any challenges to the application are offset by the benefits to the automation system and the operator.

Wireless technologies have made adding video to the HMI even more attractive. Video historically has played in the process space, noted Neil Peterson, senior manager, wireless marketing, Emerson Process Management, “but the cost of distributing cabling in these systems has been significant. However, as Internet and Ethernet technologies have become standard and video vendors introduced encoders that allowed video signals to be transmitted wirelessly, the popularity of video has grown. Imaging, encoding, and storage technologies have all come together in the last 10 years or so to produce video systems with incredible functionality. There are many choices now, and ways to meet a multitude of needs.”

The cost of installing video in the HMI system is not excessive, Peterson is quick to add, “especially if the system is wireless. And installing or making use of an existing Wi-Fi infrastructure can be done at a tremendous savings in materials and labor.”

NI’s Sopko takes wireless one step further, envisioning tablet PCs eventually taking on a role as wireless video-enabled HMIs. “They are not durable enough yet, but they will be,” he said, adding that security can be a concern, but not an insurmountable problem. “Tablets are technology with potential.”

Wireless or wired, video capabilities made pervasive through the HMI system have widespread benefits. Companies and plant managers and executives recognize that informed employees are more productive, said Cisco’s Haley. “Video is useful because you can use the same platform you use for production to keep plant personnel informed. It boosts communications and, in turn, morale. Using a common architecture, a plant can integrate live video feeds from the shop floor into HMI screens and make them available to many.”

Affordable, secure, doable

More and more, industry is seeing the value of incorporating video into the HMI system. Facilities need to see operations to react effectively and appropriately. Video-enabled HMI applications are within the reach of most facilities today, said NI’s Sopko. “Embracing video comes down to the requirements—and costs—of the video HMI system versus the discernible benefits.”

Video in the HMI gives manufacturing a more intimate look into plant systems and opens ways to improve everyday operations and maintenance. “Human interaction with these devices is very important,” continued Sopko. “Tools are available to make systems configurable with relative ease and are one primary factor in the growth of video applications. Powerful user interfaces can be developed quickly, and that flexibility is the key to creating what you want and need when you want it. As manufacturers look to increase efficiency and reduce error, multi-touch will play an integral part in the next generation of displays, enabling personnel to interact with their systems by viewing video in real time and reacting before faults occur.”

As video technology finds its place within the HMI, it will require a foundational architecture that can support its demands. “A lot of manufacturers tightened their belts during the recession and did not invest in technologies that enable it to be video-ready,” observed Haley. “That is a barrier today. Having the entire production process using a common set of protocols or architecture not only facilitates video installations, it gives the plant agility. We are experiencing a manufacturing revival in the U.S. today. Companies with the most up-to-date systems, equipment, and architectures are in a better position to compete.”

Emerson’s Peterson termed video of the future “ubiquitous and nomadic,” envisioning a world in which everyone has a camera in hand or on a hard hat. “When assessing advantages, what would happen if the video record wasn’t there, if video monitoring wasn’t in place?” he asked. “Video avoids a lot of problems. Adding a Wi-Fi infrastructure cuts costs and boosts flexibility, as well as enabling other applications. Workers troubleshooting or maintaining equipment will be able to seek help from experts 10 minutes or 10 hours away. Industry will be more efficient and in a position to easily transfer knowledge from the most to the least experienced. A few years ago, these kinds of video systems were cost prohibitive and even unavailable. Today, they are affordable, secure, and doable.”

Katzel is a contributing editor to Control Engineering. Reach her at jkatzel(at)

Online Extra

Looking at the present, and to the future: Applying video-enabled HMIs in the plant

What in the world is going on at my plant? With my process? With the proliferation of video-enabled HMIs, manufacturing facilities now have a way to tell—and to improve, refine, and secure their operations.

For security

HMI systems equipped with video capability are especially useful for monitoring remote locations. Among the most common applications are security situations. Process plants identified by the U.S. Department of Homeland Security as critical to the country’s infrastructure, pointed out Neil Peterson, senior manager, wireless marketing, Emerson Process Management, must be secured against all threats: cyber and physical. In fact, facilities with rail lines running on their property now are required to monitor and secure the periphery in some way. One viable option is to install video cameras equipped with IR motion detectors at strategic points and electronically survey the areas to determine an intrusion, sound an alarm, and record the event.

Physical conveyor systems are also an application for video, added Peterson. “Some systems integrate video with the data historian to track where a process upset or change occurred,” he said. “The systems are synchronized with the video feed so that they can see exactly where and when the problem occurred. It allows the plant to identify disruptions without having to have personnel at the location.”


For the process

Modern, state-of-the-art HMIs hold the potential for highly sophisticated process monitoring. Partnering with Industrial Video and Control, a manufacturer of sophisticated cameras, Emerson Process Management integrated an IVC video system into its DeltaV scalable process automation system to produce a solution with the flexibility to meet many needs. “In one case,” said Peterson, “we used a flame detection camera with IR and video technology to detect a flame, flare, or fire. The flame/flare detector is HART-configured and wired into a DeltaV SIS (safety instrumented system). Video wires go into an encoder and over to the HMI. The arrangement lets the end-user minimize false alarms. In this situation, the plant had been experiencing a lot of false positives that were shutting down the process when no flame or flare was present. Now, if the system detects anything, a video image pops up on the HMI screen in front of the operator. He can determine if a hazard is really present, and make the final confirmation to stop the process. Such a system is improving alarm response.”

Peterson described another installation being considered by a paper mill that needs to keep sharp eyes on its processes. “The company can’t afford to have personnel everywhere,” said Peterson. “The operation is very different from a refinery, where the inside of the pipe is not visible. At a paper mill, a lot of conditions and situations can be monitored, watched, and observed. The solution under development has the potential for an operator to monitor many points within a process, all simultaneously integrated into the HMI. As he looks at the operations, he sees what’s going on. In addition, video evidence of what it happening is being acquired. This system has the potential for monitoring the entire process from end to end,” said Peterson.

For the future

Peterson and Emerson Process Management foresee a future with the potential to extend video and HMI technology right down to the level of a personal device. “What if you could put a camera on a worker’s helmet?” asked Peterson, citing the example of a small camera mounted on a pair of glasses and configured to feed its signal back to a central control room. “With the turnover in the workforce today,” he continued, “a plant can expect to have younger, less experienced workers on the shop floor. Having one pair of older, wiser eyes at a centrally located HMI watching over a number younger ones through a video feed might save accidents, help communications, and improve operations.”

The vision is actually close to reality in the form of an existing product: a high definition camera attached to a hard hat. The front-line communication device is a video solution offered by Emerson Process Management in conjunction with AudiSoft, a French-based company focused on mobile communications. “The device uses the power of voice and video over wireless to show others just what the worker sees,” said Peterson. “It facilitates troubleshooting in the field. An engineer doesn’t have to travel to remote areas to share his expertise. He can help workers troubleshoot many issues from one location.”

In these circumstances, added Peterson, the camera is typically live over a wireless connection, although the capability to record on the device for later upload exists. The system can be set up for point-to-point communication between two people or in a gateway configuration in which numerous people can communicate with multiple workers simultaneously. For those with privacy concerns, the operator wearing the camera controls whether the video and audio are on.

For more information on applying video-enabled HMIs, visit the Emerson Process Management website at More on Industrial Video and Control’s camera systems is available at and more about AudiSoft may be found at See also the websites of the companies listed at the end of the primary article, “Adding Video to Your HMI.”

Not your usual movie experience: 3D systems move into mainstream

Among the more exciting developments accelerating the use of video technology in the HMI is IEE Inc.’s new autostereoscopic 3D display. Basically glasses-free 3D, the LCD display stack-up incorporates unique 3D display technology from 3M. “One of the primary applications that we see for this device is to improve remote virtualization,” said Steve Motter, vice president, business development at IEE Inc. “In a remote operation, a 3D display will give a more accurate representation of what the cameras are seeing, or enable particular information to stand out on the display screen and attract attention.”

The handheld 4.8-in. control display unit was initially developed for terrain mapping, remote robotics control, and enhanced video feeds and is now finding use in military display applications from remote observation to training environments. The display offers a resolution of 800 x 480 x RGB with a brightness of 200 cd/sq m in 2D and 3D modes and has an optimum viewing distance of 16 inches.

Small in size, the device features LED backlight technology to create a low-power, portable unit with higher resolution than other displays using 3D technology. The product decreases off-axis image reversals and color distortions, a common concern when using 3D technologies. It can be switched easily to 2D for imagery comparable to modern smart phones.

Still in the early phases of adoption, 3D products primarily target industrial and aerospace/military applications. Unfortunately, 3D video technology carries a certain stigma as an entertainment industry gimmick, and Motter admitted that a tendency to fail to take 3D as a serious technology could impact its acceptance. In addition, applications that require 3D glasses can be inconvenient or annoying; in some cases, prolonged viewing of 3D can cause fatigue or headaches.

“But early 3D entertainment,” said Motter, “was exaggerated. Modern 3D is more subtle in that it gives depth to the background without over-exaggerating it; it merely moves forward the information the application is focusing on. Those features, and how application engineers take advantage of them, will be keys to its acceptance and adoption.”

The cost penalty of 3D video is not excessive, said Motter, but rather incremental, running around 30% more than conventional video. “In each case,” he added, “the engineer needs to review the application and determine if it warrants the added expense. For some applications, 3D is almost essential.”

For more information on 3D video and autostereoscopic technology, go to the IEE website at, and also visit the 3M website at See also the websites of the companies listed at the end of the primary article, “Adding Video to Your HMI.”

Playback time: Video HMIs embrace a maintenance, training role

By taking advantage of the ability of video-equipped HMIs to store and record information, companies also can use these systems for training and educating workers and for making maintenance procedures available at the site. State-of-the-art HMI processing cores take advantage of modern, high-density memory today to provide storage capacities that range to tens of Gigabytes—fully capable of recording multiple hours of video information. And, as operators become more Internet savvy, they have high expectations that they will be able to simply click and access a video that will give them the help they need at any particular time.

Using the prerecorded video capability of the HMI for instructional purposes requires little more than tapping into the media player built into most HMIs, said Sam Schuy, engineering manager at Maple Systems. “We find many plants make use of the video capability to play back procedures for workers. The simple playback features gives operators and maintenance personnel access to procedures and tutorials right on the plant floor to use when needed.”

Recorded video presentations may be stored locally or accessed from a central server. Local storage has the advantage of making no additional—and therefore unpredictable—demands on network bandwidth. A server-based video training system, on the other hand, can be updated and maintained globally, quickly and more easily. Pre-recorded video files can even be stored on and accessed from YouTube, pointed out Richard Harwell, advanced solutions and connectivity manager at Eaton Corp. “The end-result is no different,” he said. “The important thing to remember is that if your files reside on a server, updating is a lot easier. Without Internet access, each HMI will need to be updated individually, taking time and possibly creating a logistics issue.”

Instructional video is among the more popular applications for video-enabled HMIs. If a facility wants to include information on the operation and repair at the machine, the capability is there, added Harwell, “although many companies find the cost to record and produce packaged video excessively high. For training, they often prefer to use drawings, diagrams, and pdfs because they simply don’t have the time or expertise to generate video clips. It’s just easier to create a manual.”

Nonetheless, the role of playback is expected to increase as video is used in more and more situations and the opportunity to create canned video grows. “Video gives companies the ability to put a high-definition camera on the plant floor, resolve an outage or line stoppage and solve a problem, and then create a permanent video record for use in review and training sessions later,” noted Chris Haley, vertical solutions architect, manufacturing, at CISCO.

“The bottom line is that video-enabled HMIs unquestionably have become more instructional,” observed Alan Yang, product manager at Advantech, “because they make it easier for an operator to review procedures at the work station before operating or attempting to adjust the equipment.”

For more information on these types of applications, visit the websites of the companies listed at the end of the primary article, “Adding Video to Your HMI.”

Video-Enhanced HMI delivers real-time control to the plant floor, boosts efficiency

By Chris Haley, Vertical Solutions Architect, Manufacturing, Cisco

In today’s difficult economy, manufacturers in all industries are under more pressure than ever to boost their efficiency and keep costs under control. Minimizing cost of goods sold (COGS) is imperative in a fiercely competitive marketplace. Every penny that an organization can save is money that could be applied to growth by acquisition, expansion to new markets and locations, and new product development.

Controlling costs in a complex manufacturing environment with multiple, interrelated processes can pose significant challenges. However, every step on the production line also represents an opportunity to streamline processes, enhance employee productivity, and identify new ways to shave costs.

To meet these objectives, manufacturers are seeking to improve their visibility and control of every aspect of the manufacturing process. Information from a centralized enterprise resource planning (ERP) system may not always provide the timeliness or granularity required for close management of manufacturing processes. The need for accurate, up-to-the-minute data is so important that some organizations are building duplicate systems on the factory floor to monitor their processes. All too often, these redundant infrastructures create more complexity, management headaches, and additional overhead.

Improved communication is another key requirement for reducing costs. As manufacturers discover new ways to boost efficiency, they need a cost-effective way to communicate new procedures and company goals to their employees—quickly and consistently.

Complete visibility with end-to-end video

To provide the rich communication capabilities and deeper visibility they require, many manufacturers are turning to network-based video solutions. Flexible and scalable, these standards-based solutions can be integrated smoothly with human machine interface (HMI) applications to dramatically improve communication throughout an organization—from the factory floor to the back-end. Best of all, most organizations have already deployed extensive IP enterprise networks, and perhaps even video endpoints. With this “video-ready” investment already in place, they can take advantage of new solutions without replacing their existing infrastructure.

At the manufacturing floor, rich video applications can deliver more timely information to employees in a more sophisticated way. Instead of a days-old printout on a bulletin board, factory floor employees and managers can enjoy in-depth, real-time visibility into work in process, supply chain details, equipment anomalies, and more.

Since video HMI solutions are based on an existing IP network, they can also support information gathering, integrating smoothly with manufacturing services platforms to share live data, alarms, diagnostic status, and report and auditing information.

Video HMI solutions also provide a powerful way to share information to groups of individuals through digital signage. For example, automotive manufacturers are increasingly adopting digital signage and integrating it with their management platforms to display real-time productivity statistics. Any employee can glance up at the screen and see a live “scoreboard” measuring actual productivity compared to manufacturing goals. Or a floor manager could check the screen and quickly determine whether key performance indicators (KPIs) are being met.

Keeping employees knowledgeable is key to boosting the efficiency of manufacturing processes and controlling costs. Studies have shown that there is a direct relationship between well-informed employees and their productivity. By delivering accurate, timely information about the production cycle to workers on the floor, organizations can help improve productivity and enhance their overall business agility.

Building a strategic foundation

Video HMI applications can clearly deliver an immediate, positive impact on manufacturing processes. And over the longer term, networked video technology can provide an ideal solution for supporting strategic initiatives, improving safety, and enhancing employee satisfaction.

For example, a video screen on the manufacturing floor might display not only current production metrics, but also provide the ability to deliver pre-recorded training, distribute the latest safety practices instantly, or present corporate messaging from executives. For example, in a challenging work environment such as mining, an organization could stream live video and safety announcements directly to job sites.

Video provides a consistent, easy way to disseminate information directly to the task worker, who may not have a phone or PC, but can still benefit from corporate strategy information. By improving the reach and impact of company communication, organizations can build a stronger corporate culture, foster an atmosphere of involvement, and help boost employee morale.

The end-to-end network used to support video can also play an integral role in helping organizations utilize information to support strategic decisions. Today’s companies accumulate a wealth of manufacturing data through process automation systems and other networked platforms. As manufacturing platforms become more sophisticated, organizations will increasingly have the ability to collect and synthesize this data, extract meaningful information from it, and apply it toward more informed, predictive decisions.

By applying the power of the network, manufacturers can take immediate advantage of improved insight and communications to unlock improved efficiency and cost control. And they can build a long-term foundation for better-informed employees, safer work environments, and sustainable success.

For more about network-based video-enhanced HMI systems, visit the Cisco website at