Using a historian to store machine data

Cover story: HMI-historian integration cover story: Integration of historians with human-machine interface software allows knowledge workers to change the future based on past collection of big data. Here’s the why and how of linking your machine HMI to a historian, and the best ways to view and analyze historical data. See 8 benefits of historical machine data and 7 ways to view and analyze machine data.


Figure 1: Trending at the human-machine interface (HMI) allows viewing and comparison of real-time and historical data for troubleshooting and analysis purposes. If an HMI/SCADA gateway is PC-based, the historian can sometimes reside on the same PC, as wiHistorians provide a great way to store and access huge amounts of machine data, but this time-based information derived from "big data" collection must often be transformed for use by machine operators, plant engineers, and management in factories and plants. More big data value can be gained by making it more accessible for viewing and analysis, often by connecting an HMI to a historian, which is then connected to other software platforms. This allows viewing and analysis of data at the HMI, the historian, and other software platforms.

Historians can take many forms, from simple spreadsheets or personal database files, to full-featured database products available from a variety of manufacturers and running on many platforms, to dedicated data historian products capable of capturing and storing up to tens of millions of data points per second. A variety of methods are available to make information stored in historians readily actionable, such as trending, reporting, and pushing the data to external data analysis platforms.

With most machines, one or more controllers are connected to a gateway for HMI/supervisory control and data acquisition (SCADA) software; the gateway acts as a data concentrator. The HMI/SCADA gateway is in turn connected to a historian. If the HMI/SCADA gateway is PC-based, the historian may be able to reside on the same PC.

If the HMI/SCADA gateway is running on an embedded platform, the historian will probably be an external PC-based system connected to the embedded platform via Ethernet. In some cases where machine connectivity is intermittent, it may be necessary to use an embedded database product to store historical data temporarily. The data can then be replicated later to a larger database or historian when a connection is available. This is common for tank farms or remote pipelines using general packet radio service (GPRS) or satellite modem communications.

The HMI/SCADA gateway sends machine operation data to the historian for long-term data storage. Other data is often entered into the historian manually, such as fault causes, detailed analysis, and quality assurance (QA) or statistical process control (SPC) product testing results.

The gateway accesses data stored in the historian for presentation to operators and engineers. The historian can also push this data to external data warehouses for security purposes, and it can push data to various software analyses platforms, or to business intelligence and overall equipment effectiveness (OEE) dashboards.

Historical machine data value

Although all HMIs provide multiple ways to view current machine operating data, it's often necessary to view historical machine operation data as it provides a number of benefits (Table 1).

Many changes are typically made to machine operation over the course of months or even years. Some are made in response to production demand, others are made manually in an effort to improve machine operation, and some are made automatically by the machine's control system. Taking these changes together, it can be difficult to know exactly which machine operation parameters were in place at any given time, and how these parameters affected machine output.

Viewing this type of historical data, often via the HMI, can allow plant personnel to compare current to historical machine operation, revealing ways to improve performance. For example, analysis of production information may show that very high quality products were being produced six months ago at a level that current production can't duplicate. Analyzing machine operating parameters can often reveal key differences, allowing operators and engineers to restore machine operation to an optimal level.

Predictive maintenance can be applied to machines by reviewing historical data and using it to adjust maintenance intervals. When a certain limit is reached, a certain time has elapsed, or certain measurements have been exceeded, the machine or process should be serviced to bring it back into operational specifications, or parts should be replaced. 


There are many ways to use historical data for statistical analysis. A very common way is to use an OEE display driven off the production data. The OEE indicates how well the plant or machine is operating, and can help pinpoint operational and production issues that might otherwise be overlooked.

Figure 2: Business intelligence dashboard and OEE displays can access data from a historian, and can be configured to provide critical operations data at a glance. InduSoft Web Studio and Wonderware InTouch Machine Edition can manage and control many thouAnother way is to use a business intelligence dashboard showing a more detailed view of the OEE data, which can be used to quickly pinpoint localized problems in a machine or process when compared to historical views (Figure 2).

Historical machine data also is used in enterprise resource planning (ERP) integration to make business decisions based on intelligent plant data. These business decisions might be as simple as ordering raw materials based on production demand, or scheduling main and backup suppliers. In addition, downstream uses of this data might inform or modulate the production line to not overrun the supply chain.

Historical data that ERP integration can provide also may be used to schedule ahead and create contracts for lower cost raw materials. The materials can be purchased based upon a historical need, rather at current market price. Regulatory agencies also may require production data and batch information to be stored for long periods of time in case of recall or other quality needs, as in the case of facilities required to comply with U.S. Food and Drug Administration (FDA) 21 CFR Part 11 and similar regulations. Purpose-built historians provide a foolproof method of saving and recalling data for these and other purposes.

Access to historical machine operation data can provide real-time benefits, but how can data be transferred between an HMI and a historian?

Going native

Linking an HMI to a historian for two-way information transfers isn't always an easy task, despite the preponderance of open systems. While it's possible in theory to link most any HMI to any historian, integration is not always easy in practice.

Both the HMI and the historian will most likely be PC-based, so a physical Ethernet connection will be easy to establish, along with a common Ethernet protocol. But linking the HMI data to specific historians can be an involved process, often consuming many human hours.

For this reason, many users prefer native solutions where the HMI and the historian are purchased from the same company. These solutions will be easier to implement than integrating products from two or more suppliers. [Parallel data structures also can help.]

For example, several HMI packages designed with the ability to manage and control many thousands of data points can easily and efficiently save that historical data several times per second to a historian, if needed. Once the HMI-historian integration challenge is solved, plant personnel can view and analyze historical data in a variety of ways (Table 2).

Trends and reports

Most every HMI product provides the ability to easily create trend screens to view data, and once the HMI is linked to the historian it's a simple matter to view and compare current to historical data on a trend screen. Viewing data in this fashion can often reveal areas for improvement. Many historians also allow easy creation of trend screens, providing another platform for viewing and analyzing data.

Similarly, most HMI and historian products provide the means to create a variety of reports. Historical data reports are required in implementations where batch information is paramount. These sorts of implementations are used in many life sciences, food, and beverage manufacturing facilities.

Long reports of analog data are, generally speaking, not very useful except perhaps in troubleshooting or when used with historical trending at-a-glance. However, batch reporting is almost always a requirement.

Figure 3: Historical data reports allow information to be presented in a variety of ways to meet the needs of plant personnel, management, and regulatory agencies. Courtesy: Wonderware by Schneider ElectricTherefore, reporting can take on a number of forms (Figure 3), and reports are usually saved as Web pages or Adobe PDF documents to preserve the integrity of the report across a wide variety of display platforms from large screens to handheld devices.

These reports are often used by management to get information needed at a glance.

Effective data transfer

Once machine and process data has been saved in a historian, it often needs to be transferred to the areas of the business that need it. Many times, the sheer amount of data in the historian may overwhelm the network connecting the historian to the business systems.

To reduce the traffic, most production historians simply push or replicate a data subset to one or more database servers or files that are made available for transfer to business systems. These replications are usually in the form of a stored procedure as one or more SQL queries, which can be exported to a variety of programs, such as spreadsheets, databases, or data analysis platforms.

These external platforms allow users to analyze data supplied to the historian by the HMI in a manner suiting their purposes, using familiar platforms. For example, many use spreadsheets to perform data analysis, as most are very familiar with these platforms. Others may use databases in their daily work and would prefer to use these tools for data analysis. Still others may require very sophisticated data analysis platforms dedicated to one purpose, such as SPC/statistical quality control (SQC) efforts.

Whatever the data viewing and analysis requirements, they can usually be met by connecting an HMI to a historian, which is in turn connected to other software platforms. This allows viewing and analysis of data at the HMI, the historian, and other software platforms.

- Richard Clark is automation and process control engineer, Wonderware by Schneider Electric; edited by Mark T. Hoske, content manager, CFE Media, Control Engineering,

Key concepts

  • Big data analysis and viewing can help by connecting human-machine interface (HMI) and historian software.
  • Historical data is more useful when it can influence events in real time. 
  • Using software from one company can simplify integration.

Consider this

What in your future might you change with real-time reminders of the past? What would that be worth? 

ONLINE extra 

This online portion of the May Control Engineering print and digital edition cover story includes more information and 3 links below for more HMI-historian integration help.

No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Additive manufacturing benefits; HMI and sensor tips; System integrator advice; Innovations from the industry
Robotic safety, collaboration, standards; DCS migration tips; IT/OT convergence; 2017 Control Engineering Salary and Career Survey
Integrated mobility; Artificial intelligence; Predictive motion control; Sensors and control system inputs; Asset Management; Cybersecurity
Featured articles highlight technologies that enable the Industrial Internet of Things, IIoT-related products and strategies to get data more easily to the user.
This article collection contains several articles on how automation and controls are helping human-machine interface (HMI) hardware and software advance.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Mobility as the means to offshore innovation; Preventing another Deepwater Horizon; ROVs as subsea robots; SCADA and the radio spectrum
Future of oil and gas projects; Reservoir models; The importance of SCADA to oil and gas
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me