Finding the right tools to delve into process data

Cover Story: New tools beyond the spreadsheet are needed to empower process engineers so they can derive insights from the data that is being transmitted.

By Michael Risse October 12, 2016

The amount of data collected and stored by process manufacturing organizations can reach terabytes per day; sometimes even per hour or minute, depending on the situation. The problem is, a great deal of this data isn’t used or leveraged to improve plant operations.

According to Gartner, 70% of data collected isn’t used for analytics or insight. McKinsey & Co. documented scenarios where the number was closer to 99% (see Figure 1). This is reflected by the reality known as data rich, information poor (DRIP) where users and organizations are drowning in data while desperately needing information at the same time (see Figure 2).

How can this be?

This is an important question for process manufacturing organizations. Delivering "actionable intelligence" has been a constant message for process analytics over the last two decades. A cloud of acronyms and, more recently, cloud computing, have been proposed as solutions to close the gap between data and insights. Even "Big Data," an expression that dates back to the 1990’s, hasn’t made a dent in the issue. Instead, more data is being created, and more is being unused. 

Process data benefits

It is helpful to consider who in a manufacturing facility is most responsible for process data insights and who is most likely to benefit from them. For most organizations, the obvious answer is the process engineer. A process engineer is a proxy for the engineering professionals within a production facility. Their actual titles are a combination of seniority, scope, system, and role. They have titles such as plant automation engineer, head of distributed control system (DCS) manufacturing, and principle asset process specialist.

Despite their different titles, what ties them together is that they are experts in their organization’s assets and processes. They know what to look for and which data sets are relevant for analysis. Process engineers represent the front line of employees with the ability and incentive to leverage production data to improve outcomes in quality, yield, and margins.

Another consideration is the tools process engineers rely on to find insights in process data. The most common bridge between data and insight for process manufacturing organizations is an engineer and a spreadsheet. This is obvious when looking at add-on products from historian vendors: every historian on the market offers a connector to a spreadsheet to help facilitate the investigation of time series data from production equipment. 

Possible solutions

The solution in accelerating the use of data to improve business outcomes has two components. First, it must enable and facilitate the process engineer’s work. This approach contrasts with many proposed innovations to the DRIP issues—Big Data, machine learning, advanced statistical packages-that focus on use and implementation by other types of employees such as software architects, programmers, and data scientists. This is an odd, but critical, miss in an environment where the engineer has the critical context in the plant and operations to derive insights from the data. The engineer should be the first rather than the last focus for production insight tools.

Second, the solution must address and/or upgrade the key capabilities offered by spreadsheets that have caused them to remain the engineer’s tool of choice for the last 30 years. Spreadsheets have become popular for a reason, and any next-generation solution to accelerate and close the gap between data and insights must improve upon this tool. Necessities include:

Data integration—According to ARC, the "average" investigation of a process manufacturing issue in an enterprise manufacturing intelligence environment requires access to seven different data systems. These include process historians, manufacturing enterprise systems (MES), enterprise asset management (EAM) platforms, pricing data in spreadsheets, and relational databases like an SQL server. Solutions that go beyond the spreadsheet must mirror their ability to ingest data in a variety of systems, formats, and types. It could be as simple as automatic interpolation and data alignment from multiple sources having different sample rates. Or it could be as complex as enabling the integration of continuous signals such as alarms or batch IDs. Spreadsheets are well-regarded for their dexterity in handling different types, and this capability must be duplicated and improved upon with any new solution. 

Flexibility—The process of a data investigation involves an iterative, trial and error approach to problem solving with many small steps such as data aggregation, data cleansing, calculations, visualizations, and collaboration. At the end of the process, a conclusion is either reached and implemented or not. In the latter case, the user moves backwards in their analysis and starts over on a new investigative path.

Having this flexibility is a critical part of the spreadsheet. The process engineer needs the ability to have full control over his or her access to data, transformations, and equations. The spreadsheet’s agility mirrors the plant environment, and it needs a motivated process engineer who can keep up with the variable changes that occur over time. The flexibility that spreadsheets provide is a critical component and a key criterion for any replacement. 

Reporting—The spreadsheet’s ubiquitous role in production and business environments allows users to share information as well as create new insights. They may be posted, printed, embedded, e-mailed, shared, and published by users. They may be viewed on any device, in an application or a browser, and run on-premise or in the cloud. Any solution proposing to replace the spreadsheet will need the same abilities. This might include view-only graphics like an executive dashboard, templates that limit user interaction to a valid range of options, or full fidelity access to enable collaboration across teams.

Beyond the spreadsheet

There are many opportunities to innovate and improve beyond these requirements. In the last 30 years, there have been innovations in Big Data technologies, data science, and horizontal scaling architectures. The true opportunity is for these innovations to be turned "inside out" and made accessible to the process engineer as an application experience or feature that can use without extensive training or specialized information technology (IT) skills.

Everyone understands this model from consumer experience such as when people use Google’s search engine or Siri’s voice recognition technology. Why can’t analytics applications mask this complexity in the same way for process engineers? This is a great opportunity to deliver the usability and flexibility of spreadsheets by leveraging the technology and innovation of our consumer experiences in a single solution. With this focus, organization will be able to tap into the opportunities from the insights that are found by closing the gap between data and information.

Putting the process engineer at the center of the process

The large and growing gap between data and insight will be addressed when vendors deliver solutions to employees with the expertise, ability, and incentive that take advantage of those insights. This means putting the process engineer front and center instead of offering solutions to employees who know IT and related technologies but know nothing about the process, asset, or plant.

Providing the process engineer with the right application to bridge the gap between data and information is essential. Modern solutions require just minutes to assemble, cleanse, and organize the data, which provides employees with more time to investigate production issues. The result is improved employee productivity and insights, which lead to better yields, margins, product quality, and safety (see Figure 3).

Michael Risse is a vice president at Seeq Corp., a company building productivity applications for engineers and analysts that accelerate insights into industrial process data. He was formerly a consultant with Big Data platform and application companies and prior to that worked with Microsoft for 20 years. Risse is a graduate of the University of Wisconsin at Madison and lives in Seattle. Edited by Chris Vavra, production editor, Control Engineering, CFE Media,


Key Concepts

  • There is more data being produced, but more of it is being unused, which is creating a widening gap.
  • Modern solutions to replace the spreadsheet must contain all of the features and characteristics that made the spreadsheet popular in the first place.
  • Process engineers need to be on the front lines in closing the widening data gap.

Consider this

What other features should the modern spreadsheet replacement have?

ONLINE extra

See related stories linked below.

Author Bio: Michael Risse is the CMO and vice president at Seeq Corporation, a company building advanced analytics applications for engineers and analysts that accelerate insights into industrial process data. He was formerly a consultant with big data platform and application companies, and prior to that worked with Microsoft for 20 years. Michael is a graduate of the University of Wisconsin at Madison, and he lives in Seattle.