Finding the right tools to delve into process data

Cover Story: New tools beyond the spreadsheet are needed to empower process engineers so they can derive insights from the data that is being transmitted.


The amount of data collected and stored by process manufacturing organizations can reach terabytes per day; sometimes even per hour or minute, depending on the situation. The problem is, a great deal of this data isn't used or leveraged to improve plant operations.

According to Gartner, 70% of data collected isn't used for analytics or insight. McKinsey & Co. documented scenarios where the number was closer to 99% (see Figure 1). This is reflected by the reality known as data rich, information poor (DRIP) where users and organizations are drowning in data while desperately needing information at the same time (see Figure 2).

How can this be?

This is an important question for process manufacturing organizations. Delivering "actionable intelligence" has been a constant message for process analytics over the last two decades. A cloud of acronyms and, more recently, cloud computing, have been proposed as solutions to close the gap between data and insights. Even "Big Data," an expression that dates back to the 1990's, hasn't made a dent in the issue. Instead, more data is being created, and more is being unused. 

Process data benefits

Figure 1: The management consultant firm McKinsey & Company has documented cases where up to 99% of collected data goes unused. Courtesy: McKinsey and CompanyIt is helpful to consider who in a manufacturing facility is most responsible for process data insights and who is most likely to benefit from them. For most organizations, the obvious answer is the process engineer. A process engineer is a proxy for the engineering professionals within a production facility. Their actual titles are a combination of seniority, scope, system, and role. They have titles such as plant automation engineer, head of distributed control system (DCS) manufacturing, and principle asset process specialist.

Despite their different titles, what ties them together is that they are experts in their organization's assets and processes. They know what to look for and which data sets are relevant for analysis. Process engineers represent the front line of employees with the ability and incentive to leverage production data to improve outcomes in quality, yield, and margins.

Another consideration is the tools process engineers rely on to find insights in process data. The most common bridge between data and insight for process manufacturing organizations is an engineer and a spreadsheet. This is obvious when looking at add-on products from historian vendors: every historian on the market offers a connector to a spreadsheet to help facilitate the investigation of time series data from production equipment. 

Possible solutions

The solution in accelerating the use of data to improve business outcomes has two components. First, it must enable and facilitate the process engineer's work. This approach contrasts with many proposed innovations to the DRIP issues—Big Data, machine learning, advanced statistical packages-that focus on use and implementation by other types of employees such as software architects, programmers, and data scientists. This is an odd, but critical, miss in an environment where the engineer has the critical context in the plant and operations to derive insights from the data. The engineer should be the first rather than the last focus for production insight tools.

Figure 2: Many process industry firms find they have a wealth of data and a paucity of insights and information. This is known as data rich, information poor (DRIP). Courtesy: SeeqSecond, the solution must address and/or upgrade the key capabilities offered by spreadsheets that have caused them to remain the engineer's tool of choice for the last 30 years. Spreadsheets have become popular for a reason, and any next-generation solution to accelerate and close the gap between data and insights must improve upon this tool. Necessities include:

Data integration—According to ARC, the "average" investigation of a process manufacturing issue in an enterprise manufacturing intelligence environment requires access to seven different data systems. These include process historians, manufacturing enterprise systems (MES), enterprise asset management (EAM) platforms, pricing data in spreadsheets, and relational databases like an SQL server. Solutions that go beyond the spreadsheet must mirror their ability to ingest data in a variety of systems, formats, and types. It could be as simple as automatic interpolation and data alignment from multiple sources having different sample rates. Or it could be as complex as enabling the integration of continuous signals such as alarms or batch IDs. Spreadsheets are well-regarded for their dexterity in handling different types, and this capability must be duplicated and improved upon with any new solution. 

Flexibility—The process of a data investigation involves an iterative, trial and error approach to problem solving with many small steps such as data aggregation, data cleansing, calculations, visualizations, and collaboration. At the end of the process, a conclusion is either reached and implemented or not. In the latter case, the user moves backwards in their analysis and starts over on a new investigative path.

Having this flexibility is a critical part of the spreadsheet. The process engineer needs the ability to have full control over his or her access to data, transformations, and equations. The spreadsheet's agility mirrors the plant environment, and it needs a motivated process engineer who can keep up with the variable changes that occur over time. The flexibility that spreadsheets provide is a critical component and a key criterion for any replacement. 

Reporting—The spreadsheet's ubiquitous role in production and business environments allows users to share information as well as create new insights. They may be posted, printed, embedded, e-mailed, shared, and published by users. They may be viewed on any device, in an application or a browser, and run on-premise or in the cloud. Any solution proposing to replace the spreadsheet will need the same abilities. This might include view-only graphics like an executive dashboard, templates that limit user interaction to a valid range of options, or full fidelity access to enable collaboration across teams.

Beyond the spreadsheet

There are many opportunities to innovate and improve beyond these requirements. In the last 30 years, there have been innovations in Big Data technologies, data science, and horizontal scaling architectures. The true opportunity is for these innovations to be turned "inside out" and made accessible to the process engineer as an application experience or feature that can use without extensive training or specialized information technology (IT) skills.

Figure 3: Information presented in a visual format allows engineers and other manufacturing experts to directly interact with data and gain quick insight. Courtesy: Seeq

Everyone understands this model from consumer experience such as when people use Google's search engine or Siri's voice recognition technology. Why can't analytics applications mask this complexity in the same way for process engineers? This is a great opportunity to deliver the usability and flexibility of spreadsheets by leveraging the technology and innovation of our consumer experiences in a single solution. With this focus, organization will be able to tap into the opportunities from the insights that are found by closing the gap between data and information.

Putting the process engineer at the center of the process

The large and growing gap between data and insight will be addressed when vendors deliver solutions to employees with the expertise, ability, and incentive that take advantage of those insights. This means putting the process engineer front and center instead of offering solutions to employees who know IT and related technologies but know nothing about the process, asset, or plant.

Providing the process engineer with the right application to bridge the gap between data and information is essential. Modern solutions require just minutes to assemble, cleanse, and organize the data, which provides employees with more time to investigate production issues. The result is improved employee productivity and insights, which lead to better yields, margins, product quality, and safety (see Figure 3).

Michael Risse is a vice president at Seeq Corp., a company building productivity applications for engineers and analysts that accelerate insights into industrial process data. He was formerly a consultant with Big Data platform and application companies and prior to that worked with Microsoft for 20 years. Risse is a graduate of the University of Wisconsin at Madison and lives in Seattle. Edited by Chris Vavra, production editor, Control Engineering, CFE Media,


Key Concepts

  • There is more data being produced, but more of it is being unused, which is creating a widening gap.
  • Modern solutions to replace the spreadsheet must contain all of the features and characteristics that made the spreadsheet popular in the first place.
  • Process engineers need to be on the front lines in closing the widening data gap.

Consider this

What other features should the modern spreadsheet replacement have?

ONLINE extra

See related stories linked below.

No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Make Big Data and Industrial Internet of Things work for you, 2017 Engineers' Choice Finalists, Avoid control design pitfalls, Managing IIoT processes
Engineering Leaders Under 40; System integration improving packaging operation; Process sensing; PID velocity; Cybersecurity and functional safety
Mobile HMI; PID tuning tips; Mechatronics; Intelligent project management; Cybersecurity in Russia; Engineering education; Road to IANA
This article collection contains several articles on the Industrial Internet of Things (IIoT) and how it is transforming manufacturing.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

SCADA at the junction, Managing risk through maintenance, Moving at the speed of data
Flexible offshore fire protection; Big Data's impact on operations; Bridging the skills gap; Identifying security risks
The digital oilfield: Utilizing Big Data can yield big savings; Virtualization a real solution; Tracking SIS performance
click me