Neural networks in process control: Neural network architecture, controls

Inside Process: Neural networks have been used in process control strategies for years, but they’re still not commonly found in industry. This technology has been applied in a number of fields with great success. With proper training to lift the veil from the technology, it can be more widely applied—without mystery—to solve some of the most nagging process control problems. This two-part series examines the process of producing a neural network application and includes tools to simplify the process. Part 1 of this 2-part series covers neural network architecture, control space, model range, data types, and dataset selection.

02/26/2016


Figure 1: The diagram shows a simplified view of a four-input, approximate-function, neural network with three neurons in the hidden layer. Courtesy: Process2Control, LLCAs the name implies, neural networks are composed of a network of neurons programmed to produce a response from external stimuli. The neuron is the basic building block of the network. It gets its name from its biological namesake. But in this case, the neuron is modeled by a small segment of computer code called a "perceptron." Several neurons are interconnected in a network that is taught how to respond to stimuli by training. It's an iterative process: present the stimuli, compare the response to a reference, and make the correction. The ability to learn gives neural networks great flexibility to capture the underlying function of a process, even with attributes that are not readily obvious, such as installation, age, fouling, or some other unmeasured parameter.

The purpose of using neural network models stem from their ability to:

  • Model a linear or nonlinear process
  • Model a process that is difficult to understand
  • Model a process that is difficult to model using first-principle equations
  • Model a process based on indirect measurements
  • Shorten model development time for a complex process.

There are several cases where neural network models may be beneficial in industrial processes. Applications where a model can be substituted for an unreliable measurement can impact profitability. In some cases, a controlled process may be manipulated only at discrete intervals due to lag in obtaining a control measurement. For example, a process may use lab analysis as feedback. The lab measurement has an inherit lag time. The neural network may be trained with the lab data to produce a virtual instrument the process can use for control on a continuous basis.

Model-based controllers can use neural network models in place of first-principle models. This can shorten development time and still allow control of multiple control variables where simultaneous setpoint and trajectory control are needed. And there's always the case of the process that is poorly understood, too complicated, or rapidly changing to apply first-principle models. Neural networks excel at finding the underlying process response from input stimuli.

Neural network architecture

Since the 1980s, different types of learning neural network architectures have been designed and analyzed. The choice of architecture depends on the application. In industrial control applications, generally speaking, models seek to mimic the function of some process variable; the target, based on the process conditions; and the inputs surrounding it. The simplest architecture for this task is the multilayer function approximation network architecture (see Figure 1). The number of neurons in the hidden layer depends on the complexity of the target function, but in general, they range from three to nine.

Control space

An important aspect of developing neural network models is the concept of measurement and control space. "Measurement space" is the multidimensional limits defined by the measurement range of each input representing one dimension. An array of inputs is called the "input vector." "Control space" is within measurement space, and its limits and shape depend on the point vector distribution of the input vector data records used for training. For example: There could be a number of independent inputs, p1 ... pk, into a model, and each would form a dimension in control space. For simplicity, assume only two inputs (see Figure 2).

Figure 2: The shape of the control space depends on the point vector distribution of the data records used for training. Assume two inputs for this example: two data records used for the model (a), the point vector formed from record 1 (b), each record foFigure 2: The shape of the control space depends on the point vector distribution of the data records used for training. Assume two inputs for this example: two data records used for the model (a), the point vector formed from record 1 (b), each record fo

Figure 2: The shape of the control space depends on the point vector distribution of the data records used for training. Assume two inputs for this example: two data records used for the model (a), the point vector formed from record 1 (b), each record foFigure 2: The shape of the control space depends on the point vector distribution of the data records used for training. Assume two inputs for this example: two data records used for the model (a), the point vector formed from record 1 (b), each record fo

The control space coverage is determined by the point vector distribution in Figure 3a. If the inputs form a point vector outside the control space as shown in Figure 3b, a neural network model may not be valid. This is because no training existed in that space. 

Figure 3: The graphs show a valid control space (a), an invalid control space (b), and the target range (c). Courtesy: Process2Control, LLCFigure 3: The graphs show a valid control space (a), an invalid control space (b), and the target range (c). Courtesy: Process2Control, LLC

Dataset selection

Of primary importance when selecting records for training a neural network is to ensure the record set covers not only the range but also the target response throughout the input range. Figure 3c shows an example of how the target response to the inputs reveals its range and function.

Figure 3: The graphs show a valid control space (a), an invalid control space (b), and the target range (c). Courtesy: Process2Control, LLC

Steps in acquiring a dataset suitable for training a neural network include choosing the target variable, selecting the input vector elements, handling of data types, historical data mining, and/or parametric testing. 

Data types: Binary, integer, and floating point are the primary data types recommended for the function approximation architecture. Enumeration and string-type data are special cases but may be used if broken it into individual binary inputs. 

Selecting target variable model input elements: Arguably, the most difficult task in creating a neural network model is selecting the process variables that make up the model input vector data elements, p1 ... pk. Model inputs should be robust, independent variables that have the most influence on the target. Experience is usually enough to perform the selection when the process is simple. However, in some cases, the process may have more than a few variables, and choosing the right ones may be challenging.

In addition, care must be given to minimize the number of inputs because every unnecessary input used in a model reduces the model's robustness, adds noise, and increases cost. Several tools exist to help exclude less significant inputs.

Figure 4: The fishbone diagram on the left shows all possible influences that could affect the target measurement. The goal is to reduce the variables to those that have significant influence on the target as shown in the fishbone diagram on the right. Co

Figure 4a shows a fishbone diagram with all possible influences that could affect the target measurement. Strive to reduce the variables to those that have significant influence on the target as shown in Figure 4b. A reduction of this kind can result in significant savings in modeling cost and instrumentation needs. In addition, it simplifies the model.

Figure 4: The fishbone diagram on the left shows all possible influences that could affect the target measurement. The goal is to reduce the variables to those that have significant influence on the target as shown in the fishbone diagram on the right. Co

There are commercially available programs that perform analysis of variances (ANOVA) to determine each variable's significance to the target. However, generating an ANOVA becomes much more difficult as the number of variables increases.

Another commercially available program uses neural networks to determine an input's significance. This method allows the user to use a shotgun approach. That is, as shown in Figure 5, the program uses a dataset containing all the inputs as shown in Figure 4a and identifies the significant inputs as in Figure 4b. 

Figure 5: This neural network significance analysis graph shows significant inputs above zero. Inputs below zero are either insignificant or in the noise band. Courtesy: Process2Control, LLC

Historical data and parametric testing: It is possible to do some historical datamining to acquire a data set for training. However, process conditions and the state of the system aren't always known from when the data was generated. It is highly recommended to create a measurement and control baseline before acquiring a data record set. To create a measurement and control baseline:

  • Calibrate all transmitters used as inputs into the model.
  • Check that valves, drives, and heaters are in good working order.

There are also some prerequisites to consider before parametric testing. They include:

  • Perform a hazard analysis prior to manipulating the process.
  • Define the limits of valves, speed controller, and so on, used in performing the test.
  • Have a safe condition to place the process in case of an unusual event.
  • Record any noise filter settings.
  • Define settling time after each parametric test maneuver.
  • Know safety limits and constraints when executing the parametric test.
  • Note any expected measurement lag or transport delays.
  • Make sure all data points are being recorded by the historian.
  • Make sure operations is aware of and agrees to the test procedure.

For new processes or retrofits, datasets are usually generated during system parametric testing after startup and tuning (see Figure 6). Experience has shown the best data for the model is collected after a variable has reached steady-state after a parametric test maneuver.

Figure 6: For new processes or retrofits, datasets are usually generated during system parametric testing after startup. These graphs show the manipulation of two valves (a) and three flow measurement responses from three different flowmeters. Courtesy: P

Figure 6: For new processes or retrofits, datasets are usually generated during system parametric testing after startup. These graphs show the manipulation of two valves (a) and three flow measurement responses from three different flowmeters. Courtesy: P

Realize that datasets are coupled to the current equipment and process configuration; consequently, datasets acquired prior to a retrofit or process change may not live up to expected performance. If the performance degrades a parametric test may be required again. 

Acquiring training dataset records: Now that the parametric test is complete and the data is stored in the historian, the objective is to obtain a record dataset for training the neural network. There are several historian software packages available on the market. Many have plug-ins for spreadsheet software packages. Figure 7 shows a predesigned spreadsheet.

Figure 7: A training record set was extracted from a historian to this partial spreadsheet. Courtesy: Process2Control, LLC

"Part 2: Neural networks in process control" will focus on preparing the dataset for training, neural network model training and validation, implementing a neural network model on a control platform, and human-machine interface (HMI) requirements. 

Jimmy W. Key, PE, CAP is president and owner of Process2Control, LLC in Birmingham, Ala. He is a professional control systems engineer with more than 30 years of experience in the pulp and paper, automotive, chemical, and power industries. Key has an MS in control system engineering from Oklahoma State University. He launched Process2Control in 2013 to integrate neural network modeling with other advanced control concepts and applications specifically for the process control industry. Edited by Jack Smith, content manager, CFE Media, Control Engineering, jsmith@cfemedia.com.

MORE ADVICE

Key concepts

  • Neural network models can be beneficial in industrial processes.
  • The choice of architecture depends on the application.
  • An important aspect of developing neural network models is the concept of measurement and control space.
  • It is possible to do some historical datamining to acquire a data set for training. 

Consider this

Understand how the abstract concepts of neural networks and advanced process control can be applied to real-world control scenarios. 

Resources

"Predictive process neural network model base controller," 2009, Process2Control, LLC

Hagan, Demuth, Beale, "Neural Network Design." ISBN 0-9717321-8

Minitab

Neural network significance analyzer, Process2Control, LLC 

ONLINE extra

Learn more about Process2Control as well as examples of neural networks.

See related articles below, offering more information about neural networks and advanced process control. 



WILLIAM , CA, United States, 03/07/16 09:11 PM:

For clarification, a "neural network" is what engineers have long termed a "curve-fit". The typical algorithm is actually poorer than the traditional non-linear fit (trial & error search, such as Levenberg-Marquadt, ex. Excel's "Solver"). A N-N weights more recent data stronger, which can be an advantage in some cases (follow web trends), otherwise best to "shuffle" the records to fit. One advantage is that the algorithm is faster (important for day traders).
Jimmy , United States, 08/20/16 07:30 PM:

The neural network referenced in this article uses the function approximation architecture. Training the neural network to "fit" any linear or non-linear function may use any of several optimization routines although I like Levenberg-Marquadt as it seems to reach a solution quicker.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Sensor-to-cloud interoperability; PID and digital control efficiency; Alarm management system design; Automotive industry advances
Make Big Data and Industrial Internet of Things work for you, 2017 Engineers' Choice Finalists, Avoid control design pitfalls, Managing IIoT processes
Engineering Leaders Under 40; System integration improving packaging operation; Process sensing; PID velocity; Cybersecurity and functional safety
This article collection contains several articles on the Industrial Internet of Things (IIoT) and how it is transforming manufacturing.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

SCADA at the junction, Managing risk through maintenance, Moving at the speed of data
Flexible offshore fire protection; Big Data's impact on operations; Bridging the skills gap; Identifying security risks
The digital oilfield: Utilizing Big Data can yield big savings; Virtualization a real solution; Tracking SIS performance
click me