Neural networks in process control: Neural network architecture, controls

Inside Process: Neural networks have been used in process control strategies for years, but they’re still not commonly found in industry. This technology has been applied in a number of fields with great success. With proper training to lift the veil from the technology, it can be more widely applied—without mystery—to solve some of the most nagging process control problems. This two-part series examines the process of producing a neural network application and includes tools to simplify the process. Part 1 of this 2-part series covers neural network architecture, control space, model range, data types, and dataset selection.

By Jimmy W. Key, PE, CAP February 26, 2016

As the name implies, neural networks are composed of a network of neurons programmed to produce a response from external stimuli. The neuron is the basic building block of the network. It gets its name from its biological namesake. But in this case, the neuron is modeled by a small segment of computer code called a "perceptron." Several neurons are interconnected in a network that is taught how to respond to stimuli by training. It’s an iterative process: present the stimuli, compare the response to a reference, and make the correction. The ability to learn gives neural networks great flexibility to capture the underlying function of a process, even with attributes that are not readily obvious, such as installation, age, fouling, or some other unmeasured parameter.

The purpose of using neural network models stem from their ability to:

  • Model a linear or nonlinear process
  • Model a process that is difficult to understand
  • Model a process that is difficult to model using first-principle equations
  • Model a process based on indirect measurements
  • Shorten model development time for a complex process.

There are several cases where neural network models may be beneficial in industrial processes. Applications where a model can be substituted for an unreliable measurement can impact profitability. In some cases, a controlled process may be manipulated only at discrete intervals due to lag in obtaining a control measurement. For example, a process may use lab analysis as feedback. The lab measurement has an inherit lag time. The neural network may be trained with the lab data to produce a virtual instrument the process can use for control on a continuous basis.

Model-based controllers can use neural network models in place of first-principle models. This can shorten development time and still allow control of multiple control variables where simultaneous setpoint and trajectory control are needed. And there’s always the case of the process that is poorly understood, too complicated, or rapidly changing to apply first-principle models. Neural networks excel at finding the underlying process response from input stimuli.

Neural network architecture

Since the 1980s, different types of learning neural network architectures have been designed and analyzed. The choice of architecture depends on the application. In industrial control applications, generally speaking, models seek to mimic the function of some process variable; the target, based on the process conditions; and the inputs surrounding it. The simplest architecture for this task is the multilayer function approximation network architecture (see Figure 1). The number of neurons in the hidden layer depends on the complexity of the target function, but in general, they range from three to nine.

Control space

An important aspect of developing neural network models is the concept of measurement and control space. "Measurement space" is the multidimensional limits defined by the measurement range of each input representing one dimension. An array of inputs is called the "input vector." "Control space" is within measurement space, and its limits and shape depend on the point vector distribution of the input vector data records used for training. For example: There could be a number of independent inputs, p1 … pk, into a model, and each would form a dimension in control space. For simplicity, assume only two inputs (see Figure 2).

The control space coverage is determined by the point vector distribution in Figure 3a. If the inputs form a point vector outside the control space as shown in Figure 3b, a neural network model may not be valid. This is because no training existed in that space. 

Dataset selection

Of primary importance when selecting records for training a neural network is to ensure the record set covers not only the range but also the target response throughout the input range. Figure 3c shows an example of how the target response to the inputs reveals its range and function.

Steps in acquiring a dataset suitable for training a neural network include choosing the target variable, selecting the input vector elements, handling of data types, historical data mining, and/or parametric testing. 

Data types: Binary, integer, and floating point are the primary data types recommended for the function approximation architecture. Enumeration and string-type data are special cases but may be used if broken it into individual binary inputs. 

Selecting target variable model input elements: Arguably, the most difficult task in creating a neural network model is selecting the process variables that make up the model input vector data elements, p1 … pk. Model inputs should be robust, independent variables that have the most influence on the target. Experience is usually enough to perform the selection when the process is simple. However, in some cases, the process may have more than a few variables, and choosing the right ones may be challenging.

In addition, care must be given to minimize the number of inputs because every unnecessary input used in a model reduces the model’s robustness, adds noise, and increases cost. Several tools exist to help exclude less significant inputs.

Figure 4a shows a fishbone diagram with all possible influences that could affect the target measurement. Strive to reduce the variables to those that have significant influence on the target as shown in Figure 4b. A reduction of this kind can result in significant savings in modeling cost and instrumentation needs. In addition, it simplifies the model.

There are commercially available programs that perform analysis of variances (ANOVA) to determine each variable’s significance to the target. However, generating an ANOVA becomes much more difficult as the number of variables increases.

Another commercially available program uses neural networks to determine an input’s significance. This method allows the user to use a shotgun approach. That is, as shown in Figure 5, the program uses a dataset containing all the inputs as shown in Figure 4a and identifies the significant inputs as in Figure 4b. 

Historical data and parametric testing: It is possible to do some historical datamining to acquire a data set for training. However, process conditions and the state of the system aren’t always known from when the data was generated. It is highly recommended to create a measurement and control baseline before acquiring a data record set. To create a measurement and control baseline:

  • Calibrate all transmitters used as inputs into the model.
  • Check that valves, drives, and heaters are in good working order.

There are also some prerequisites to consider before parametric testing. They include:

  • Perform a hazard analysis prior to manipulating the process.
  • Define the limits of valves, speed controller, and so on, used in performing the test.
  • Have a safe condition to place the process in case of an unusual event.
  • Record any noise filter settings.
  • Define settling time after each parametric test maneuver.
  • Know safety limits and constraints when executing the parametric test.
  • Note any expected measurement lag or transport delays.
  • Make sure all data points are being recorded by the historian.
  • Make sure operations is aware of and agrees to the test procedure.

For new processes or retrofits, datasets are usually generated during system parametric testing after startup and tuning (see Figure 6). Experience has shown the best data for the model is collected after a variable has reached steady-state after a parametric test maneuver.

Realize that datasets are coupled to the current equipment and process configuration; consequently, datasets acquired prior to a retrofit or process change may not live up to expected performance. If the performance degrades a parametric test may be required again. 

Acquiring training dataset records: Now that the parametric test is complete and the data is stored in the historian, the objective is to obtain a record dataset for training the neural network. There are several historian software packages available on the market. Many have plug-ins for spreadsheet software packages. Figure 7 shows a predesigned spreadsheet.

"Part 2: Neural networks in process control" will focus on preparing the dataset for training, neural network model training and validation, implementing a neural network model on a control platform, and human-machine interface (HMI) requirements. 

Jimmy W. Key, PE, CAP is president and owner of Process2Control, LLC in Birmingham, Ala. He is a professional control systems engineer with more than 30 years of experience in the pulp and paper, automotive, chemical, and power industries. Key has an MS in control system engineering from Oklahoma State University. He launched Process2Control in 2013 to integrate neural network modeling with other advanced control concepts and applications specifically for the process control industry. Edited by Jack Smith, content manager, CFE Media, Control Engineering, jsmith@cfemedia.com.

MORE ADVICE

Key concepts

  • Neural network models can be beneficial in industrial processes.
  • The choice of architecture depends on the application.
  • An important aspect of developing neural network models is the concept of measurement and control space.
  • It is possible to do some historical datamining to acquire a data set for training. 

Consider this

Understand how the abstract concepts of neural networks and advanced process control can be applied to real-world control scenarios. 

Resources

"Predictive process neural network model base controller," 2009, Process2Control, LLC

Hagan, Demuth, Beale, "Neural Network Design." ISBN 0-9717321-8

Minitab

Neural network significance analyzer, Process2Control, LLC 

ONLINE extra

Learn more about Process2Control as well as examples of neural networks.

See related articles below, offering more information about neural networks and advanced process control.