Designing real-time process controllers

The most important constraint in a real-time system is that the time it takes to process an input and produce an output must be well known.

By Mario Torre, Halliburton October 23, 2014

Today, almost all automation and control systems are designed, developed, and built using data processors, microprocessors, DSPs, or any other processing device that executes instructions derived or compiled from a software program. Very few and specialized control devices still use plain hardware or programmable hardware using a field programmable gate array to accomplish the job for which they were designed.

Nevertheless, to design and develop software applications for a controller or automation device, special skills and a good understanding of the automation and control problem are required. Many designers and programmers build web servers, information systems, business process management systems, and many other important and valuable infrastructures based on computing platforms. But when it comes to industrial process controllers, the designer recognizes that a new design and programming paradigm is required. This is well accepted throughout the automation and control industry.

Real-time processing

Similarly, many concepts involving signal acquisition, transducers, control setpoints, and others are related to process-state variables, measurements, and control. But mostly, they are closely connected with real-time processing. The concept of real-time processing must be considered when any engineer designs, develops, and deploys a new automation and control system. This is what differentiates automation software designers from designers of any other application software. It’s not better or worse, easier or harder; it’s just different.

Automation controllers must be real-time systems because they must control physical processes or plants that demand real-time control. At this point, the automation controller designer must have a very good idea of what a real-time system is.

Hermann Kopetz, in his book, "Real Time Systems: Design Principles for Distributed Embedded Applications" (2011), stated that "A real-time computer system is a computer system where the correctness of the system behavior depends not only on the logical results of the computations, but also on the physical time when these results are produced. By system behavior we mean the sequence of outputs in time of a system."

For any physical plant to be governed, it requires a controller capable of acquiring some input signals and producing some specific output signals within a very specific time frame. If the controller outputs occur outside such time frame, those outputs will no longer be valid, and will produce malfunctions or even a catastrophic failure in the physical plant.

Real-time explanation

How can we determine that a computer system is a real-time system? Suppose there is a very simple system, with just one input and one output. Such system generates an output signal every time it receives an input signal (see Figure 1).

Now, suppose that we repeatedly (but not periodically) apply the same input signal to this system. We may expect that this system will generate the same output signal every time, at exactly t seconds after the input signal is applied. Unfortunately, this is not what happens in actual controllers. The same output signal will be generated by the controller every time, but the time t that it takes the controller to produce each output may increase slightly (see Figure 2). If we repeat this experiment, we will find that the controller response times fall into a variation interval, say Δt. Some literature refers to Δt as "latency jitter."

The question is: Who determines this magnitude Δt? It strongly depends on how the controller hardware was designed, which components were used, and how the application software running inside it was designed and developed.

It should be noted that the time magnitude t is not related to the time variation value, Δt. The former depends on the function that the controller is supposed to perform, while the latter depends on how the designer implemented this function inside the controller.

It is important to emphasize that no controller system can reach Δt = 0. Therefore, the controller designer must know the maximum that the physical plant would allow without causing failures or damages, and consequently he or she must build the right real-time controller that takes into account such constraint.

Hard real time, soft real time

Depending on the physical plant and the type of control to be performed, the controller may be classified as "hard real time" or "soft real time." If the specific characteristics of the plant or process to be controlled are such that noncompliance of the Δt constraint will produce a malfunction or a failure, then the controller must be a "hard real-time" controller.

If, on the other hand, the specific characteristics to be controlled are such that noncompliance of its Δt constraint will generate a degradation of the plant functionality, but will not produce any malfunctions or failures, then a "soft real-time" controller may be used.

Latency, reaction time

The value of Δt to be used as the main constraint for a process controller design depends heavily on the nature of the process to be controlled. Each physical process has its own "latency," that is, the mean time the process reacts to a change in one or several of the inputs. This latency is tied to the physical, chemical, and electrical laws governing that process. For example, the latency of an oil production process is very different from the latency of an electric power transmission system. Average latencies vary by process, which is one reason specific process knowledge-as well as control engineering knowledge-is so important (see Table 1).

Based on experience, a basic rule-of-thumb for automation controller design is that the controller’s Δt must be at least 5 times smaller than the latency of the process the controller is intended to govern. Of course, this also depends on many other considerations, such as power consumption, heat dissipation, available space, and many other constraints not discussed here.

Designing a hard real-time controller is much more difficult than building a soft real-time controller. Depending on the physical process and the specific application, the designer must choose to build one or the other.

Most controllers for industrial applications available in the market are soft real-time controllers. The good news is that almost all industrial automation and control applications may be governed by soft real-time controllers. Only in a few, very specific situations are hard real-time controllers justified and installed. But, as long as a good controller is designed or chosen, in which its latency jitter Δt is small enough compared to the process latency, a reliable automation solution may be provided.

Designing the real-time controller

The most important constraint in a real-time system is that the time required for processing an input and producing an output must be well known. In other words, the processing time of a real-time system to get a solution is part of the solution. Of course, for a particular system, such time is not always exactly the same; it may vary. This deviation, which we have called "latency jitter" or Δt, essentially defines the "determinism" that a real-time system is providing. The closer Δt approaches zero, the more deterministic a real-time system is.

To be governed, any physical plant requires a real-time controller, which is, per se, a real-time system. The controller Δt must be at least 5 times smaller than the latency of the plant. This is a very general rule-of-thumb, and may be different depending on the application and other considerations and trade-offs. However, the controller designer must use a specific Δt when designing a real-time controller.

So, now the question is: How can we reliably design a controller that guarantees some specific Δt? For simplicity, the discussion here will:

  • Consider the design of an electronic controller for a plant.
  • Assume that all sensors, transducers, and actuators are readily available, and that all inputs and outputs are electrical signals.
  • Consider that this controller will be a digital controller, that is, analog-to-digital conversions are made, and the control functions are being made by logical and mathematical operations.

In brief, this controller will be developed using (mostly) digital electronic components, PC boards, and related parts.

The most basic approach for any electronic designer to come up with a controller is to go with discrete components. Any digital controller may be implemented using logic gates, flip-flops, and buffers. Those components have a great performance in regard to latency jitter. The Δt of most of those components is very low (depending on gate technology, it is about 40-50 nanosec), so the overall controller latency jitter typically won’t exceed 1 microsec, which is good enough for most real-time applications-even hard real time.

The main issue with this hardware approach is its lack of flexibility and expandability. If there’s a need for an extra input or a change to controller functionality, then a new controller must be designed. However, it is important to say that this hardware approach was widely used in the past, and it is still being used in very specific applications-most of them in the hard real-time domain.

To date, almost all digital controllers are microprocessor-based controllers. The designer is no longer entitled to build the hardware for a controller. Controller hardware has been standardized, and all signal treatment, filtering, and processing are managed by microprocessors. Most of them have several processing cores, depending on the type and purpose of the controller. It may sound simple, but to date the work of a controller designer has been reduced to:

  • Selecting the right controller hardware according to the application
  • Choosing the operating system to be used
  • Selecting the programming environment or framework that will further help with the development
  • Developing a software application to implement the actual controller function that is needed for the plant that the control designer envisions to control.

Selecting the right hardware controller for the application may be, by itself, a complicated task. The characteristics of hardware controllers will depend on the elapsed time t required for the controller to provide the solution, and the latency jitter Δt that is expected from such controller. It is also determined by many other factors and conditions that go beyond considerations here. For this discussion, let’s say that the correct hardware selection criteria will need to meet the following requirements:

  • Must be fast enough to handle the required elapsed time and latency jitter
  • Must be able to handle processor interrupts in real-time
  • Must provide an "adequate" development and run-time software environment in which the designer can design and deploy the required software application.

After the hardware is selected, the designer creates the software application using a computer language according to the chosen software development environment. Currently, hardware controllers include a rich, intuitive software development environment, where applications may run over some specific operating system and a programming framework. Although it is not mandatory, the use of the hardware manufacturer’s suggested programming framework is advised. Please note that designing software applications without using any software environment is similar to building the hardware out of discrete components and chips, as explained earlier.

Suppose that the designer already selected the hardware and is about to design a new software application on top of the selected operating system using a programming language. Depending on the programming language preferred, there may be more software layers between the hardware and the actual controller application: frameworks (such as Microsoft.NET), virtual machines (such as Oracle Java), or even hardware manufacturer proprietary frameworks or programming interfaces.

All these layers further separate the controller application from the input and output signals. Also, it must be noted that each of those layers adds latencies and processing times. Figure 3 shows the processing layers for a typical controller implementation.

As displayed in Figure 3, an input event must be handled from its occurrence, through all the layers, up to the actual application that will process it. After it is processed, the application must send the results to the actual output port, which also must go through the same layers. The total response time for this simple control application, using the above stack of hardware and software layers, can be calculated as:

t = t1 + t2 + t3 + t4 + t5 + t6 + t7

However, under this scenario, how can the designer guarantee that the total response time t will be the required one for the plant? How can the control designer ensure that such response time will be within a specific latency jitter ∆t? Table 2 shows a detailed explanation of each time component and how each one affects Δt and also gives some initial recommendations on how to achieve the desired results.

Each of the time components mentioned above provides a portion of the total resulting controller latency jitter Δt. One of the controller designer’s main duties is to assess every one of those components, and adequately weight them to make sure that:

  • Resulting controller latency t is the required one
  • The sum of Δts from all time components will not go beyond the desired latency jitter.

This is not an easy task. Please note that what is exposed in Table 2 is a very simplistic first view of a real-time controller design. There are many more considerations that must be made by the controller designer and assigned team that also affect selections of hardware, operating systems, and frameworks far beyond the strict real-time performance of the controller.

Mario Torre is a real-time systems design and architect specialist, focused on digital oil field automation, control, and real-time information systems. He is an electronics engineer with a master’s degree in systems engineering. He is currently a consulting advisor in the Intelligent Operations Solutions Global Group at Halliburton Landmark Software and Services in Houston.

This article appears in the Applied Automation supplement for Control Engineering and Plant Engineering