What are you really controlling?
When designing a system and automating it, process narratives are often at a much higher level than programming languages. Numerous professionals are involved with each of the various areas: process, mechanical, electrical, programming, safety, environmental compliance, and others. Communication thus becomes a serious issue. It is further compounded by this requirement for a variety of languages, from the process language down to the program language.
The narrative may discuss opening a valve or draining a tank. But that is not what is being controlled. Those things are simply the desired end results from the arbitrary automated actions. It is also true that those desired goals may not be precisely measurable. And in fact they may not even be possible. For example, pumping a tank dry can lead to cavitation of your pump through vortexing long before the tank actually drains down. So in this case we could have designed a batch process that requires a 5,000-gal flat-bottom tank to drain empty before starting the next batch, yet we need to leave 500 gal in the bottom of the tank to protect the pump. Yes, there are ways around this and every seasoned engineer has faced this problem numerous times, but we have to understand the problem first, then seek the solution. And this problem highlights these issues fairly clearly and simply. Our process narrative at this point is not truly representative of our actual process anymore.
Where do we stand?
When a process narrative is not truly representative of the process, but just the abstract desired, then the result is unreachable because the control algorithm is based on a faulty narrative. Then the program, in a much lower level language, delivers orders that are not true to the algorithm but just attempt to fulfill those requests. Another problem is that many actuators are either manually set or operate on open-loop control, leaving the programmable logic controller (PLC) blind to many things happening in the physical process.
The narrative might say to fill the tank and let the mixture reach a steady state, then drain the tank. The algorithm looks for the signal from a high-level switch on the tank, starts a timer, and waits for the timer to stop. Then it then tells the automated control valve on the tank to open. Next, the pump starts and runs until the low-level switch signal reaches the PLC. The pump then turns off and the valve closes, sort of.
The program waits for a signal. It then sends a signal to the motorized valve. In a cost-saving concern, the valve is open-loop control. The program thus waits a calculated length of time that should allow the valve to open. Now the pump is turned on, maybe with a soft start, maybe on a variable frequency drive (VFD). The pump will run, we hope. Eventually, a signal reaches the PLC from the low-level switch. The PLC shuts the pump down. It again waits a calculated length of time for the pump to run down and stop. It then sends a signal to the valve to close it.
Now think of everything that could have gone wrong. Literally at every step in the process, the order has been to NOT "drain the tank." Nor is there certainty that the tank should be drained. The system has been given time for the desired reaction to occur-bleaching, for example-but the process may not be complete.
Feedback on the control process
Only two of the steps in the process had a feedback on the control, and then with zero redundancy. That feedback was a simple level switch. If the high-level switch failed, then either the tank probably overflowed or the pump turned on when the tank was only half full. If the low-level switch failed, then the pump cavitated or stopped running early, leaving the tank half full. If the valve did not open, then the pump cavitated and the batch probably sat there too long and was ruined. If the valve did not close, then the tank would be draining through a centrifugal pump and all future batches, as well as the downstream process, would be compromised. Isn’t that fun? If the pump did not start, the result probably was an overflow or a wasted batch. And with every issue, there is always an associated safety and/or environmental compliance consideration.
Most do not program the outcome. Programming usually comes in many steps to try to get what we want. The result may not be attainable, leaving something to settle on. If there are any communication errors or incorrect assumptions from anyone in design or implementation, from original process scope through to operation, results are unlikely to be as expected. Results may not work. If left with too much manual operation and open-loop control in the system, the PLC is going to operate blind. It will send orders based on assumptions, and proceed based on the hope that the previous orders were obeyed and there was enough time to fulfill them. Problems can cascade quickly at this point. Another downside is program buffers in the timings, which, when all added up, can drastically slow down processes. One of the reasons for automation is to optimize process run time, not to slow it down.
Failure mode effects analysis
There is no magic potion. There are, however, tools that can help. One helpful tool is a failure mode effects analysis (FMEA), which follows the process above, but a little more formally. Take every component and every task and ask:
- What can go wrong?
- How likely is it to happen?
- How much personal, environmental, financial, or process damage will it cause?
- What can we engineer into the system ahead of time to deal with it?
Results are surprising. There are numerous templates and instructional materials online to help with this. There are also consultants who can help develop a structured FMEA program.
A second tool is to take each process step in the narrative and break it down into those specific PLC inputs and outputs and again perform an FMEA on each one, and then string them together. Do this not just on the individual steps but also follow the steps with the failures in mind and see what the end result will be. Doing so builds a decision tree, letting the results of each action, good or bad, create a new branch. The bellow figure is a simplified example of an FMEA for a motorized valve. This example includes three possible scenarios for one task-opening a valve. Now when placing this into an overall process tree, three new branches sprout in the process tree. Other things can go wrong with the valve and the actuation, of course; there are more possible results and more options to deal with them as well. Each project requires its own analysis.
Be aware of what is being controlled
What this all sums up to is that there must be awareness of what is being controlled. We are not "draining a tank." That is simply the hopeful end result. Translate the extremely high-level language of the process narrative into the detailed language of the control program. Be aware that the control narrative is not telling the components what to do, but presents a step-by-step attempt to achieve a result that matches the control narrative. Be aware that many decisions are based on faith that the operators have all the 462 manual valves properly set and that the 208 open-loop actuated components have operated properly, for instance. The concept being read from the instruments is a conceptual representation of what is really happening. For example, a thermocouple in a tank is reading the specific temperature at a point, not the average temperature of the complete tank contents, nor the maximum or minimum: just the specific reading at one specific point.
Keeping these things in mind while incorporating tools like an FMEA and process tree will help automation achieve the results that the process narrative seeks.
– Paul Brake has a mechanical engineering degree with a business management minor, an electrician’s certification, and nearly three decades of combined engineering and manufacturing experience. His experiences cover water and wastewater equipment and systems design, oil refineries, forestry, mining, smelting, manufacturing, and custom factory automation. He specializes in machine design with an emphasis on the water and wastewater industries. He is listed as inventor for membrane bio-reactor patents in Canada and the United States. Edited by Joy Chang, digital project manager, Control Engineering, firstname.lastname@example.org.
See related stories below.
- There must be awareness of what is being controlled. Translate the extremely high-level language of the process narrative into the detailed language of the control program.
- Be aware that the control narrative is not telling the components what to do, but presents a step-by-step attempt to achieve a result that matches the control narrative.
- There are tools that can help. For example, a failure mode effects analysis (FMEA), which follows the control process more formally.
What kinds of tools are you using to determine what is being controlled during the automation process?