Ensuring effective decision support in a crisis

Machines can support humans when facing a stressful situation. Using a standards-based approach can reduce the likelihood of a problem escalating.

06/02/2017


Figure 1: The Boeing B-17 procedure checklist became a prototype for many more to come in aviation. Courtesy: Yokogawa Electric CorporationOperator error during periods of abnormal operations has been put forward as one of the causes of many major recent incidents. But before we give humans a bad rap, incident reports suggest the problem often stems from poor procedures, inadequate training, and the lack of sufficient resources. In many cases, with the right skills and tools, a good operator can help avoid these situations.

Arguably, the most advanced decision support systems may be found in the aircraft industry. But even these can go wrong sometimes, and it comes back to the skills and training of humans to avoid potential disasters, aided by a standards-based approach. 

Putting humans under stress

Process control systems have evolved over the years to the point where we can measure, display, and alarm almost anything in almost any color. We can provide many different alarms on the same measurement, including various high and low values, as well as rate of change. We build operator displays that look artistically great, but can confuse the operator in an emergency. But when configured correctly, these alarms and displays can help rather than confuse. Unfortunately, we often don't use this system intelligence to benefit the process operator.

On March 23, 2005, there was an explosion in the isomerization unit of the BP Texas City Refinery, which at the time was BP's largest facility. The explosion killed 15 people and injured 170. The incident centered around the raffinate splitter.

BP's incident investigation, led by J. Mogford, issued a report showing several basic procedure-related errors, such as a level alarm acknowledged but not acted upon, a heat-up ramp-rate that was too fast, and operators trying start up the unit in manual when procedures indicated it should be in automatic. Moreover, operators turned on the burners before verifying liquid was circulating. Later, we will examine how a standards-based approach may have averted this incident.

Another clear example of operator overload happened on Sunday, July 24, 1994, when a lightning strike started a fire on the crude distillation unit at the Texaco Milford Haven refinery, which eventually led to an explosion on the fluid catalytic cracking unit (FCCU). Although the media put the blame on the lightning strike, the incident report stated, "These events, though significant in initiating a plant upset, were not the cause of the release and explosion that occurred five hours later. These consequences resulted from subsequent failures to manage the plant upset safely."

Luckily, although there were some serious injuries, no one was killed. Among many other things, the report cited bad alarm management, poor human-machine interface (HMI) display design, and a failure to follow procedures. For example, the report stated, "From the limited amount of alarm information relevant to the event, which was preserved from just one of the journals, it was seen that in the last 10.7 minutes before the explosion, the two operators had to recognize, acknowledge, and take appropriate action on 275 alarms. At times during the morning, operators were doing nothing but acknowledging alarms."

The report went on to say the chances of operators restoring control manually were reduced as the incident progressed due to them being overloaded by a "barrage of alarms." There were 2,040 alarms configured, 87% of which were high priority. During the incident, the operators had to cope with alarms coming in at a rate of one every 2 to 3 seconds, which resulted in many simply being cancelled. There was no evidence that a vital high-level alarm on the flare drum that went off 25 minutes before the explosion was ever seen.

In addition, the report indicated the FCCU HMI graphics were not designed in a way to help the operators control the process. Process data was limited and color use was confusing, so important data was not highlighted. Much of what was displayed illustrated the structure of plant equipment and had no relevance to operations. Critical procedures had fallen into disuse from lack of practice and documentation. 

Figure 2: Capturing best-practice procedures using appropriate management of change can result in significant improvements. Courtesy: Yokogawa Electric CorporationThe role of procedures

These incidents show how the effective use of procedures is one of the key items in maintaining safe and reliable operations under all conditions. In fact, if configured correctly, well-planned alarms can trigger procedures in many abnormal situations, and a well-designed HMI can bring a developing incident to the attention of an operator in a timely manner.

For example, the airline industry is among the safest and most automated in the world. In fact, most modern aircraft could not fly without the use of computer guidance, yet procedures play a big part in the way aircraft are operated. Pilots need to go through many procedures before, during, and after a flight.

History suggests recorded procedures were introduced by test pilots in 1935 after the crash of a B-17 Flying Fortress in Dayton, Ohio. The B-17 was the most advanced bomber at the time, but the crash almost caused the program to be abandoned due to a gust lock still being engaged at takeoff. It was said that the plane was too complicated to fly.

In response, test pilots developed procedures for use during takeoff, in-flight, before landing, and after landing. Boeing eventually delivered more than 12,000 of the aircraft to the U.S. Air Corps, and they flew 1.8 million miles without a serious mishap. An example of the B-17 procedures is shown in Figure 1. Every type of aircraft from small private planes to the largest jumbo jet now use procedures for all aspects of the journey, and not following them could lead to a pilot losing his or her license, or worse.

Another example of outstanding use of procedures is the now famous "Miracle on the Hudson." Captain Chesley (Sully) Sullenberger and his crew saved U.S. Airways flight 1549 on Jan. 15, 2009, when the plane struck a flock of geese just after takeoff from La Guardia airport in New York. They landed the plane safely on the Hudson. It turned out that none of the crew had flown together before, but the procedures drilled into all airline crew enabled them to do all the necessary things by rote.

In the process industries, we use standard operating procedures (SOPs) for all aspects of running a process, under all conditions. However, some of the better operators often tweak procedures to improve them. As experienced operators are retiring with often less experienced operators replacing them, plants try to capture these tweaks to develop best-practice procedures (see Figure 2).

These procedures can be run semi-automatically, where the control system runs the steps to a point where the operator must confirm it is safe to continue, or the control system runs the procedure completely automatically. The machine runs the process, but there is always a need for human oversight. 


<< First < Previous Page 1 Page 2 Next > Last >>

The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Maximize ROI with integrated control system approach; Microcontrollers vs. PLCs; Power quality; Accelerate and rewire IIoT; Traits for excellent engineers
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Maximize ROI with integrated control system approach; Microcontrollers vs. PLCs; Power quality; Accelerate and rewire IIoT; Traits for excellent engineers
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Maximize ROI with integrated control system approach; Microcontrollers vs. PLCs; Power quality; Accelerate and rewire IIoT; Traits for excellent engineers
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me