Migration gone bad
One extensively documented process safety incident is particularly germane to this larger discussion because it involves a control system migration with new distributed control systems (DCSs) and human-machine interfaces (HMIs) but with inadequate operator training. The incident happened at the Bayer CropScience plant in Institute, W. Va., on August 28, 2008, when a runaway chemical reaction in its methomyl pesticide production unit caused a 4,500-gal pressure vessel partially filled with flammable solvents to explode. There were two fatalities and much damage to the plant from the explosion and fire that burned for more than four hours. This incident has been thoroughly studied by the CSB (Chemical Safety Board), and its report released in January 2011 provides the information for this discussion. The entire report makes for fascinating reading which is also unsettling in many respects. Its summary called out five causes for the explosion, three of which touch on the control system migration project the plant had just executed with the second hitting the nail on the head:
"2. Operations personnel were inadequately trained to operate the methomyl unit with the new DCS."
Here are some key facts from the report:
- The pesticide produced at this plant is used seasonally in agricultural applications, so the company only produced it at one time each year. The idle time on the unit was used for maintenance and in this case, for the control system upgrade.
- Two related production units were located side-by-side in the facility, Larvin and methomyl, and shared a control room although the two operated separately.
- The Larvin unit had undergone a similar DCS migration the year before. That changeover had worked well and everyone at the plant was satisfied with the implementation.
- The integrator that installed the Larvin system created a comprehensive training program for the operators, including time on a process simulator. Before the unit restarted, documentation was complete, and the operators were confident and well prepared to move to the new system.
Such was not the case when the methomyl unit was getting ready to restart. Quoting again from the CSB report:
"Management concluded that comprehensive formal training and practice using the new DCS on the methomyl process w[ere] unnecessary. They incorrectly assumed the methomyl and oxime board operators had become proficient from the many operating hours using the DCS on the Larvin unit. Methomyl and oxime board operators had minimal training on a few specific processes, but general training took place during the operators’ shift as time allowed and was self-directed and self-paced. Informal, on-the-job training intended to develop the necessary skills to run the system can lead to inappropriate or incorrect practices that became the norm in the absence of proper training tools and instruction (CCPS, 1994). The CSB concluded the training was inadequate."
"Prior to the methomyl startup, management provided operators time on the console during the DCS upgrade to practice using the new system. However, management did not require any methomyl operator to use this time to learn and practice operating the methomyl unit, and operators could decide for themselves how much time they needed to become familiar with the new DCS. Management also assumed that operators directly involved in designing the mimic displays and other customizable features would have had adequate exposure to the new system."
The report makes the point that the upgrade project followed normal procedures. The operators were involved in analyzing the old HMI graphics and participated in design of the new ones, but contrary to their assumptions, it was not enough to infuse the level of competence necessary. Starting up a process unit is tricky under the best of circumstances, but with inadequate training on a complex system, the result can be disastrous.
Peter Welander is a contributing content specialist for Control Engineering. Edited by Chris Vavra, production editor, Control Engineering, CFE Media, firstname.lastname@example.org.
See related stories on high-performance HMIs and more information on DCS migrations linked below.