Some best practices recommended for DCS migration
It’s dawning on major manufacturers that they must update or migrate their outdated and unsupported distributed control systems (DCS).
How did they get to this point?
In 1959, the computer revolution met the industrial revolution when the first industrial control computer system was built at the Texaco Port Arthur, Texas refinery. Over the next 30 years, industrial process control migrated from pneumatic control loops and relay logic to an ensemble of distributed control systems (DCSs) from different vendors.
At the time, these cutting-edge systems brought to bear high-value-adding technologies like graphical operator displays and improved data analysis software. New control strategies were implemented and tested, at significantly reduced engineering time, with quicker alarm response and diagnostics. DCS suppliers guaranteed long product lifecycles for these robust systems.
Today, however, these same DCS suppliers have consolidated their obsolete products or even discontinued them. Companies reliant on DCS technology realize migration is difficult and the process delicate. If a DCS is the “brain” of the facility, migration carries a degree of risk analogous to brain surgery. With that amount of risk, why would anyone choose to migrate? The answer is two-fold:
- To avoid the enormous cost and risk associated with losing control of a process
- To leverage innovative technologies that improve operational efficiency and performance.
Even knowing they need to migrate to stay competitive, uncertainty and doubt can persist moving forward, especially when key personnel bandwidth is limited. Migration is a lot of work. Risks run high without the right team of experts to plan and implement. To this end, consider the risks inherent to a migration and how modernization can help overcome these hurdles.
Hardware obsolescence is the main driver for migrations. If equipment fails, the associated risk can be enormous ─ both from a safety and financial perspective. Safety instrumented systems (SISs) help avoid a crisis, especially if the DCS fails. However, these systems are rarely tested with a view to an entire unit being shut down, such as would be the case in the event of a redundant controller failure, where actuators attempt to revert to their failsafe mode of operation.
Beyond safety issues, when a facility shuts down, it is no longer profitable. Downtime outside of preplanned outages and turnarounds is costly. Older DCS equipment, while built well, has a finite shelf life and will break at some point. Many legacy DCS parts are increasingly unavailable. If a facility needs spare parts, they end up purchasing more expensive and less reliable parts ─ the dreaded “eBay and pray.”
Another financial driver for doing an upgrade or migration is increasing lack of knowledge and service support for legacy hardware and software. Over the years, control engineers and technicians with legacy DCS experience have moved on from technical roles or are nearly at retirement age. With little original equipment manufacturer (OEM) support or training, this poses a problem for current systems but also pertaining to future system additions. Filling this knowledge gap can be difficult and costly as demand and salaries for experienced, knowledgeable engineers are high.
Installed cybersecurity vulnerabilities also must be examined. Legacy systems are from a time when it was assumed that the “air gap” protected systems from external, remote threats. However, with the advent of the Industrial Internet of Things (IIoT), the process-control system world is today connected to the business world. Technology and tools exist to secure, manage, version, analyze, track and report automation-related process information and data. However, with the “air gap” bridged, legacy systems are ripe for exploitation and represent a huge risk to any facility.
In short, at a certain point ─ which is now for many facilities ─ the cost of maintaining systems exceeds the cost of an upgrade or migration.
Today’s technologies provide opportunity to improve asset utilization and fully realize the benefits of Big Data, increased processing power, and enhanced connectivity. Many legacy control systems rely on proprietary data structures that lack systems interoperability. Increasingly, manufacturers look to use this data for a variety of functions, including analytics, environmental calculation and monitoring, enterprise resource planning integration, asset management, predictive maintenance, digital twins and various other industrial internet of things (IIoT) applications. Modern control systems deliver accurate, real-time data to enable these capabilities.
With increased process power, more complex control techniques can be applied. DCS platforms of 30 or 40 years ago bore constraints that are no longer valid. Process control techniques like initialization, windup protection and override selection are no longer an issue and are easily incorporated. Model predictive control and fuzzy logic no longer require additional software and licenses, allowing facilities to use simulation, whether simple loopback or more dynamic methods, instead of hardware-based testing. Offline configuration testing and operator training are conducted more easily, inexpensively, accurately, and safely than ever.
Legacy systems’ reliance on outdated user interfaces and unintuitive programming styles creates a situation that increases implementation time and costs, creating an obstructive barrier for new engineers or technicians. Maintaining and manipulating large databases using Microsoft Excel or XML formats instead of older text dumps enhances quality control and abilities to build and edit large amounts of programming in bulk.
Graphical function block configuration is a common feature in modern control systems and is easier to learn, maintain, and document than the older, text-based forms. Sequential logic programming and troubleshooting is also easier. Homing in on the stuck valve is significantly quicker than looking at a valve lineup to try to make sense of a poorly commented 20-year-old structured text program. It’s also possible to improve the operator human-machine interface (hmi), incorporating high-performance graphics for optimal operator awareness and alarm response.
Migration leads to a well-documented system. Legacy systems have been annotated by generations of workers using different standards. Abandoned logic obscures the system. Migration is a “once-in-a-career” opportunity to clean the slate and correct past bad habits and inconsistencies.
How to modernize
A complex DCS migration requires a planned, thoughtful approach. Because a failed project leads to cost increases, mitigating risk is paramount. If done right, a migration happens very infrequently, so it shouldn’t be taken lightly. Before starting, consider the following.
Domain knowledge: Domain and business expertise are what counts. You don’t hire an electrician to do plumbing, so why trust someone without experience to do a DCS migration? Every process unit in a facility has different inputs, equipment, operating conditions, safety risks and peculiarities, not obvious to someone without that experience. If lack of resource bandwidth is a concern, consider consulting a platform-independent third-party partner with experience in legacy systems and modernization tools and techniques.
Planning and budgeting phase: Emphasis can be placed on sound planning and budgeting via a front-end loading (FEL) process. The FEL process reduces operational project costs, minimizes scheduling issues, decreases scope changes after authorization, and increases likelihood that a project meets or exceeds financial goals. Defining an optimal future state with consideration about future features and IIoT functionality is another key part of this effort. This way avoids accidents and downtime, especially considering how intertwined process units are, with products and byproducts from different units being fed and reprocessed throughout the facility.
Ideally, for example, cutting over a control system should occur while the unit is down for maintenance. Some ancillary units may be able to be cutover during partial turnarounds or equipment inspections. Others only come down for full facility shutdowns, which may only occur every couple of years. Understanding each process unit’s maintenance schedule and interactions with other process units is key to an efficient FEL migration plan.
Commercial considerations: The point of manufacturing something is to generate revenue and profit. DCS migrations should protect and enhance that ability. Justifying how much the effort will cost and how long it will take is key to getting enterprise-wide buy-in. FEL planning should accurately determine the effort and time required for a migration.
For instance, it should include time and cost estimates to reverse-engineer and document an old system and engineer and test a replacement. It should also include DCS installation, electrical, hardware, and software costs. Discovery and documents must be modified and refined as the project progresses, resulting in overall project cost savings and lower risk. An experienced third-party automation team can help with proper, upfront FEL planning.
System and vendor selection: Choosing an appropriate replacement system is important. A wrong decision results in millions of lost dollars and years of frustration. Too often, the slogan “you’ll never pay more for less” will be heard. The selection process comes down to two possible choices: 1) If possible, stick with the same vendor, or 2) switch.
Sticking with a vendor can be a positive experience in that they typically have a well-defined path for migration plan and execution. Another benefit is that many control strategies can be ported more easily with control features with similar functionality to that of the legacy control system ─ which may not be the case with a new vendor.
A risk common to either path is that many vendors have a conflict of interest. They want to sell additional hardware and software and sometimes stoop to offering unnecessary or sub-optimal solutions. Moreover, during actual implementation, DCS vendors oftentimes rely on automated tools that do not accurately capture the original code. Correcting this code manually can be a huge undertaking.
During the typical “dog and pony show,” each vendor is predisposed to flaunting its “best-in-class” product and solution for every application, making it difficult to identify what’s best. To make an informed, unbiased decision, a facility should specify critical-to-quality (CTQ) parameters by working with key stakeholders: operators, process engineers, maintenance personnel, production managers and others. Parameters to consider include considerations like advanced process control (APC) support, scalability, life-support track record, high-performance graphics, migration path, industry and region install base, IIoT and control strategy portability.
Again, consulting an unbiased third-party partner with experience with the vendors and technologies leads to a holistic view of how these vendors stack in each of the CTQs.
Change is painful. Modernizing a control system is a huge, risky, but necessary, undertaking. Taking a proactive and planned approach allows for the required steps to be taken to mitigate risk and migrate the system accurately, safely and more cost effectively. The improved functionality and profitability gained from a modern, optimized control system will make it so that your engineers, technicians, facility managers, procurement and even investors will thank you.