7 ways to mitigate brownfield automation integration risks

Smart automation integration requires even smarter planning. Here are the key considerations to make sure your next facility automation and controls upgrade is a success.

Control system upgrade insights

  • Key planning considerations and typical challenges every manufacturer should know about before investing in automation upgrades and new programmable logic controller (PLC) and distributed control system (DCS) integrations.
  • Automation system upgrades are not always as simple as plug-and-play. This is especially the case in processing industries such as pulp and paper, which call for both machinery and software expertise.
  • Pre-integration planning is essential to mitigate risks, such as legacy equipment compatibility issues and downtime from workflow disruptions. Examine existing facility infrastructures and legacy system configurations and network capabilities.
Figure 1: Here are seven key planning considerations and challenges manufacturers are facing when integrating new automation technologies into an existing facility. Article provides some solutions to address them. Courtesy: Eosys

As manufacturers look for ways to become more globally competitive, many are turning to brownfield programmable logic controller (PLC) and distributed control system (DCS) integrations to enhance their existing process controls and automation capabilities. These upgrades offer long-term benefits, such as increased throughput, reduced errors and waste and optimized resource utilization for lowering labor and energy costs.

For example, many paper and pulp processing facilities have upgraded to advanced control systems to maintain proper temperature and pressure as wood chips are pulped. Robotic machinery is now automating the once-manual process of stripping logs of bark and chipping them for pulp preparation.

However, in the paper and pulp industry (and this is the case with many industries), integrating new technology into existing automation systems is not as simple as plug-and-play. Without experienced integrators focused on ensuring the seamless integration of the control systems, the machinery may not perform at optimal levels. This can have downstream consequences, such as lost productivity and wasted resources spent on repairing and maintaining poorly configured machinery.

Figure 2: Rewiring control panels, often tight quarters, can go faster with staging kits and pre-wired components. Courtesy: Eosys

Mitigating these integration risks begins with taking a hard look at existing facility infrastructures and their legacy system configurations, their network capabilities, and their ability to integrate with new technologies to make their operations more competitive.

7 ways to mitigate risk of a control system upgrade

Here are seven key planning considerations and challenges manufacturers are facing when integrating new automation technologies into an existing facility and some solutions to address them: Current control system configurations, network capability and functionality, machine safety and compliance, end-user experience, compatibility risks, downtime challenges, change management and workforce training. See more on each below.

1: Current control system configurations

While there are many potential long-term benefits from brownfield PLC/DCS upgrades, most end users are primarily concerned with ensuring their production line will run the same way as it did before the system upgrades.

This is why it is important to educate end users about any specific changes that will be required for a control system upgrade. This may require a baseline understanding of how the legacy controls equipment functioned before the upgrades. Experienced system integrators can help with this by conducting an onsite assessment.

With the paper and pulp industry example, a system configuration assessment may reveal that the drives running motors and other field devices have configuration software that is outdated. End users may have concerns about making any changes to the system configuration. Mitigating these concerns requires preemptive data collection during the onsite assessment, such as trending and charting the performance of the legacy equipment.

Once the upgrades are complete, end users can refer to the charts with the legacy equipment data to compare how the upgraded system is performing. They can then see the benefits of the configuration changes and can see for themselves that their production line is still essentially running the same way as it did before the upgrades.

Figure 3: Site risk assessments for control system upgrades may include simulations to mitigate risks. Courtesy: Eosys

2: Network capability and control system functionality

A common mistake with brownfield system integrations is assuming that the current network is ready for an upgrade. A manufacturer’s legacy equipment may be communicating over a protocol that they want to upgrade so they can put more devices on Ethernet. There is an assumption that they can “just put it on the network” and keep adding devices ad hoc.

But new automation integrations are rarely as simple as adding a device. The process requires assessing and documenting current network capabilities to determine if the network is effectively working with the facility’s current equipment configuration and if it has the capacity to scale in the future.

Thinking in terms of reliability, functionality and security, it is important to outline exactly what is expected from the operational technology (OT) network. Will the new upgrades require adding new topology and hardware to optimize plant operations? How will this impact reliability? What about security concerns? Security is more than a login. If the network is a jumbled patchwork of exceptions, there could be security risks.

These risks are the result of a lack of planning prior to a new automation integration. The planning phase should include assessing how the entire network of equipment will function with the upgraded automation system. This can prevent mistakes, such as not segmenting equipment between production lines. If two production lines are plugged into one switch, when one line is shut down for a new integration, the other line would stop running.

3: Machine safety and compliance

With new automation integrations, there is often a tendency to install the new equipment into the same safety system that was there before. However, the new system may require complex, hardwired circuitry. This could cause issues, such as the safety system failing to reset when new equipment is installed.

And there are always potential safety risks with new systems that will perform differently. For example, after installing a new control system, the machine may execute faster than the old system, because today’s control systems have newer and faster processors.

There is also a common assumption that legacy equipment is up to current safety standards. OSHA and other requirements around burner management, process safety, and machine safety have evolved.

This is why it is a good idea to have an integrator do a risk assessment before replacing obsolete equipment. Just because no one has been injured while operating the current machine, it does not mean it is a safe and OSHA-compliant piece of equipment. The assessment is also a good time to re-evaluate the need for and use of controller redundancy, safety processors, input/output (I/O) module redundancy and intrinsically safe I/O.

Figure 4: Control system upgrades requires assessing and documenting current network configuration and capabilities to determine if wiring and networks need upgrades for future requirements. Courtesy: Eosys

4: The end-user experience for using controls, automation

One of the benefits of integrating new controls automation is that it makes it easier for a new generation of manufacturing technology end users to become comfortable troubleshooting issues, such as hardwire systems that could pose safety risks. With many young workers more comfortable with iPads and iPhones than hands-on electronics and maintenance, why not make interfaces more comfortable for them?

Some legacy systems may not even have an operator interface, such as a visual screen where the end user can view diagnostics and see what is going on with their machine. Fortunately, there have been many advancements in operator interface screens, such as 3D imagery for virtual troubleshooting.

But beware of operator interfaces that are too heavy on fancy imagery and flashing lights and that are light on helpful diagnostic information, such as help locating a fault. The end user should be able to look at the screen and quickly determine if the system is running to spec.

It is also a good idea to assess the control room experience. Is the workspace ergonomic and conducive to long periods of attention? Are the monitors adequately sized with an easily readable font? A control systems upgrade offers an opportunity to review alarm rationalization and work towards an ISA-18.2 compliant system.

5: Compatibility risks for control systems

As noted earlier, very few automation upgrades and PLC/DCS integrations are as simple as plug-and-play, especially when it comes to legacy system compatibility. In the planning phase, a systems integrator will conduct a risk assessment to determine if there are compatibility risks at the machine level, the electrical level, or with the current operating system.

Machine compatibility: For example, one machine may be communicating with other pieces of equipment in the production line without the end users even knowing it. But the integrator can look at the machine’s programming to see if other equipment is writing to it, such as a paper machine making 8 ½ x 11-sized sheets of paper that is communicating with a pulping system. This communication is essential for the machine to know when to shut down, such as when it is out of stock. There may also be separate controllers or even different platforms, such as Rockwell and Siemens, that do not talk natively to each other.

Electrical compatibility: With electrical compatibility, manufacturers often want to move to a low-voltage DC. But many legacy systems are 120-volt AC. They may come to the integrator and indicate they want to replace everything with 24-volt DC controls. This is a great idea, but there will be compatibility issues if no one has done the electrical engineering to figure out if the expected voltage drop is acceptable for the end device.

 The voltage drop for 24-volt DC is a bigger issue than for an AC system. Manufacturers may have to replace all their control wiring if they make the decision to switch. These electrical compatibility risks need to be identified and assessed before the manufacturer invests in expensive sensors and IO cards that are 24-volt DC. Their wire may not be big enough to carry the lower voltage the distance they need it to go.

Operating system compatibility: In a typical factory setting, it is not unusual to have a machine still running on a Microsoft Windows XP computer or another older operating system. Or maybe the machine is running off a custom app that someone wrote.

This is why integrators will gather all the data from these computers that are running a manufacturing control system. They can then determine which software they will need to bring forward. This may require balancing IT’s desire to upgrade to a newer operating system versus the factory end users who understand that it does not take the latest and greatest version of Microsoft Windows for the machinery to run as intended.

6: Downtime challenges with automation upgrades

With any new automation integration there are always downtime risks. In a factory setting with multiple production lines that are making the same product, it is easier and faster to complete a new integration. But if there is a production line that is a feeder to other product lines, it is important to find ways to minimize that downtime.

Adding more bodies and electricians is not always a solution, especially when working with limited space, such as when it is necessary to rewire a two-by-two-foot panel. Fortunately, integrators can employ multiple methods in the planning phase to reduce downtime, such as staging kits with pre-wired panels that can quickly be installed to replace current wiring. There are multiple ways to stage new equipment and roll inputs on the fly.

It is also important to have a fallback plan that is tested and ready to go. No one ever wants to roll back, but sometimes it is necessary. Never wait until the last minute to execute a rollback. Set a timeline for the new upgrades and any necessary outages. If the task is not completed by the set date, the rollback plan will be ready to execute. Good planning can significantly reduce downtime and related costs.

Figure 5: Factory acceptance testing (FATs) and other levels of simulation during the development phase help ensure a successful control system upgrade and integration, lowering risk and avoiding surprises. Courtesy: Eosys

 7: Change management, workforce training for automation integration

A detailed change management plan is key to a successful automation integration. The plan should clearly define goals and identify employees who will be impacted by the change. What are the expectations of the new automation? Increased productivity, cost reduction or improved quality? How will the workflows change? It is important to answer all these questions before a new system upgrade.

Next comes training. Integrators can develop a formal training program that may include digital twins to simulate the upgraded system. The simulations do not have to be complicated; they could be as simple as showing an end user what to do if a valve fails to open and how to walk through a fault recovery. Or they can be more complex, such as a 3D simulated system of an entire plant for conveyor and robot performance simulations.

These training simulations can ensure quality, which is paramount in particular industries, such as food processing. With a single simulation, employees can learn how to confirm proper amounts of ingredients are added to a batch and that temperatures and other conditions are optimal.

Steps to mitigate automation integration risks

From production downtime to legacy system compatibility issues to safety and compliance concerns, there are always risks involved with brownfield automation integrations. This is why it is important to partner with an integrator who can conduct thorough site assessments, develop detailed integration plans, and create training programs and simulations to mitigate risks. Here are some additional steps seasoned integrators take to reduce risks.

Requirements gathering for the automation project

An onsite assessment is key to evaluating current control systems and machine performance and to provide a baseline for performance improvement. Requirements gathering is often the first step in this assessment, including data collection from legacy machines and working with the facility operations teams. This can help prevent downstream issues, such as a new machine not working as intended due to a problem with the legacy controls program.

For a process control application, such as a reactor, it is a good idea to pull the trends, such as how the temperature profile looked before the automation integration. And for a motion system where there are multiple axis of motion, it is helpful to have a trend of the torque profile for comparison before and after the system upgrades.

Design for automation integration

In the real world, electrical drawings are not always complete or accurate. This is why the design phase is critical to successful automation integration. This often requires going into the field to take pictures and confirm the part numbers of the actual devices that are going to be replaced.

Development, simulation for automation integration

Factory acceptance testing (FATs) and other levels of simulation during the development phase are some of the best ways to ensure a successful integration. An FAT creates opportunities to demonstrate the upgraded user experience, such as the operator interface. Including the operations team during testing is crucial to ensure they are not caught by surprise when they see a different interface screen.

It is always a good idea to test as much hardware as possible to avoid issues such as a variable frequency drive (VFD)/motor control that is dead on arrival because it was never powered on. Input and output modules should be tested to make sure they can receive signals from the field.

Commissioning plan for automation projects

After the FAT, the integrator works with the operations and maintenance teams to develop a detailed commissioning plan. This plan should include straightforward language, such as: “Here is a detailed list of activities we will perform onsite to commission the new system. Here is a copy of the current running code. Here are the steps you are going to do daily. Here is the date we need to have all these things done, and here is what we are going to roll back if we do not make that date.”

The plan should include the location of the lockout/tagout points, so there are no surprises when it is time to put the lock on the machine for powering down and replacing devices. Think about what is needed to support checkout. Where are the specific valves located? Go in and make sure the right valve is opening when the control system says the valve is open.

Site acceptance test for automation, controls

Once the commissioning plan has detailed everything required for onsite implementation, it is time to commission a site acceptance test to ensure everything is operating exactly how it was intended. This is where integrators really need support from operations. The integrators did not write the original program; they are just updating the program with new technology. There is rarely a functional description available for a piece of 20-year-old equipment.

Planning for future automation and upgrades

Not every manufacturer is ready to upgrade their facilities with new automation and control systems. However, manufacturers, processing facilities, and other industrial operations can build a plan today for future expansion. Taking a step back, what would make electrical panels easier to replace in the future for adding new automation? What can be added to each machine to make it easier for system upgrades in the future? What if the conveyor had sensors? Start planning for the future by asking these questions today. A solid systems integration partner can help guide you to find these answers and be set up for a successful automation upgrade.

Chase Davis is director of technology, Eosys. Edited by Mark T. Hoske, editor-in-chief, Control Engineering, WTWH Media, [email protected].

Keywords

Control system integration, automation upgrades, project management

Consider this

Have you planned your next automation and controls upgrade?

You also might like

Global System Integrator Report

More on control systems

https://www.controleng.com/control-systems

Written by

Chase Davis

Chase Davis, director of technology, Eosys, leads automation technology strategy at Eosys, bringing over 20 years of hands-on experience delivering modernization projects across a wide range of industries. As a contributor to the Open Process Automation Forum, Chase focuses on helping manufacturers navigate the challenges of legacy system integration and brownfield facility upgrades, ensuring reliable, future-ready automation solutions.