When dealing with obsolescence becomes a challenge

A system integrator upgraded a decades-old obsolete system to a current high-speed system over a weekend.

By George Bruce March 8, 2023

Obsolescence Insights

  • To upgrade and separate the equipment, five systems controlled by a central PLC needed to be migrated to either ControlLogix or CompactLogix and have all communication transitioned to Ethernet.
  • The team needed to map the three decades of piecemeal undocumented upgrades that were currently keeping systems operational.
  • The largest system consisted of a 17 slot PLC-5 rack and 12 hard wired drives, with a mix of 120V digital inputs/outputs and analog inputs.

Engineers were contracted to update and replace obsolete hardware in a pickle plant. They knew that if the one centralized PLC-5 that controlled five systems responsible for over 50% of pickle production failed, they would be shutting down the entire plant to identify and fix the issue, leading the company to lose valuable time and revenue during peak production periods.  To upgrade and separate the equipment, the five systems controlled by the central programmable logic controller (PLC) would need to be migrated and have all communication transitioned to Ethernet.

Three identified challenges

Because there were a number of hurdles to take into consideration when beginning this upgrade, it took careful planning to ensure this project could be completed without disrupting production or extending the deadline.

First, the team needed to map the three decades of piecemeal undocumented upgrades that were currently keeping systems operational.

Second, understanding the facility used only fresh produce, and at certain times ran at close to 110% production capacity. This meant upgrades had to be completed over the weekend and often tested early Monday before they began production, requiring our team to pivot quickly and mitigate any potential hiccups that might occur.

Finally, the plant itself was located in Michigan, and some of the panels in need of upgrade were outside. If shut down in winter, it could potentially damage the pumps responsible for moving hundreds of gallons of water and pickles from frigid conditions into the plant. There was a small window between cold conditions and production ramp up to execute, test and ensure operation of the modernized systems. At this point, clear communication and setting realistic expectations with the site was now critical.

Solution

Open communication with the onsite operations team was invaluable as it made it much easier to directly identify those five systems in need of audit and document the existing setup. This allowed the team to lay out the best solution for an install required over a very truncated timetable.

Courtesy: E Tech Group.

The first system consisted of 17 slot PLC-5 rack and 12 hardwired drives, with a mix of 120 V digital inputs/outputs (I/Os) and analog inputs. Example of before/after photos. Courtesy: E Tech Group 

The decision was made to tackle the conversion of the largest system first. It consisted of a 17 slot PLC-5 rack and 12 hardwired drives, with a mix of 120 V digital inputs/outputs (I/Os) and analog inputs. All of this was converted to over an Ethernet connection. NAT modules were then used to connect the PLCs to the network. This would allow the new ethernet control system to remain isolated while still enabling the controller to communicate with the network.

While the hardware installation was simple, programming proved to be a bit more challenging.  Since the original PLC-5 had to stay in place, code for the newly independent system had to be disabled in the PLC-5 once the new controller came online. Numerous messages between the PLCs had to be mapped and tested, including some with other controllers. All the IO was then remapped and as much of the process functionality as possible was tested and confirmed operational.

Courtesy: E Tech Group

Communication with the original PLC-5 still had to be set up and tested, while messaging between the PLC-5 and each of the new controllers had to be configured. Courtesy: E Tech Group

While systems 2 and 3 were smaller, they were no less complicated. Communication with the original PLC-5 still had to be set up and tested, while messaging between the PLC-5 and each of the new controllers had to be configured. Control was then disabled in the original PLC so the new ones could be tested. From here, they were able to remove the disabled logic in the original PLC-5 for system 1, as it had been running for several weeks with no issue and established confidence in the controller.

The conversion for systems 4 and 5 was similar to system 1. PLC-5 IO racks utilized hard-wired drives, and both were converted to drives over Ethernet. As in system 1, NAT modules were used to keep the systems on the plant network but remain isolated. The same cutover approach was repeated as with the first three systems where logic was disabled in the original PLC as systems 4 and 5 came online.  Messaging between the original PLC-5 and the new controllers was configured, along with communication to several other controllers. Disabled logic in the original controller for systems 2 and 3 was removed at this point. The team was then able to remotely remove disabled logic for systems 4 and 5.

Courtesy: E Tech Group

The conversion for systems 4 and 5 was similar to system 1. As in system 1, NAT modules were used to keep the systems on the plant network but remain isolated. Courtesy: E Tech Group

Project results

As with any large installation to be completed quickly, the need for troubleshooting issues can increase. Because a decades-old and obsolete system was upgraded to a current high-speed system over a weekend, there was more testing and calibrating necessary than if it were a more simple, incremental upgrade done over time.

Learning the client wanted to walk in Monday morning and resume production at 100%, our team had to manage those expectations by communicating the need to ensure functionality at all stages of production. Instead, they would need to begin at 50% and increase gradually to 100%. This would allow for an easier shut down to solve any potential bugs or hiccups that may arise before reaching full production.

The team’s clear communication and ability to manage the client’s expectations, coupled with the expert work of our engineers, has opened the door to a multi-year project with the potential for future work.

George Bruce is group engineering manager at E Tech Group. Edited by David Miller, Content Manager, Control Engineering, CFE Media and Technology, dmiller@cfemedia.com.