Get the latest updates on the Coronavirus impact on engineers.Click Here
Machinery and Equipment

Repair or replace: Things to consider in industrial programming

Industrial programming is a challenging project even under the best of circumstances. Understanding the situation can help determine whether a project needs to be repaired or overhauled.
By David Breen March 25, 2020
Courtesy: Breen Machine Automation Services, LLC

I had a recent project where the work was already 95% done before I ever got there. Problem was it was over budget, kept getting delayed, and the original team of developers no longer wanted to work on it. I was called in to provide an estimate to complete the work. 

What I encountered was one of the messiest programs I have ever faced. The hardware was a relatively straightforward conveyor system; a very repetitive series of components. The code was a collection of ladder routines, one for each individual conveyor, of which the logic was mostly copy/pasted with tags renamed. 

Doesn’t sound so bad yet, right? This is where the details of the remaining work become relevant. Each conveyor routine needed to be customized for its expected function, so each routine was a little different. I had to figure out why the existing implementation failed in 1% of situations. This amounted to 100 conveyors, 100 routines and 50 rungs each of poorly constructed, repetitive and error-prone logic. 

This is a common problem in industrial programming: maintainability. Unless you’re already familiar with the code, it can take hours to get a feel for and understand before you’re even ready to make changes. The limitations of organization and structure common in industrial programming make larger projects like this messy. 

Courtesy: Breen Machine Automation Services, LLC

Courtesy: Breen Machine Automation Services, LLC

A similar issue related to maintainability is scalability. One of the planned upgrades for this project was to double the size of the system. Maybe triple. In the current implementation, that potentially meant 200 new routines. Sure, copy/paste will get you halfway there, but then you have to edit every single tag. This code is not wellsuited to expansion. 

To summarize: The code doesn’t scale well and isn’t easily maintainable by the next programmer (you), and it doesn’t entirely work. However, there’s already months of labor from several people to get this far, which indicates a significant effort required for a complete rewrite. 

So what do we do? Debugging and improving the current code maybe a little faster than redoing it. It’s not a guarantee given the scope of the changes but it’s likely. However, that still leaves us with a messy result that isn’t going to cooperate well with the future expansion. 

There’s a useful quote from John Woods, an old school PC programmer, that’s very appropriate here (and in every situation where you find yourself writing code): “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” 

Everything as it is, we have an obligation to redo this. If the problems needed to go away overnightwe might find ourselves in a different situation, but this is going to require significant effort regardless of the decision. Might as well do it right and leave things better for the next guy, psychopath or not. 

We’ve considered a lot of different factors here to reach this decision. Another time, another place, we may find ourselves making different choices. The important thing is to be aware of the pros and cons of each option and be able to justify our actions. 

I did end up keeping a lot of stuff that was better off unchanged. The hardware was hooked up well, so the I/O was generally left alone. The safety program was functioning well enough and not so complicated that it was beyond understanding – no need to tinker. The logic to control auto vs. manual control was all fine. Even though I chose to redo the core conveyor control doesn’t mean we can’t find plenty of rungs worth keeping. Remember the old adage, “if it ain’t broke, don’t fix it.” 

For this specific situation, the decision to rewrite was successful. Three thousand rungs of ladder were replaced with about 500. It was cleaner, more maintainable, and still did everything it was supposed to. It took almost three weeks to redo everythingbut that includes most of the work for the future expansion, as well. By changing just a couple of tags the whole system iready to double in size. 

Personally, I feel the flexibility upgrade is worth it almost every time. Your clients shouldn’t depend on you specifically when they need help because you’ve written unreadable code. Unless the effort is too much or the timeline is unacceptable, most projects could stand to be redone at some point. You can only slap on so many patches before it’s time to do them over the right way. 

This article originally appeared on Breen Machine Automation Services’ blog. Breen Machine Automation Services LLC is a CFE Media content partner. Edited by Chris Vavra, associate editor, Control Engineering, CFE Media and Technology, cvavra@cfemedia.com 

MORE ANSWERS

Keywords: industrial programming, ladder logic, machine programming 

Maintainability can be a problem with programming industrial machines. 

In some cases, it’s better to repair the program in the short-term; sometimes a complete overhaul is needed. 

Unless the costs are too much, going with an overhaul is often best. 

CONSIDER THIS  

What is the biggest consideration you have when deciding to repair or replace for an industrial project?


David Breen
Author Bio: David Breen, lead programmer at Breen Machine Automation Services, LLC.