9 ways to avoid crisis upgrades to automation IT

Engineering and IT Insight: No one may want to claim support for old automation or IT systems. When systems are old and without support, documentation, or labels, it may become a matter of: If you touch IT, you own IT. See 9 practical ways to avoid crisis upgrades.


Here's how to avoid crisis upgrades related to unsupported automation IT legacy systems that may be without documentation or anyone who remembers anything about them.

We recently had to fix a number of 12-year-old routers that were used for building automation, utility data collection, and alert notifications to maintenance and quality staff. This system fell between the cracks of automation IT and business IT support. It was standard business IT hardware, but the applications were not standard business applications and some were integrated with the site's automation and utility systems. It was not a production critical system but was a critical support system. The system was in sad shape with intermittent memory problems and corroded connections because of the router's locations outside of a controlled IT environment. When the system finally failed, it became a situation of, "If you touch it, then you own it." 

No router documentation

No one wanted to touch the system; the interface was an RS-232 connection using a command line interface that no one understood. None of the IT staff even had a system that had an RS-232 interface. There was no documentation on the router configuration, and no one who worked on it 12 years ago could be found to help. There were no working backups of the configuration, no installation documentation, and an untested and unworkable disaster recovery plan. It was originally installed by the automation group, but no one remembered why they were involved or who was even in that group at the time. Because an automation professional was involved in the original installation, this became a case of "because you touched the IT, now you own the IT." After multiple attempts to fix the old routers, they were finally replaced with four new switches that support a web interface, with new installation documentation, and with a tested disaster recovery plan.

In retrospect, the automation group was the best one equipped to handle the situation. Automation groups regularly handle systems that are 10, 20, or even 30 years old. Automation groups regularly have to maintain internal documentation on MULES (Mature Unsupported Legacy Execution Systems). Automation groups often have experience in text-based and direct-connect interfaces that were common in the 1980s through the early 2000s. Automation groups often have to deal with production and production support critical systems and coordinate with the production, quality, and maintenance groups to schedule support activities.

Inherited systems

Similar situations where automation IT assets are difficult to support will become more likely in the future. Systems developed in the mid-1980s and later are heavily based on commercial hardware and software, which will no longer be supported by the business IT functions. The business IT group will focus on hardware and software that is usually no older than 10 years, while automation IT will have to support systems much older. As older production and production support systems roll off business IT support, they will roll onto automation IT support.

9 ways to avoid crisis upgrades

There are some simple tasks the automation IT group can do to handle replacements and upgrades in a less "crisis-driven" manner.

1. Ensure that every system is identified and labeled (with a permanent marker, please; pencil lines disappear in 10 years or less).

2. Include on the label which department is responsible for support, and the cycle time for checking that status of the equipment.

3. Don't assume the departments remain the same over 10 years, so be prepared to update the labels.

4. Perform an annual review of all owned devices, making sure the installation, disaster recovery, and backup documentation is available.

5. If you don't have a formal tracking system, just set up an Outlook calendar with the review dates and other information on each review event.

6. Plan for the long-term in information management.

7. Don't rely on the same servers being available in 5 to 10 years.

8. Don't assume that floppy disk, CD, or even DVD readers will be readily available in 10 to 20 years.

9. Assume that the end users of the system will know nothing about the system; to them it is plumbing, hidden behind walls and only interesting if it breaks.

If you will become the group that supports MULES, it is best to be prepared for a completely different IT environment. We can't predict the future environment, but if history is any indication, it will be wildly different from today's web-based, Ethernet everywhere, server-based environment.

- Dennis Brandl is president of BR&L Consulting in Cary, N.C. His firm focuses on manufacturing IT. Edited by Mark T. Hoske, content manager, Control Engineering, mhoske@cfemedia.com.

ONLINE extra

This posted version contains more information than the print / digital edition issue of Control Engineering.

At www.controleng.com, search Brandl for more on related topics.

See other articles for 2015 at www.controleng.com/archive.

See other Manufacturing IT articles

See the Information Control newsletter

No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Additive manufacturing benefits; HMI and sensor tips; System integrator advice; Innovations from the industry
Robotic safety, collaboration, standards; DCS migration tips; IT/OT convergence; 2017 Control Engineering Salary and Career Survey
Integrated mobility; Artificial intelligence; Predictive motion control; Sensors and control system inputs; Asset Management; Cybersecurity
Featured articles highlight technologies that enable the Industrial Internet of Things, IIoT-related products and strategies to get data more easily to the user.
This article collection contains several articles on how automation and controls are helping human-machine interface (HMI) hardware and software advance.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Mobility as the means to offshore innovation; Preventing another Deepwater Horizon; ROVs as subsea robots; SCADA and the radio spectrum
Future of oil and gas projects; Reservoir models; The importance of SCADA to oil and gas
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me