Real World Engineering

This is a blog from the trenches—written by engineers at Maverick Technologies who are implementing and upgrading control systems every day across every industry. This isn’t what they teach you in engineering school. These are lessons learned from years on the job, encountering the obstacles and issues that are part of the real world of control and process engineering.

Real World Engineering

When worlds collide

Does a meeting of the IT group and the controls group always have to result in a collision? Can we avoid the broken glass?

November 06, 2012


Recently I attended a session with a panel of security experts who were discussing process control system security. There were quite a number of surprising revelations from this gathering including the need to ensure that your facility’s control system isn’t found by a search engine called Shodan. It may sound like the villain of some cyberpunk novels, but can be a real threat to the security of your control system. 

I’d never heard of Shodan, so I was more than a bit taken aback that something like this existed and was regularly being used by hackers to attack industrial control systems. Hearing that the number of systems that can be found using this search engine numbers in the thousands was even more of a shock.

Having worked in this business for a very long time, I know that most of the installed control systems that I’ve been in contact with still have the default administrator’s password in place, so hearing that everyone should go back to their plants and do an audit of this vulnerability was old news. I was surprised to hear from one attendee that in his plant, all of the USB ports of the PCs used in the control system had been sealed with superglue. I’ve never seen even the most draconian IT person suggest that as a security measure.

Which brings me to the topic of this blog: the collisions that occur between the control system group and the IT group.

More often than not, when I am talking to the process controls group about expanding their controls infrastructure, their first question is if we can do it without having to engage the IT people. While they may react that way initially, it doesn't take long for them to realize that such an approach simply isn't practical. It's more important to work through conflicts over issues like security and reliability.

When I talk to the IT people, usually I hear complaints about how uninformed the process control people are about the latest technologies. The folks running the plant get especially concerned when the control system and the corporate systems have to communicate. The concern deepens when some manager wants to view control system data remotely.

A lot of the conflict comes from the need to protect the systems while still making it easy for operators to use. Unless you are in a heavily regulated industry like biotech, your plant probably has generic user names and passwords for functions like operators and technicians. You also have turned off the auto-logout feature and don’t have a requirement that passwords expire. God forbid that operators should have to log in when they start their shift. If you do have individual user names and passwords, they are probably tied to your corporate active directory so if anyone is successful in hacking into that, they automatically have access to the control system. Remote access to the system is probably protected by a VPN if you’re off site, but if you’re on site, you skip that. Consequently, if someone hacks the corporate network, they’ve hacked your control network. You probably haven’t patched your software recently because as the system manager for the control system, you’re also most likely doing a lot of other things that have a higher priority, at least they’re higher priority in your mind. After all, the control system isn’t open to the Internet and even if it is, IT should be guarding that door.

So what’s a control system manager to do to minimize the risks to his or her system? You can start by learning about ICS-CERT and what it has to offer to help you. It even offers free training via the Control Systems Security Program (CSSP). (The training is free, but travel and living costs are on you.) This includes a week long advanced cyber security course that ends with a 12-hour exercise pitting the students against hackers. Best of all, the director of this section within Homeland Security is headed by a former process controls engineer, not some bureaucrat who doesn’t have a clue about what we do. You should also inquire of your control system suppliers if they comply with the Achilles Assurance Platform guidelines or have Achilles Certifications. Finally, you need work through the differences between control systems and IT people so they understand why not all good IT policies are good control system policies and vice versa. This last recommendation has to take place at a high enough level that the decisions made get implemented. 

How has your company coped with this digital divide? Have you and IT made peace? If so how did you do it?

This post was written by Bruce Brandt. Bruce is the DeltaV technology leader at MAVERICK Technologies, a leading system integrator providing industrial automation, operational support and control systems engineering services in the manufacturing and process industries. MAVERICK delivers expertise and consulting in a wide variety of areas including industrial automation controls, distributed control systems, manufacturing execution systems, operational strategy, and business process optimization. The company provides a full range of automation and controls services – ranging from PID controller tuning and HMI programming to serving as a main automation contractor. Additionally MAVERICK offers industrial and technical staffing services, placing on-site automation, instrumentation and controls engineers.