Securing Legacy Control Systems
Very few of the process control platforms operating today were installed with any cyber security protection built in. Most predate wide deployment of the Internet. Can these systems be protected against today’s threats?
Peter Welander, Control Engineering -- Control Engineering, 7/1/2009
|
Cyber security issues have taken center stage over the last few years, and their visibility seems to increase almost on a daily basis. New IT and industrial control system platforms are incorporating vastly improved security functions, but the problem for industry is that huge numbers of control systems predate these cyber security efforts, and many even predate large-scale deployment of 
the Internet. Connections to those systems from the outside provide the means for hackers to get in and do all sorts of damage, unless barriers are put in place to keep them out. The question is, can old systems be adequately protected?
“Traditionally, a control system was purchased, engineered, configured, implemented, and pretty much forgotten about for the next 20 or so years,” says Ernie Rakaczky, principal security architect for Invensys Process Systems. “In many ways that’s still the mindset of the control community, but a lot has happened in the last five or 10 years. Many of those early systems had no capability to be connected to anything else. They were designed to do one thing in life: they open and close a valve or they measure a level, so the cyber world really doesn’t play a role.”
The problem is, the world didn’t stay that way. With the growth of information technology in general, the desire to extract information and to provide connectivity to outside users grew too. Rakaczky adds, “At one point, a traditional control system was 100% focused on controlling a process. Now it’s probably 50% control and 50% information exchange. When that started to happen, and we started to move data off the control platform to a historian or plant network or enterprise network, it was time to start adding some best practices, being more cautious, and putting some functionality in to protect yourselves.”
Safe to connect?
Some experts take the position that the only truly safe connection is no connection. “In almost all cases, proprietary systems had proprietary networks,” says Kevin Staggs, CISSP, engineering fellow, global security architect, Honeywell Process Solutions. “Adding any connectivity puts them at a significant risk, because they’re not even designed to protect themselves, and there is nothing to allow them to be protected. For those old systems, you must absolutely understand and know where those connection points are, and know what technology is used for those connection points. In a lot of cases those points will be historians.”
But if maintaining an air gap is your main line of defense, the isolation needs to be absolute to be effective. “People tend to believe that isolated networks are secure, and put all their eggs in one basket, believing that because it’s isolated it’s secure,” says Todd Stauffer, manager of process automation systems for Siemens Energy & Automation. “But there are many scenarios where people have an isolated network, but somebody brought in a memory stick from outside, or temporarily connected a laptop, or temporarily connected a network, and really messed up that isolated network. Unless you have the practices in place to make sure people don’t bring memory sticks, CDs, or DVDs into an isolated area, then you need to have security on this isolated network or you’re going to be very vulnerable when that does happen.”
Understanding multiple generations
Old control platforms cover a long time span, with some still running that date back to the earliest DCS deployments. For all practical purposes, they can be divided into two broad categories: Proprietary networks and Microsoft Windows-based architecture.
Some that operate older systems comfort themselves believing that even if a hacker does break in, he or she simply will not know what to do with that obsolete technology. But is that really a protection strategy? John Cusimano, senior consultant for Exida likens that strategy to skating on thin ice. “If someone gets into one of the older systems, and they have knowledge of the system in terms of what the command structure is, I think it would be quite easy to violate it,” he advises. “The distinction is that the intruder has to have a higher skill set to be able to violate an older system. But once you have access to it, you could probably execute any command you wanted.”
The problem is that the population of potential qualified hackers has probably grown a great deal in the last year or so. Ken Pappas, security strategist for intrusion prevention system provider Top Layer warns, “The fear today, because the economy is what it is with companies laying off people in the thousands, they’re now dealing with disgruntled employees that already have inside knowledge as to how these systems work. So if you put one-plus-one together, these people know how to get in the network, and they know what to do when they’re in there, so they can cause a lot more havoc than the average hacker who just knows how to break into a network. It’s not that the hackers got smarter, it’s that we have a new class of hackers in post-workers.”
Some users believe that once Windows moved into the plant in the 1990s, that it helped provide a more secure environment. Cusimano says this is not the case, but he “worries mostly about the systems that came out roughly 15 years ago, when Windows was first getting introduced into the control system world. There are systems still out there running Windows NT-based HMIs. That was the generation where systems were first starting to go open, but used those older versions of Windows where security was pretty much nonexistent. That’s a generation in particular that needs to be looked at.”
Staggs agrees, and advises, “Older Windows systems should not be connected to the business network. They absolutely must be compartmentalized, and you must understand what traffic has to flow from them into the business network. You must have a very tightly configured firewall, and if you can, you should flow that information through a more modern server. You can protect the more modern server, and it serves as a bastion device. Most of the time it’s a historian, so get those upgraded and make them the most modern technology.”
Old in this context means any Windows generation that is no longer supported. Windows 2000 is on extended support through June 30, 2010. Anything earlier than that falls into the unprotected category and will have no more security updates.
What can you do?
“Security by obscurity doesn’t apply in the modern, interconnected situation,” warns Sean McGurk, director of the control systems security program (CSSP) for the U.S. Department of Homeland Security (DHS). “We’ve done analysis of vulnerabilities at facilities for years now, and the vast majority of those vulnerabilities, about 46% of them, are in the DMZ (demilitarized zone) between the process control network and the business network. That’s an unacceptably high ratio when you think that’s the area where connectivity is most important. You need to look at ways to lock down those communication channels.”
True enough, but easier said than done. Protecting something that was never intended to be protected or that has passed out of manufacturer support is not easy. Todd Nicholson, chief marketing officer for Industrial Defender explains, “The challenge with this environment is that everybody certainly has a need for a secure perimeter, especially if you’re connecting to the outside world, but the legacy nature of this environment—hardware, software, operating systems—is challenged because these environments cannot utilize modern security technology like anti-virus at the plant system level without dramatically impacting performance and availability. You have a very fragile environment, and you have to think about that when laying out your defense strategy.”
Nevertheless, there are strategies. Going into extensive detail is beyond the scope of this article, however there are resources to help start the process in the sidebar. Most strategies begin by analyzing your current architecture in detail, and cataloging all software running on your control networks. Finding all the outside connections is another major step, and is often an eye-opening experience for companies that have lost track over the years. Staggs advises beginning with your historians and upgrading those, but don’t stop there in your search for connections. He suggests, “To find potential connection points, you should be looking for OPC, serial gateways, modems, safety system connectivity, Modbus serial or IP, and even Foundation Fieldbus or Profibus.”
Time to migrate
Are cyber security concerns enough to drive a migration? Ultimately, the answer may be to migrate to a newer platform that has a higher level of protection. Of course that’s no small task, particularly during an economic downturn. “I think most of the engineers that are actually running the system understand the risk, but whether they can quantify the risk and give it as a reason to migrate is another issue,” says Ken Keiser, migration marketing manager for Siemens Energy & Automation.
“I don’t know if they can articulate it to management. It’s easier to say, 'It doesn’t work any more and it will affect production,’ rather than try to articulate that it’s a security issue—although they do realize that risk,” Keiser says. “It helps that people tend to want to upgrade one of the most vulnerable parts of the system first, and that’s the HMI. It’s easier to connect to a controller from the top down, rather than trying to attach leads to a remote instrument and send it the other way.”
J.T. Keating, vice president of marketing for CoreTrace, sees migration as too slow an approach. He advises, “While cyber-security concerns are definitely an appropriate rationale for upgrading older systems, it’s usually unrealistic to contemplate replacing these systems, at least in any practical timeframe. Security means now, replacing means next year. Since these are often critical systems, replacing them must be a carefully crafted process, and in 2009, fiduciary requirements often mean making do with what you have, instead of ripping and replacing what could amount to large sections of infrastructure. Across the energy sector the sentiment is to 'make do with less.’ Hardly the ideal for increasing the security posture of these critical systems, but that is the reality.”
Will federal regulation force a large scale change? What about all the new NERC (National Electric Reliability Corporation) requirements? McGurk points out that 80% of critical infrastructure is in the private sector, and even NERC only covers a relatively small portion of the larger energy industry. He acknowledges a sad reality, “Frankly, there’s been the mindset for quite a while to find ways to avoid compliance as opposed to recognizing the need for security.”
The human element
So far the discussion has largely been about technical solutions. But there are human elements as well. Keeping a system secure requires that your people participate in the process. One people-related problem common with older systems is a lack of documentation. After a system has been in place for a decade or more, like other parts of the process, there may not be current documentation that reflects what is actually operating. It’s common to find vulnerabilities resulting from undocumented system changes.
Nicholson observes, “Not only in the plant environment but in enterprise IT, change management is a very large challenge. For example, when you open up a port to allow access for someone from the outside, how do you effectively track and monitor the changes to the system? Someone might forget about having a port open here or there that exposes the system.”
Matt Luallen, co-founder of Encari and Control Engineering cyber security blogger, stresses the importance of procedures when dealing with problems. He says, “Effective information security programs must consist of sufficient and effective procedures for handling incidents.” These include event identification, containment, root cause identification, eradication, recurrence prevention, defined call trees, tie-ins into change management documentation to identify approved and unapproved changes, and the appropriate escalation and reporting procedures, he says.
“We typically see modifications to control system software and hardware go undocumented, unnoticed, and subsequently not centrally approved,” continues Luallen. “This in its own right is a security violation; however, this is a scenario that is not easily solved solely through the use of technology. In most cases, the PLCs, RTUs, relays and other control system hardware and software do not have a capability to ensure an approval process for control modifications.”
Procedures are important, but people have to understand their role in keeping the plant safe. The DHS reports that social engineering is one of the biggest attack vectors. McGurk laments, “How often do we see vulnerabilities and exploits that are conducted as a result of poor operational practices because people don’t understand the need for security.”
Marty Edwards, Idaho National Laboratory DHS CSSP manager, outlines the kind of cultural change that needs to happen: “One of the biggest challenges we have in security—whether it’s in control systems, or IT, or physical security—is creating that security culture, and you can do that regardless of the vintage of the equipment that you have. It’s your personnel. It’s your training. It’s the culture that they operate in.”
From a safety perspective, industrial and processing areas have had that culture for some time, says Edwards. “You don’t do anything in a plant without thinking about what the safety ramifications are,” he says. “We must instill that same culture, so that before I do anything, I think about the security ramifications. Should I post a network drawing at a user group conference that contains all the most intimate details of our control system? That’s a change that everybody can make immediately, and it costs a lot less than replacing equipment.”
View All Control Engineering Webcasts from CTL
