10 Control System Security Threats
The North American Electric Reliability Corp. (NERC) is charged with improving the reliability and security of the bulk power system in North America, as part of a larger critical infrastructure protection mandate. To ensure uninterrupted service, NERC watches over the electric utility grid, whose systems depend on a vast network of computer-supported regulation.
Peter Welander, Control Engineering
The North American Electric Reliability Corp. (NERC) is charged with improving the reliability and security of the bulk power system in North America, as part of a larger critical infrastructure protection mandate. To ensure uninterrupted service, NERC watches over the electric utility grid, whose systems depend on a vast network of computer-supported regulation. It maintains a list of cyber security vulnerabilities that are accepted outside the utility industry as a model for protecting all industrial networking.
“ The Top 10 Vulnerabilities of Control Systems and their Associated Mitigations—2006 , is the third revision of the list,” says Scott R. Mix, CISSP, manager of situation awareness & infrastructure security. “The document is maintained by the Control System Security Working Group (CSSWG) of NERC’s Critical Infrastructure Protection Committee (CIPC), and is updated each year to reflect changes in vulnerabilities, as well as to document improved mitigation strategies,” he says. It has grown from a simple listing of vulnerabilities in 2004, to include three levels of mitigations for each of the documented vulnerabilities.
Here are the 10, with advice from industry suppliers and consultants on the importance of each to overall system strength.
1. Inadequate policies, procedures, and culture governing control system security.
Security begins with a culture and mindset of all those involved. “There is a tendency to think of security in terms of a technical solution: firewalls, passwords, etc.,” says Bob Huba, Delta V product manager for Emerson Process Management. “While those elements may cover 20% of the overall solution, common sense approaches to security implemented by plant personnel should make up the remaining 80%. To quote one industry practitioner, 'just stop doing dumb things.’ Ask the question, 'Does your facility have a security policy?’ It can be as simple as asking a stranger why he is in a control room, or making sure your users know not to bring in portable media from the outside to play music or install non-approved programs.”
Kim Fenrich, project solutions manager, power generation, ABB Inc., observes, “Without an effective security policy that addresses procedures, mitigation strategies, and periodic training, all other security programs will be less successful. To be successful, security must be viewed as an ongoing process, not a one-time investment into firewalls, intrusion prevention or detection, encryption technologies, etc.”
Operators believe, says Bryan Geraldo, lead principal, power and energy vertical, Symantec Consulting Services, that “control systems are relatively safe from opportunistic attacks or inadvertent disruption because they are 'indirectly’ connected to the Internet, or composed of different software and hardware components, some of which have the vendors’ own built-in security features.” While most IT products have built-in security measures, such as passwords and encryption options, or basic firewall/filter-type mechanisms, Geraldo says, “many of these features are deactivated—or worse—left in default or incorrect configurations, which lends a false sense of security.”
The general migration away from proprietary system architectures requires change, suggests Marilyn Guhr, senior marketing manager for lifecycle services, Honeywell Process Solutions. “As the control system environment moves to open systems,” she says, “new policies and procedures are required and often control systems people are not of aware of these requirements or they believe someone else is taking care of it. The IT organizations within their companies are very aware of these things but that awareness hasn’t necessarily filtered down to the process control area.”
Lack of knowledge produces errors. “Over and over I see mistakes occur on industrial sites that can completely invalidate the entire security effort,” says Eric Byres, CEO, Byres Security. “For example, during one particular site audit I ran, network cables were discovered that circumvented the SCADA firewalls. The reason later given was that there was no risk analysis showing that the firewalls were important, nor was there a policy stating that bypassing them was unacceptable.”
2. Inadequately designed networks with insufficient defense-in-depth.
Defense requires more than just a strong perimeter. “To secure a control system successfully requires taking a systematic and comprehensive approach,” advises Todd Stauffer, PCS 7 marketing manager, Siemens Energy & Automation. “One of the most common (and dangerous) misunderstandings is that by simply installing a control system firewall, the system is protected. This is far from correct. Instead, a layered approach called defense-in-depth is recommended by security practitioners and agencies, such as the U.S. Dept. of Homeland Security. Defense-in-depth advocates the creation of a nested security architecture whereby the plant is divided into multiple secure and closed cells (zones). Each cell must have clearly defined and monitored access points to control access and communication in and out.”
Control systems must have hierarchical levels of protection, says Kevin Staggs, global security architect, Honeywell Process Solutions. “The more critical the access, like controls and HMI, the deeper it needs to be defended. Control systems at a minimum should be firewalled off from the business network, and they should never be allowed to access the Internet. The IT realm understands how to use defense-in-depth networks, but that expertise hasn’t necessarily been brought down to the control system level.”
Byrnes says, “No IT department in its right mind would just install a firewall and then say 'we’re secure.’ IT departments install antivirus software, personal firewalls, automatic patches, etc., on every single server, desktop, and laptop, so that these computers are tough enough to defend themselves with or without the firewall. Yet in the SCADA and control systems world, companies install one firewall between the business network and the control network (if that) and completely ignore the security of mission critical devices like the PLC, RTU or DCS. The whole control security paradigm is 'crunchy on the outside and chewy in the middle’ but that doesn’t work. Like good safety design, a good security design has to offer layers of defense so that when one layer fails another will stand in its place. That means making every device on the control network secure enough that it can defend itself when the bad guys or bugs eventually get through the firewall. It isn’t easy, but it can be done.”
Security can have its downside, as Adam Stein, VP of marketing Mu Security cautions: “With SCADA-based control systems, defense-in-depth really only hardens the edges of the network. Users won’t tolerate the kind of latency that internal defense mechanisms create in a system. When the operator sends a signal to close a valve or stop a dangerous process, he doesn’t want to wait the extra time needed for that to get through multiple firewalls.”
Emerson’s Huba has seen the effects of mixed platforms common to most plants. “Many of today’s control networks are made up of loosely integrated controllers from different companies, with a common HMI interface and common off the shelf hardware for communications. Most often these are engineered by system integrators on an ad hoc basis and security may or may not have been considered. As these systems proliferate, it is important that end-users insist that proper restrictions to the control network be engineered as part of the solution.”
3. Remote access without appropriate access control.
“Controlling which persons and programs have access to the control system is critical to maintaining security,” Stauffer advises. “In general, user accounts should be set up to grant access and permission based on the defined role of the user (engineer, operator, maintenance technician, remote view-only connection, etc.). This follows the principle of minimal rights whereby users and computers are configured with the minimum set of access rights necessary to perform their role.”
But denying any remote access hampers end-users’ ability to work with control system vendors for remote services that could really be advantageous to them, including more “intellectual firepower” during a customer situation, says Guhr.
Bryan Singer, CISM, CISSP, principal consultant, industrial security, FluidIQs Inc., and chairman of ISA SP99, Manufacturing and Control Systems Security committee, says, “Terminal services, wireless networks, radio telemetry equipment, modems, and unsecured computers abound. Where electronic security is not feasible, we should have good physical security. This also extends into our ability to detect rogue or additional devices. Most networks are not managed or configured to stop unauthorized devices, so additional control systems, PCs, or even attackers’ workstations can often be joined to the network and never detected.”
4. Separate auditable administration mechanisms.
This includes system updates, user metrics, and the like that are not part of the control system implementation. “This vulnerability goes back to the people who are running the control systems,” says Guhr. “Their core competencies may not be in the IT area and the vulnerabilities that are discovered in these systems are IT-related. You need a capability in place that indicates 'what’s the latest thing you added to the system or what’s changed since the last time you had a properly running system?’ If something goes wrong, you need to know what’s changed.”
Stauffer seconds that advice: Since hackers are continuously working to find new vulnerabilities, he says, processes should monitor the control system continuously to ensure that its software is kept up-to-date.
Auditing systems and software doesn’t always come naturally to process operators, and they may need to learn new techniques. “Most process control systems and related programs are designed with alarms and event generation capability, but they are process-focused,” Singer explains. “It is very difficult to detect an attack or compromise from such logs, and computer forensic methods are also quite complicated on control devices. Some online auditing and monitoring solutions, such as intrusion detection systems, are woefully inadequate when dealing with controls protocols, and many times even if these systems and firewalls are in place, the logs are not monitored.”
5. Inadequately secured wireless communication.
“Wireless security isn’t just a big issue for control systems, but for all uses, mainly because wireless is becoming so pervasive,” says Staggs. “It’s very easy to plug wireless in almost anywhere. But you have to be able to find the signals and know if someone has put in a rogue point.
“Before installing wireless, it’s important to do a complete assessment to identify the best areas for wireless use and ensure that leakage out of the plant is minimized. Wireless leakage occurs when you have transmitters or wireless-enabled workers walking around with tablet PCs or handheld devices. Those devices may be transmitting in an area outside a plant.”
Singer encourages studying wireless propagation: “On the wireless network side, technologies such as 802.11b and g are often in place, operating in the 2.4 GHz spectrum. Often they have been deployed without a suitable site survey to determine if coverage is adequate and to evaluate if spurious emissions are limited so that people external to the facility must work hard to find these networks.”
Problems with open emission technologies fall into four basic areas, says Ken Steinberg, CEO of Savant Protection: unauthorized use, on-air interception, frequency interference, and unauthorized extension. “Security professionals need to make sure to cover all areas in order to remain secure and effective,” he adds.
Hesh Kagan, Invensys director of technology and president of the Wireless Industrial Network Association (WINA), says dangers stem from a “poorly or incorrectly managed network, as well as poor underlying technology. An insecure network will often be a fragile network as well. The lack of robustness is as troublesome to operations as the lack of security is to IT.”
Sometimes separation is the best approach, advises Symantec’s Geraldo: “If possible, segment the wireless networks from the rest of the control network. Additionally, it is strongly advisable to secure wireless access methods to include requiring authentication and enforcing strict access controls for communications leading from the wireless network into the rest of the control network.”
6. Use of a non-dedicated communications channel for command and control.
This would be the case with Internet-based SCADA. This vulnerability also could include inappropriate use of control system network bandwidth for non-control purposes, such as VoIP (voice over Internet).
“Many IT folks have bought the 'converged network’ line and think it’s OK,” says Singer. “We have seen cameras, VoIP, business systems processing payroll, and a whole host of other issues, cause denial of service conditions on control networks. IT professionals typically look at application performance, and near real time for control is a foreign concept. Taking 300-500 ms extra to receive e-mail or a Webpage is largely unnoticeable; 300-500 milliseconds for control messages or safety messages could be disastrous. Often, what is an acceptable level of saturation or utilization from an IT perspective can spell disaster for controls.”
Staggs warns that well-meaning individuals can make mistakes about infrastructure since “it costs quite a bit of money to add additional channels, and it’s hard to add infrastructure wiring after the fact. But you really need to understand where the information is and where it needs to flow, and lay out your networks accordingly. Keep the control traffic off the business network and vice versa. Don’t use the same channels. It’s really a bad practice.”
Using the control system for non-control communications, says Huba, “regardless of how much 'extra’ bandwidth appears to be available, can only lead to problems getting the mission-critical control information distributed as quickly as possible.”
7. Lack of easy tools to detect/report anomalous activity.
This includes inadequate or non-existent forensic and audit methods. “Developing a prevention approach to plant control systems will require a new approach to network security between the plant network layer and business/external systems. It’s only logical that we implement the tightest layer of control on our systems as technically possible in order to maintain the continuity of our business,” says Ernest Rakaczky, business development manager, control system security, Invensys.
The range of tools is extensive, adds Todd Nicholson, chief marketing officer, Verano Inc.: “Risk mitigation tools include perimeter protection (firewall, anti-virus, intrusion protection, content filtering, etc.), network intrusion detection (scanning the network for intrusions, rogue devices, changes in traffic levels, etc.), host intrusion detection (detecting file/process/socket changes, monitoring message queues, login failures, removable media insertion, abnormal exits, etc.), and performance monitoring.
“The unique aspects of control system designs also impact the requirements for cyber security risk mitigation. For instance, control system cyber security solutions must be totally passive, extract information from the actual control applications, monitor system performance, and operate effectively on older systems/networks.”
Inadequate methods are the problem, says Staggs. “The tools are available and are commonly deployed in business networks and IT networks, but they really are not understood or deployed on control systems. Right now, the control systems that are out there really don’t have enough of the accounting security capabilities to provide forensic trails when things do go wrong.”
Singer agrees, but isn’t sure he likes what he sees: “There are some tools starting to emerge, but they often have the flavor of 'IT-related tools,’ created by IT professionals, for IT professionals, and only for traditional IT systems , not necessarily for controls.”
8. Installation of inappropriate applications on critical host computers.
Most importantly, Stauffer says, the control system needs to safely and effectively control the process. “The only necessary applications are those that are directly involved with the control of the process. Additional software programs such as e-mail, games, and media players are not necessary and can make the system vulnerable. To harden the system, it is necessary to remove all unnecessary applications and to prevent new ones from being introduced. Unwanted programs or malware can be introduced any time data is exchanged with the world outside of the control system.”
Guhr recall, “A customer was experiencing slow downs on certain operator stations and they couldn’t figure out what was wrong. The 'problem’ was tracked down to a TV hook-up on the operator station.”
9. Inadequately scrutinized control system software.
“Some of the most common ways to compromise a system involve problems with poor coding practices, such as using static buffers, or libraries that clearly have vulnerabilities,” Singer notes. “Often, developers rely on some sort of 'tool’ to analyze source code once written, which means vulnerability detection is limited to the capabilities and patch level of the tool. Coding standards and writing secure code are available disciplines today, and should be followed. End-users, system integrators, and consultants should all insist upon rigorous application testing, viewing coding standards for vendors, etc.”
Steinberg warns that some flaws will always remain: “There is no way to remove all of the code flaws from these systems, nor create all known good and bad test cases. The best way to mitigate the potential for problems is to minimize the application set complexity, perform a rigorous review of operating system and application code, and avoid interpreted solutions when possible. Depending upon cost and time, it also makes sense to generate two application sets using two different development teams to minimize the potential for injecting the same logic flaws.”
10. Unauthenticated command and control data.
“Not all controllers out there today authenticate who’s making the change and authorize that the change is allowed for that user through the controller,” Staggs notes. “This security step on most control systems is performed at a layer in the control system above the controllers. This leaves the controllers vulnerable, and that’s why defense-in-depth is absolutely required. You’ve got to make sure the controllers are deep down in the security infrastructure, with multiple layers of defense above them. If you’re not doing that, then your controllers are basically wide open on the Web.”
Steinberg stresses people management: “When it comes to authenticating command and control, the only choice that providers have is to augment the human aspect, specifically with respect to problem analysis, chain of command, and communication flow. Proper policy, practice, and procedure will buy time for older command infrastructures to be re-thought and replaced.”
The next step
There are mitigation strategies for all these vulnerabilities, and they range from software packages to changing corporate culture. (The online version of this article includes the full text of the NERC document with three-tiered strategies for each vulnerability.) Returning to the opening comment of vulnerability no. 1, remember that technical solutions only cover 20% of the issue. The other 80% involves common sense and changing people’s behavior. That is generally the larger challenge.
Peter Welander is process industries editor. Reach him at PWelander@cfemedia.com .
NERC and the CSSWG are grateful to U.S. Department of Energy’s National SCADA Test Bed (NSTB) for developing the initial mitigation strategies. The next revision of the list, due later this year, will add details describing the vulnerabilities. Mitigation strategies mentioned here are augmented in the full document available online at