Cyber Security Lessons from Electric Utilities Industry
Global economic commerce has become almost wholly dependent upon constant, reliable availability of electricity. This is nowhere more true than in the interconnected Internet world, which drives global commerce and has become deeply enmeshed in modern society. Paradoxically, the capabilities of an interconnected world have created a great vulnerability point in ensuring uninterrupted flow of el...
Global economic commerce has become almost wholly dependent upon constant, reliable availability of electricity. This is nowhere more true than in the interconnected Internet world, which drives global commerce and has become deeply enmeshed in modern society. Paradoxically, the capabilities of an interconnected world have created a great vulnerability point in ensuring uninterrupted flow of electric power, and have raised
Protecting legacy process control systems, in Archives or July 2009 at www.controleng.com . Also:
security of the utility industry’s critical assets to a level of paramount concern. Assets that have a dialup or IP connection—including SCADA, HMI, and other control systems—became subject to new rules this year that aim to lessen the threat.
As of June 30, 2009, all high voltage electric transmission and distribution (T&D) operators in the bulk electric system have to be in compliance with regulations specified by version 2 of the North American Electric Reliability Council’s Cyber Security Standard (NERC CIP). And they must begin collecting and logging data to become auditably compliant by July 1, 2010. The power generation owner and operator deadlines follow these dates by six months. This is a major milestone in moving toward securing critical cyber assets (CCAs) in the electric utility infrastructure of North America. It also offers lessons for other industries.
In general, the NERC CIP regulations have tasked operators to: comprehensively identify CCAs; develop security management controls; have personnel training in place; have detection and prevention measures in force; and have response plan, and notification and recovery procedures spelled out.
There is considerable room for interpretation, and hard clarity awaits evolving versions of the standard and the cold reality of audit assessments by Regional Reliability Councils, which include fines for non-compliance as high as $1 million a day for each infraction. There is little question, however, that the impact on SCADA, HMIs, and other facility control systems will be profound.
“Anyone in charge of a SCADA system needs to step back and ask themselves what would happen if someone did get into their system,” says Eric Knight, senior knowledge engineer for LogRhythm, which offers a software solution for mandated logging activities.
“And operators need to understand that there is more to protection than just technology,” says Walter Sikora, vice president of Industrial Defender, a provider of cyber protection solutions. “There are a lot of human factors involved. The NERC CIP regulations effectively touch all elements of the organization—from
‘Operators need to understand that there is a lot more to protection than just technology.’ —Walter Sikora, Industrial Defender
operations to HR, maintenance, procurement and legal. You need to understand that compliance is not an end point—but an ongoing process.”
The potential consequence of breached control system security is dire. Says Ron Blume, vice president of Dyonyx, a Houston-based infrastructure consultancy: “If someone could get into an electric utility provider’s control system, they could take down the grid.”
In addition to a strong physical security perimeter (such as a locked control room) and electronic security perimeter (such as firewall, malware/virus detection/prevention), operators must have documented security management control policies and procedures in place; and a robust means of securing, monitoring, and controlling access to CCAs.
“Each responsible entity has to start with having a good cyber policy in place. Outside of IT, this hasn’t existed before,” says Roger Pan, Ovation Security Program manager for Emerson Process Management. “It can be viewed as a pain in the neck, but it’s just good business.”
It all comes down to mitigating or eliminating the threat envelop. “The risk equation is a factor of threat times the probability of attack,” says Blume. “You have to assume you’ll be attacked. If you don’t have a firewall, the risk is high. But with multi-layered defense, and having a DMZ [a demilitarized zone, also known as a data management zone], although your threat might be high, the probability is low.”
You can also reduce “the threat envelop by reducing what you’ve exposed to the network,” says Paul Henry, security and forensic analyst for Lumension, an endpoint security company . “If you have ports open that aren’t needed, you’ve increased your risk.” Almost all servers that run SCADA today have USB ports, and flash drives have become ubiquitous. “So there has to be a strong policy about what devices get plugged into [servers],” says Henry.
NERC CIP Reliability Standard CIP-005 requires that all critical cyber assets, including SCADA, reside within an electronic security perimeter (ESP). Access to the ESP is securely controlled and monitored. Source: Industrial Defender
Access has to be tightly monitored, authenticated and controlled. “In the past, authentication was based solely on the belief you had a trusted operator,” says Matt Luallen, cofounder of Encari, a cyber security consultancy. “And if you received a communication on the network to execute a procedure, you trusted it came from a reliable source.”
Windows provides a certain level of access control, but you need to look beyond simple Windows-based password control. “You have to decide how granular you want to be, based on the makeup and structure of the organization,” says Todd Davis, business development manager for Schneider Electric. “If you have multiple operators in a control room, each with differing levels of authorization, do you want to control it at the workstation level, or screen by screen—or even more granularly, at the object level within each screen in the HMI. Most SCADA systems enable a combination of these, but the system administrator needs to put them in place.”
Additionally, administrators may want to add three-point authentication, which might include a password, some personal information known only to the individual, and perhaps some form of biometrics—which can be added for about $100, Davis says.
Henry of Lumension also encourages that policies enforce a “rule of least privilege. Simply stated, any users on the system should be granted minimum authorization required for them to get their job done.” Access authorization, however, also must consider maintenance, engineering service, and vendor support technicians. All should be identified by name rather than job class, and HR has to be involved with certifying that they all have met training policy requirements and have proper background checks.
System maintenance, patch management and configuration changes are also areas of security concern with regards to SCADA and HMIs. Many substation devices were never really meant to be connected and have no real TCP/IP error checking capabilities, such that it’s possible for someone running a system scan to cause the network to crash. Pre- and post-patch and -configuration tests are also warranted to maintain security and meet compliance requirements.
Though the standards are not as definitive for what constitutes compliance, Dale Peterson, president of Digital Bond, a control system security research and consulting practice, says clarity will come in time. “It’s like Sarbanes-Oxley—it took time for clarity. It’s important, however, that you’ve acted in good faith and can defend the decisions you’ve made.”
John Shaw, executive vice president of GarrettCom, an industrial networks product company, reduces NERC CIP to a core set of practices. “Identify all critical cyber assets that could affect operations. Then identify who needs to be able to reach them. Get rid of all unnecessary open ports. Keep track of who has authorization. Log and keep track of records of everything.”
“Compliance is determined by looking at records you keep about security decisions,” Shaw adds. “and by the overall state of network security. In theory, you can be fined up to a million dollars a day if you’re not in compliance. That’s enough money to get the attention of any company.”
Frank O Smith is a contributing writer to Control Engineering North America.
Keep up to date on NERC CIP-004-1, R1 security issues
Control Engineering ’s Industrial Cyber Security blog provides regular updates and advice to help DCS/SCADA systems engineers be aware of security vulnerabilities and ways to respond. Bloggers Matt Luallen and Steve Hamburg, through their NERC CIP compliance consulting firm Encari, are also providing specific NERC CIP-004-1, R1 materials that can help with compliance.
Encari is providing quarterly security awareness Webinars focusing on challenges commonly encountered at electric power market organizations. The first, held in July, addressed both security best practices and recent incidents and regulatory developments. Also, beginning in August, Encari will email bi-monthly security awareness bulletins that can be distributed to employees, contractors, and peers. Topics addressed will include proven information on security best practices, and recent incidents and regulatory updates.
“The first requirement of the NERC CIP Reliability Standard CIP-004-1 succinctly states: Your organization needs to 'document, implement, and maintain a security awareness program to ensure personnel having authorized cyber or authorized unescorted physical access receive on-going reinforcement in sound security practices,’” says Luallen. “These no-cost materials cover two of the most critical elements of required security awareness programs.”