Software patching is vital to secure operations, but introduces more risks

End of Microsoft Windows XP support raises concerns about industrial networks, connections to PC-based assets, and software patching. Assess software patching risks with 11 critical questions. In manufacturing plant floor applications, security and safety are an integrated concern. Control system cyber security is not the same as desktop PC security.

04/30/2014


GE Measurement & Control’s Cyber Asset Protection (CAP) Testing Lab helps assess the risks and priorities of industrial software patching. Courtesy: GE Measurement & ControlIt's common to think of security updates as self-contained packages, as if the latest anti-virus or Microsoft Windows update was simply a new feature that gets added to the security stack, keeping trouble that much farther away. Yet, when it comes to patching cyber assets on industrial control systems (ICS), one needs to take a little more care than for an office or home PC.

When the office or home PC gets updated (automatically of course), it's understood that there's a possibility of unexpected consequences. Unless there is a major glitch like a lock-up, blue screen, or a primary application's malfunction, the assumption is that everything will work out for the better. In the worst case, the PC gets a reboot, and the expectation is that the next set of updates will correct the inconvenience.

In an industrial plant setting this kind of thinking and lack of awareness begs for disaster. The continuity of operations is critical. Even a minor communication hiccup or loss of view can have undesired results such as interruption of operations, or even catastrophic damage to major equipment [which can increase risk for personnel as well as production].

Regularly applying tested and validated software patches helps maintain access to plant infrastructure and provides critical cyber protection and reliability for daily operations. When operators/owners take a do-it-yourself approach to patching, they often experience unanticipated challenges and risks because of the bandwidth and resources required to properly identify and test software updates before uploading them onto the cyber assets. Manufacturer-provided patching is an excellent starting point for operators to safely execute updates and maintain operational conditions in the plant. 

Is the patch needed?

Do we really need this patch on the PCs?

Maybe! Software manufacturers continuously update, test, and retest their products to improve security and operational efficiency. Hackers continually attempt to find vulnerabilities. This combination leads to the release of updates more frequently than many operators would like to see. Yet, are all of the updates really needed by the plant? Just because a company like Microsoft, which has numerous users operating across a broad range of environments, says that a particular update is critical, it may not be the case for an individual plant's operations. In fact, while some updates may be critical for millions of users, they may be irrelevant for many others. On the other hand, a critical and timely update, for an application such as .NET, could be overlooked by a plant operator due to the lack of knowledge of the internal software functions. This is why it is beneficial for plant operators to ask their equipment manufacturers for help to identify, test, and upload patches following a systematic process. 

Assess patch risk: 11 critical questions

Assessing the relevance of a given patch can be a complex exercise. Knowledge base articles from software manufacturers that provide details on updates are generally comprehensive, and quite detailed. Questions to ask include:

  1. Are the operating systems it affects in use in your operation?
  2. If so, are the vulnerabilities it addresses active on your machines?
  3. What antivirus signature update may detect and delete a .DDL from my SCADA application?
  4. Is the system using SQL server or Internet Explorer?
  5. What about Java or Adobe?
  6. What other third-party applications are in use? (The list of third-party applications on many PCs can be longer than expected.)
  7. Will the update affect my firewall settings or host intrusion detection application (HIDS)? You may find that a patch labeled "critical" protects Windows machines using a DVD authoring app from a possible Trojan horse infection. If DVD authoring is not installed on your systems, then this is one you can live without.
  8. What are patches? Gather all patches for the computer operating system, the application, and other third-party applications.
  9. Which patches are critical? Figure out which ones are critical.
  10. How should the patches be tested? Determine how to test these patches. The cycle starts over every 30 days.
  11. What are the risks and priorities? For that critical patch from Microsoft, should it go into the standard cycle or should you just install it? The patches that pass the relevance test are the ones that will not cause any noticeable changes to the work environment and continue to provide additional protection against security threats. [What are the related operational security and safety risks?]  

Troubleshoot control system interactions

The preferred way to validate patches is to run a set of controlled tests on a representative hardware/software platform. A maintenance system or simulator typically provides an environment where a bad patch result will not interrupt plant operations. Once the patch set has passed this series of tests, the manufacturer begins an incremental installation on the actual plant control systems. This can be a tall order with many different testing environments required, depending on the heterogeneity of the installed base cyber assets.

A secure lab environment with a variety of representative equipment, various operating systems, and typical configurations provides the ideal conditions for testing patches to ensure an error-free update. For most companies, the problem of comprehensive testing before installation is the most challenging step. Securely updating a plant's software is time consuming and requires a significant level of continuous expertise. [subhead]

Selection, validation testing

Mark Hammer is a product line manager at GE Measurement & Control, responsible for developing and creating implementation procedures for control system cyber security programs in the power generation and oil and gas industries. Courtesy: GE Measurement &Many operators are required to keep systems with the most current patches and updates by regulation or company policy. For others, it is an industry best practice that is highly recommend. A good process of gathering, selection, and validation testing should be used to avoid the nightmare scenarios and even minor disruptions to plant operations. Thoroughness is the key, and patching is an essential part of ongoing maintenance to keep plant assets reliable and safe.

- Mark Hammer is a product line manager at GE Measurement & Control, responsible for developing and creating implementation procedures for control system cyber security programs in the power generation and oil and gas industries. Edited by Mark T. Hoske, content manager, CFE Media, Control Engineering, mhoske(at)cfemedia.com

ONLINE

www.controleng.com/archives May, under this headline, find additional advice, links, and resources about the end of Microsoft Windows XP support.

Control Engineering has an online cyber security training series of videos

Key concepts 

  • Company policies, regulations, and best practices can guide best practices.
  • Gathering, selection, and validation testing should be used to lower risks
  • Thorough patching process is an essential part of ongoing maintenance to keep plant assets reliable and safe.

Consider this

Price of poor patching could include unplanned outages, risk to safety, or loss of critical company assets and information.

ONLINE extra 

More about the author: Mark Hammer is a product line manager at GE Measurement & Control. He is responsible for developing and creating implementation procedures for control system cyber security programs within the power generation and oil and gas industries. He has more than 25 years of experience in the controls and automation industry with a number of leading automation and safety system vendors. He holds both a bachelor's degree in mechanical engineering and master's in business.

- See related articles below.



No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
Each year, a panel of Control Engineering editors and industry expert judges select the System Integrator of the Year Award winners.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Learn how to create value with re-use; gain productivity with lean automation and connectivity, and optimize panel design and construction.
Go deep: Automation tackles offshore oil challenges; Ethernet advice; Wireless robotics; Product exclusives; Digital edition exclusives
Lost in the gray scale? How to get effective HMIs; Best practices: Integrate old and new wireless systems; Smart software, networks; Service provider certifications
Fixing PID: Part 2: Tweaking controller strategy; Machine safety networks; Salary survey and career advice; Smart I/O architecture; Product exclusives
The Ask Control Engineering blog covers all aspects of automation, including motors, drives, sensors, motion control, machine control, and embedded systems.
Look at the basics of industrial wireless technologies, wireless concepts, wireless standards, and wireless best practices with Daniel E. Capano of Diversified Technical Services Inc.
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
This is a blog from the trenches – written by engineers who are implementing and upgrading control systems every day across every industry.
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Control Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.