A software patchwork quilt

Computer system patching is in the news again as numerous attacks target unpatched operating systems and applications. Software patches are an unfortunate common practice with any major O/S, database, or system service application. A software attack can use any known vulnerability and can occur within days, even hours, of a vulnerability being discovered.

IT organizations use several methods to protect against these attacks, including network firewalls, personnel firewalls, and intrusion detection systems, but one of the most important is patching vulnerable systems and applications. Many IT organizations have fulltime staff in a Patch and Vulnerability Group (PVG) dedicated to testing new patches in the corporate environment, checking the patch level of all systems and applications, and ensuring that all systems are patched to the approved level.

Two complementary documents provide an excellent starting point for understanding the issues of patch management processes. The British National Infrastructure Security Coordinate Centre (NISCC) has released the “Good Practice Guide for Patch Management,” available at www.cpni.gov.uk/Products/guidelines.aspx . The NISCC report describes a recommended patch management process that has four stages: assessment and inventory; patch identification; evaluation, planning and testing; and patch deployment. The NISCC process can be applied to any type of organization.

The National Institute of Standards and Technology (NIST) has released a report on “Creating a Patch and Vulnerability Management Program,” available at csrc.nist.gov/publications/nistpubs/800-40-Ver2/SP800-40v2.pdf . The NIST report describes patch and vulnerability management processes, patch and vulnerability metrics, management issues, and U.S.-government sponsored resources. The NIST report provides guidance for U.S. agencies, but the processes, metrics, and resources also apply to manufacturing organizations. An important part of both the NIST and the NISCC reports is the estimate of patch management costs compared to the cost of repairing compromised systems. Most economic studies show that the money spent on continual patch management is small compared to the cost of repairing compromised systems after a successful attack.

Neither guideline specifically addresses the problems of patch management in the automation environment. Automation systems often have failure modes that can cause equipment damage, product loss, injury, and even death. Because of these characteristics, the normal IT processes for patch management are not sufficient for automation.

The NIST report does mention that some systems must be handled separately because of their criticality, and many automation systems fall into this category. The continual patching processes described in the NIST and NISCC reports must be modified to handle applications that cannot be shut down without major safety or quality issues. These systems require a test environment that allows the PVG to test patches in as near a realistic situation as possible. Because confidence testing can take a significant amount of time for a large automation system, the patches tested and applied may be weeks or even months old. But even with this delay, there is a significant economic advantage to patching systems.

Patch management is an ongoing process and manufacturing IT organizations must be involved in the process and in the corporate PVG organization. The PVG will be responsible for patching operating systems, databases, network services, and application serves that are used by business applications and automation applications. Your manufacturing IT organization should ensure that the patching processes for the critical automation systems and applications have realistic test environments and sufficient time to perform confidence tests, and are scheduled into the production environment. Finally, production operations should consider patch updates for automation systems as a preventive maintenance activity and should plan and schedule the updates the same way they do other preventative maintenance activities.

Author Information

Dennis Brandl is president of BR&L Consulting in Cary, NC, dbrandl@brconsulting.com

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.