Red/blue training brings ICS cyber security weaknesses to light

Cyber security war games can provide insight into the techniques that bad guys are using against your control systems and networks.

By Bruce Billedeaux, PE April 23, 2013

As a follow up to last week’s discussion on  five industrial control system cyber security mistakes, I want to talk about a cyber security exercise I participated in at the ICS Cyber security (301) Training I attended recently. The five-day event featured hands-on training in cyber security and the week concluded with a red team / blue team exercise that took place within an actual control system environment.

The 12-hour exercise had participants either attacking (red team) or defending (blue team). My team, the blue team, was assigned a three-fold task:

• Provide cyber defenses for a corporate environment
• Maintain operations to a batch mixing plant, and
• Protect an electrical distribution SCADA system.

The biggest lesson I learned is that it takes a team to provide adequate security. No single person knows everything that needs to be known. In a mid-size business there are many network users and many require real-time data that is generated from the control systems. Access to that data from various levels in the company is critical to remain competitive. To many people, adding security that interrupts this flow can be more damaging to a firm than potential hazards that may never occur. So a team needs IT people, data users, physical security people, and ICS technicians working together to be successful.

As a team, we were reasonably successful in protecting the network, but we had three times the number of people on our defensive team as there were attackers, and we knew the red team was working hard trying to find our vulnerabilities. I don’t think that many firms are willing to assign such extensive resources to their defenses. To make matters worse, in the real world, the numbers are reversed: there are thousands of bad guys out there that would like to exploit an ICS network, and only a few to protect it. Essentially, the red team could have been more successful if they had been more aggressive. I don’t think such restraint is a problem in the real world.

Your best defense against an ICS network attack is to know your network completely. A well-documented system map is your best option—you can’t protect what you don’t know. Once your map is in place, an audit is important because business needs will cause the network to change over time and expediency might encourage some to allow forgetting network security measures. Often, an outside group is the best choice for creating and auditing your system map.

The red team / blue team exercise was great training and I had the opportunity to collaborate, learn, and share with others in the industry. The IT professionals shared the complexity and difficulty in protecting a network, and the data users and ICS technicians shared that open data flow is critical to business growth. We all agreed that it takes balance to make any cyber security program successful and sustainable in the long run.

This post was written by Bruce Billedeaux, PE. Bruce is a senior consultant at MAVERICK Technologies, a leading system integrator providing industrial automation, operational support, and control systems engineering services in the manufacturing and process industries. MAVERICK delivers expertise and consulting in a wide variety of areas including industrial automation controls, distributed control systems, manufacturing execution systems, operational strategy, and business process optimization. The company provides a full range of automation and controls services – ranging from PID controller tuning and HMI programming to serving as a main automation contractor. Additionally MAVERICK offers industrial and technical staffing services, placing on-site automation, instrumentation and controls engineers.