Working in the cyber security red zone

Do you have enough first responders within your company when it comes to dealing with cyber security incidents and network violations? How can you make sure you aren’t developing critical staffing gaps?

By Tim Conway April 19, 2013

As I examine a growing problem across the identified critical infrastructure sectors within the U.S., I believe we need the 1996 movie “Multiplicity” to become a reality. In the movie, Michael Keaton plays Doug Kinney, the main character who is overwhelmed with his responsibilities at work and home. All he can see is an ever-growing list of things to do on the horizon with little or no hope for success. To resolve this issue, Doug works with a scientist to clone himself so his surrogates can divide all those responsibilities and get everything done with no one the wiser. While this comedy is obviously fictional, the immediate needs for a talented, trained, and capable cyber and operations workforce is far too real, and there are individuals within the CI/KR (critical infrastructure / key resource) sector that we definitely need to duplicate.

While staffing needs ebb and flow in all organizations and fulfilling strategic staffing demands is a continuous effort, many believe there is a real issue developing that will impact CI/KR-essential roles. A term that was recently introduced to me is “red zone jobs.” These are positions or roles that are absolutely essential in performing an operational mission in normal and emergency conditions. Red zone jobs across the CI/KR sector would typically be classified as those roles that have real-time response requirements and perform an essential operational or operational support role in a real-time environment. Throughout the CI/KR sector, technology jobs often perform system, application, network, communications, security, or security responder engineering roles within an operations technology (OT) department.

Apply traditional management?

Many organizations have looked at the impact these positions have on the operations environments of an organization through traditional management processes such as business impact analyses, workforce planning initiatives, or organization pandemic planning. In these traditional activities, organizations attempt to identify a condition that could create an operational problem and then begin efforts to identify steps that could be put in place to prevent the problem from occurring. These traditional approaches identify operational risks as a result of technology loss for a period of time, loss of specific skill sets or knowledge, and potential loss of employees necessary to perform a critical operations role. 

The problem facing the CI/KR sector red zone jobs is a mixture of the traditional problems identified above and challenges in the available qualified workforce for the industries that need them. The pipeline of people moving into the workforce that have the necessary skills, knowledge, and capabilities to perform the critical red zone jobs compared to the pipeline of people exiting those positions is not balanced. This unbalanced condition seems to be worsening as the number of individuals exiting is increasing, the need across multiple sectors is growing, and the available programs or development capabilities has remained flat. This problem is unique in that entities do not control the process that educates, trains, and develops the necessary capabilities of candidates until they are hired into the workforce.

Most companies cannot independently solve this issue. They can, however, influence a direction that will improve the industry overall and strengthen their own workforces. Many entities have worked with traditional educational institutions or specific training providers to develop programs that will help meet the growing needs of the red zone jobs. The focus is almost always on training content and knowledge assessment, which is an essential first step. However, the gap that remains in these development approaches is the capability or “right fit” issue that exists as a component of all red zone jobs across the CI/KR sector. These companies and other entities will continue to face challenges in assessing a candidate’s capability to be successful in a red zone job or training candidates to ensure a successful fit within a role. To combat this issue, many entities are moving to technology implementation of active policy enforcement systems or intelligent monitoring and alerting tools. This helps alleviate the reliance on a knowledgeable, qualified, and capable workforce to perform these processes; however, it also needs to be acknowledged that adversaries are also automating and implementing intelligent tools and evasion tactics. Therefore the number and complexity of attacks will grow, and the very complex attacks will require a knowledgeable, qualified, and capable workforce to detect and defend the environment.

Evolving job demands

A topic that also needs to be discussed is the growing reliance on technology for all critical operations across the CI/KR sector that is creating an increase in red zone jobs. Entities across most CI/KR sectors would have identified a very different set of red zone jobs 30 years ago than they would today. For example, within the electric sector circa 1983, most utilities would have included linemen, substation engineers, switching operators, and dispatch operators for transmission and distribution environments as critical roles. Generation environments would have likely identified generating station control room operators, instrumentation and control engineers, and relay engineers as critical roles. 

Looking at these same environments in 2013, while the roles previously identified are still critical, they are now performed in a dramatically different fashion, and in many cases rely on additional capabilities and roles that did not previously exist. In addition there are now new functions that have moved into an absolutely critical role that likely were not considered all that critical 30 years ago. Consider the criticality of control centers today, RTOs and ISOs, the systems and support functions for communications, and market functions. The interdependencies have grown immensely, and too often individuals do not fully understand how they may impact others within the organization. As mentioned previously, this interdependence applies across the CI/KR sector and companies need to begin to understand those dependencies in depth. Additionally, in today’s red zone jobs that are technology or automation driven, a complex dependency exists on the technology utilized throughout the organization. Within an organization many trusts exist: trusted communication paths, trusted users, trusted external organizations, and trusted applications. As organizations identify these trusts and dependencies, they can identify and mitigate the security risks more effectively.

Think about what the phrase “red zone” conveys in American football: when defensive players have their backs to the goal line, the situation demands peak performance because the threat is imminent and has to be turned back. Similarly, defender roles within the CI/KR sector’s red zone need to be ever present and the capabilities of the individuals in those roles need to be fully developed to achieve peak performance.

Recommended actions

Companies and other entities can begin the analysis process by looking at a few straightforward measures. The first step is to assess their current staff capabilities or limitations:

  • Identify red zone job positions or roles within your facility that are essential to real-time operations and operational support
  • Assess organizational capabilities and identify red zone job areas for improvement
  • Join in industry wide efforts to better equip individuals currently in red zone jobs or better prepare new candidates, and
  • Understand the underlying technologies utilized by operations and the complex interdependencies that exist within and external to the organization.

These steps can help guide your ongoing efforts to filling these critical positions, since unlike the movies, I don’t think we will have human cloning for security purposes anytime soon.

Tim Conway is technical director, ICS and SCADA for the SANS Institute.

www.sans.org 

Key concepts:

  • Your company’s ability to respond to a cyber violation often depends on the actions of a few key individuals
  • A few simple analysis steps can help you evaluate your staffing situation and determine a direction