Cybersecurity exercises for critical infrastructure are essential to protect our nation and our communities. Hackers and terrorists want to exploit vulnerabilities at the core of our day-to-day lives. Utilities need to be able to quickly safeguard their operating technology (OT) networks—SCADA and the related systems and field devices—in the event of a cyber attack.
When hackers shut down part of Kiev’s power grid in Dec. 2015 (https://www.wired.com/story/crash-override-malware/), the need for cybersecurity readiness in the utility sector became more urgent overnight. So today, utilities across the country and around the world are running exercises to safeguard against cyber attacks.
Hackers gain entry through IT networks. So, to protect SCADA and the power-distribution systems (the OT network), a utility company needs to separate the IT network from the OT network. For fans of Star Trek: The Next Generation, disconnecting the IT network from the OT network equates to the Enterprise separating the saucer section, which has the bridge and command crew, from the secondary hull, which has the engines and warp core. While the command crew on the saucer section deals with the threat, the main power systems are kept at a safe distance.
With increasing IT/OT convergence—data flows crisscrossing between the SCADA side and the IT network side—dealing with a disconnection like this becomes more complicated. The disconnection procedure to protect the OT network is straightforward: shut down those router or firewall ports and pull the cables. The more difficult part becomes handling the workload after losing automation and system features. That said, when a real cyber attack occurs, people need to act quickly and know what to do. It’s best to work out the bugs now.
So, how does a company prepare for this type of threat?
Preparation begins with introducing the idea of network disconnection to the support teams, whose production systems will be impacted by the exercise. Perhaps the most important part of preparing for this exercise is the “people factor.” It is critical to establish an air of learning and identifying areas for improvement among the participants. Many of the automated methods and software features those teams use will be unavailable upon disconnection, which means the teams will need alternative/manual methods to do their jobs. Creating and documenting alternative procedures could be a project in itself for some teams, so they will need support. If a team does not have alternative procedures, they may feel at risk of “failing.” This is preparation for readiness, not a pass/fail test of team performance, and leadership needs to clearly communicate that to everyone involved.
Disconnecting the IT and OT networks causes many systems, software features, and data flows to be unavailable, including scheduled jobs. The project team needs to discuss this disconnection scenario with all of the teams supporting and using those networks, systems, and applications. Much of the work involves mapping the IT/OT convergence from these discussions and identifying the impacts to those systems and support groups. Having the conversations with all teams, external vendors, and regulatory agencies reveals connections, functions, firewall rules, and potential complications that otherwise would not be foreseeable.
The documentation for these exercises should be updated annually. Moves, adds, and changes will change the system impacts, test items, and possibly the teams’ alternative procedures for such a disconnection. These documentation updates also remind the various teams of recent changes to the environment. (BTW, to validate the expected lost and retained functionality during the disconnection, the overall list of test items will be lengthy.)
The other major challenge to performing an exercise like this is scheduling. Scheduling this is difficult due to all of the participants involved. Customer needs, weather, maintenance, vacation periods, peak workload times, and major local events all influence scheduling for utilities. Ask teams about scheduling conflicts early and often to find dates and times that will work.
Again, the most important element in conducting this exercise is to foster an atmosphere of learning and preparation among the people involved—one of cooperation, improvement, and readiness. Teams that feel stressed about the exercise may feel like their reputations are on the line. Preparing them with alternative procedures will instill the confidence they need to perform at their best to ensure a successful exercise to protect our critical infrastructure.
ABOUT SDI GUEST BLOGGER
Liam Hickey has over 15 years of experience as a technical writer, analyst, and consultant with a background in network engineering. Liam Hickey is a technical writer, analyst, and consultant with a background in network engineering. He is currently working cybersecurity disconnection exercises in the utility industry.
.jpg)
.jpg)
.jpg)
