Why Machine Learning Alone Cannot Save the Power Grid: The Human Factor in Industrial Cyber Defense

Industrial control systems run the machinery of modern civilization—power grids, water treatment plants, chemical refineries, and shipping logistics—yet many of the devices inside them were designed decades before anyone imagined a connected world. A new study from Cardiff University argues that the biggest weakness in defending these systems is not the technology itself but a flawed assumption about who will actually use the security tools built to protect them. The research, published in Frontiers in the Internet of Things, combined decades of embedded industry experience with user-centred design methods to reveal why so many sophisticated machine learning defenses never make it out of the laboratory and into a real control room.

The threat landscape for industrial control systems, or ICS, has shifted dramatically from the era when operators relied on what security researchers call “security in obscurity.” The principle held that because the specialized devices inside factories and substations were relatively unknown to most attackers, they were unlikely to be targeted. That assumption collapsed with the Internet of Things, which pushed to connect devices that were traditionally isolated from the wider internet. The 2015 attack on the Ukrainian power grid made the new reality brutally clear: attackers gained access through remote access software installed on internet-connected SCADA workstations, took at least 27 substations offline, and cut power to roughly 225,000 customers for several hours. Much of the information about the supposedly obscure devices came from open-source resources freely available online.

Even disconnecting ICS networks from the internet offers no guarantee of safety. The Stuxnet worm, which infiltrated an Iranian nuclear facility, gained its foothold through a USB connection rather than a network cable. Since operational technology infrastructure is now so tightly interwoven with information technology, physical disconnection is not a feasible option for most organizations. The scale of the problem is staggering: Kaspersky reported in 2021 that one in three ICS devices were the target of malicious activity, and the company alone blocked 20,000 malware variants on ICS hardware in a single year. The NotPetya attack on shipping giant Maersk demonstrated how fragile the entire infrastructure is, with incident costs estimated at up to 300 million dollars.

Matthew Nunes, Hakan Kayan, Pete Burnap, Charith Perera of Cardiff University’s Department of Computer Science and Informatics, and Jason Dykes of City University London conducted the study using Action Design Research, a methodology that grounds real-world problems in iterative design cycles. The team held multiple semi-structured interviews with a panel of three industry experts, each working for a different organization specializing in ICS security, with decades of combined experience embedded in the field. The interviews lasted just over two hours and used open-ended questions designed to capture how ICS security differs from traditional network security—not just in devices and protocols, but in processes and personnel expertise.

What the experts revealed took the researchers by surprise. The team had initially prepared several network data visualizations they believed would be useful for monitoring ICS security, including a customizable dot plot, a side-by-side dot plot showing source and destination information, and a parallel coordinates plot. But the expert panel made clear that a visualization alone would not suffice. Most people working in an industrial environment, the experts said, do not have the expertise necessary to analyse network or physical data. Safety engineers do not tend to look at intrusion detection system alerts. The researchers had assumed the level of expertise inside ICS settings would be similar to that of a network administrator—a flawed assumption that rendered many of their initial sketches unhelpful.

The core insight is that operators of ICS systems have fundamentally different expertise and priorities compared to traditional network administrators. When it comes to making a decision about a cyber-attack, many stakeholders are involved: some with an operational technology background, some with an electronics background, some that are business owners, and some that are network administrators. All of them participate in the decision-making process, yet each interprets information differently. Taking a network offline to stop an attack will not just affect profit but can have profound safety implications. The experts stressed that the decision-making process should not be automated but made easier for actors from different backgrounds to make an educated response.

To formalize these requirements, the researchers created personas representing each end-user group within ICS. Each persona is characterized along three variables: physical process knowledge, meaning what is being controlled or produced within the plant; cyber knowledge, meaning understanding of the network and its security; and profit/loss margins, meaning the financial operation of the company and its tolerance levels for disruption. The personas capture the central dilemma that each actor in an ICS environment has different levels of knowledge about each aspect of the system. After the first evaluation session, the panel suggested adding the question “Do I have the people?” for operations roles, reflecting strict safety rules about how long staff can work around dangerous equipment. For the Director role, the question “What is the potential reputation damage?” was included, since recipes stored within a human-machine interface could be stolen or altered from within the OT infrastructure, leading to loss of confidence in the product.

The researchers then produced wireframes showing what a desired security solution could look like, split into three pages: an overview page with a dot plot visualizing network traffic, a device page describing the physical process the device is part of along with its financial value and importance, and an alert page with a confidence rating supplied by the machine learning classifier. The wireframes follow the “Overview First, Zoom and Filter, Details on Demand” pattern described by Ben Shneiderman. The expert panel was particularly positive about the alert page, describing it as a translation layer clearly communicating information regarding an incident to multiple personas. They liked the idea of attaching importance to a device within a process based on the percentage of network traffic flowing through it, though they cautioned this might make DNS servers seem unimportant since DNS queries do not happen that often.

The expert panel also requested a number of features that do not currently exist within the ICS cyber-security space. They pointed out that incidents typically have a fixed cost and a variable cost—personnel cost is fixed, while availability downtime may vary by season. If a third party needs to be brought in, such as firefighters, that must be factored in. They suggested breaking down costs using the CIA triad, meaning the cost of an incident can be shown with regards to its impact on confidentiality, integrity, and availability. They also recommended providing the time until each device is next maintained, so patches can be applied during scheduled windows, and attaching confidence levels to alerts with an audit log explaining how the confidence rating was calculated. The panel acknowledged it will be impossible to correctly calculate these costs precisely, but as long as an uncertainty value is provided by the system, they did not feel this would be a problem.

The study’s findings challenge the field to rethink how security solutions for industrial environments are designed. Rather than coming away from the interviews with a matured network visualization, the researchers developed blueprints for a complete system that would fill many of the gaps within ICS security. No user-centred studies within ICS were found in the literature prior to this work, meaning the information needed to design effective solutions was simply not available. The researchers hope that by sharing validated requirements, personas, and wireframes, they will inform those working in this space and increase the likelihood of their solutions being adopted within a real environment. The work was supported by EPSRC PETRAS and represents a step toward security tools built from the ground up for the people who actually keep the power flowing, the water clean, and the goods moving.

Subject of Research: Exploiting user-centred design to secure industrial control systems

Article Title: Exploiting user-centred design to secure industrial control systems

Article References: Nunes, M., Kayan, H., Burnap, P., Perera, C., & Dykes, J. (2024). Exploiting user-centred design to secure industrial control systems. Frontiers in the Internet of Things, 3, Article 1436023. https://doi.org/10.3389/friot.2024.1436023

Image Credits: AI Generated

DOI: 10.3389/friot.2024.1436023

Keywords: Exploiting, user-centred, design, secure, industrial, control, systems, scientific research