Is your water system more concerned with cybersecurity? New WBRC survey says yes

BIRMINGHAM, Ala. (WBRC) – A new exclusive WBRC FOX6 News survey of water systems across Alabama finds all of those who responded are spending more time preparing for and defending against potential cyber-attacks than they did even two years ago. That finding, while eye-opening, may be good news given that a recent successful series of cyberattacks and new EPA warnings illustrate how water systems across Alabama may be the most vulnerable parts of the state’s critical infrastructure to a cyberattack.

“We’ve moved into a different world within the last two years on AI and what it can do,” warns Alabama Public Service Commissioner Jeremy Oden, part of the three-member PSC charged with regulating most of the state’s key utilities.

“Our regulatory authority is really making sure that their systems are intact, and what they have is going to be protection enough. We’re going to have to monitor this AI technology and figure out how do we monitor the bad stuff and apply the good stuff to these situations,” says Oden.

“It’s pretty simple: we are woefully unprepared and asking for utter chaos,” warns Joseph Brunsman, a former electronic warfare officer in the U.S. Navy and now a cybersecurity expert and president of a cybersecurity insurance broker and advisory firm. “If we just take water systems…these systems are old. When they were designed, cybersecurity didn’t really exist in any meaningful sense. So, we are starting from a position of complete weakness and then trying to build cybersecurity on top of that. We’re just trying to put Band-Aids on Band-Aids.”

“Utilities, most of the ones we regulate and that I look at, are pretty much on target with this,” Oden tells WBRC. “They’ve come up with systems, and they’re still working. But it’s a fluid world. You’ve got some systems, they may be smaller and don’t have dedicated personnel in the IT system trying to find out ‘hey, is this impenetrable or not, what are our weaknesses?’ Some of those systems haven’t don’t that yet, so they don’t really know what their weaknesses are. Our bigger systems that we regulate – Spire and Alabama Power – they’re on top of it, they do this continually.”

“Water is definitely the easiest to attack and also, there are so many of them out there,” says Dr. Ragib Hasan, a cybersecurity expert at UAB. “In the whole country there are about 150,000 water systems and unlike the other critical infrastructure that get more attention like power or gas—they get a lot more attention and they have a long tradition of focusing on security aspects. Compared to that, water agencies or systems, they are not protected at all because they are smaller agencies and don’t have the budget to invest in security personnel. They use the default password, and that makes them low-hanging fruit for hackers.”

String of Recent Attacks

That vulnerability came into stark relief in January when hackers, now thought to be possibly linked to a Russian government-affiliated hacktivist group, hacked water systems in three small Texas panhandle communities, causing water to overflow from a water tank in Muleshoe, TX for about 30-45 minutes, the city’s manager tells CNN.

The same attack targeted nearby Abernathy, TX.

“We were the very first city to be hit in a string of hits when it happened throughout the country,” Abernathy City Manger Don Provost tells WBRC.

“We monitor our water system through an HMI device, a Human Machine Interface that is connected through a VNC, a Virtual Network Connection that we use to monitor our wells when they pump so we can shut them off and turn them on however we want remotely. They were able to get in through our VNC dashboard, is where they were able to penetrate and get into the system and change passwords and stuff like that – but there was no disruption,” Provost says.

“We monitor that 24 hours a day and the person on call at the time was able to, because he was monitoring the system, he noticed that changes were being made within the HMI system. When he noticed that he didn’t really know what was going on, he pulled the plug right away and disconnected it from the network. They weren’t able to effect anything whatsoever, we didn’t really have any disruptions in our water at all because we were able to catch it so quickly. That’s a testament to constantly monitoring the system and being aware of what’s going on.”

“A group that calls themselves Cyber Army Russia Reborn basically claimed that they had disrupted those systems, those water systems,” says John Hultquist, Chief Analyst at Mandiant Intelligence, which is part of Google Cloud. “We’ve been tracking that group because we’ve known they’ve been used as a front for a military intelligence front out of Russia and so we immediately sort of focused in on that problem. They basically found a method they could repeat against multiple targets. That basically meant getting into this vendor which allowed them to essentially manipulate controls behind the scenes.”

Provost says, at the time, his city’s water system wasn’t behind the main firewall the city maintains for most of the other computer systems at city hall, but that has since changed.

“It sucks that it happened, but at the same time, it was kind of a good thing for us because it showed where our weakness is within our network, which we corrected immediately and spent the money to upgrade our entire network so now it would be fair to mention we’re probably one of the safest, most secure cities in Texas,” Provost says.

“Absent some kind of unique circumstances, you don’t have to outrun the bear, you just have to outrun the other guy,” warns cybersecurity expert Joseph Brunsman.

“So, if you have a nation-state coming after your local town, there’s probably nothing you’re going to do that’s going to stop them,” says Brunsman. “But if you’re worried more about random hacker guy getting into your water supply, it’s ‘hey we just have to be a little harder to get into than the next guy.’”

We’re going to see kind of big picture world chess type maneuvers where we have foreign adversaries coming into our critical infrastructure for obvious reasons, because it’s much cheaper, much more effective, and you can do much more damage. We’re also going to just kind of see random people around the world who happen to get lucky, and try to make money.

The hardest part here is, imagine you’re a really smart kid from some third world country. You could go work in the mine, work in the field, do manual labor, or in one day you could make more money than all of your ancestors in history combined and you could live like a king for the rest of your life. That’s a very hard problem to fight against. They can make a ton of money. They have to be right once, we have to be right every single time and that asymmetric, hyper-localized threat is really difficult to defend against.”

“It’s actually fairly unusual for us to see the physical manifestation of a lot of these incidents,” Hultquist says. “We do see intrusions, we do see attempts to manipulate things every now and then, but it doesn’t always end up in a situation where everything changes, physically.”

The Texas hacks came about two months after hackers believed to be affiliated with Iran compromised a Pennsylvania water system using a controller made by an Israeli company, causing the controller to display “Down with Israel,” but not severely disrupting the system’s operations.

In March 2024, National Security Advisor Jake Sullivan and the EPA Administrator sent a letter to Governors warning them of “disabling cyberattacks striking water systems throughout the United States,” and warning that a Chinese state-sponsored cyber group known as Volt Typhoon is “pre-positioning themselves to disrupt critical infrastructure operations.” The letter warned Volt Typhoon has already compromised the IT of “multiple critical infrastructure systems.”

How Are Alabama Water Systems Responding?

Given the escalating challenge, how are Alabama water systems responding? Most of the 123 public water systems we surveyed either didn’t respond or were unwilling to speak about their cybersecurity protection. The reply we got from Madison Utilities exemplifies much of what we heard on and off the record from local water system managers.

“We will not be able to comment in detail on your survey, as this would not be wise,” Madison Utilities Wastewater Manager Mark Bland told us. “I can tell you that Madison Utilities takes cyber security very seriously. We are very aware of the most recent cyber attacks to water systems.”

“We take this seriously and have partnered with CISA and outside consultants to test our systems,” Trussville Gas & Water’s General Manger Mike Strength tells us. “In addition to properly securing and testing our electronic systems, we use non electronic solutions to provide reliability in the event the electronic systems are compromised.”

CISA is the Cybersecurity and Infrastructure Security Agency, the federal agency tasked with protecting core infrastructure including water systems.

“Industry best practices for tabletop exercises and self-audits suggest that organizations conduct both at least annually,” CISA’s Region 4 Director Julius Gamble tells WBRC.

All but one of the water systems who responded to our survey said they have conducted a tabletop cybersecurity exercise in the last two years or were about to undertake one, and ADEM confirms it participated in a cybersecurity overview and tabletop exercise hosted by the EPA for Alabama water systems on February 15.

“The water sector doesn’t necessarily have the same amount of resources at hand, and they’re dealing with, in some cases, threats from spies,” warns John Hultquist, Chief Analyst at Google Cloud’s Mandiant Intelligence. “These are really hard problems for anyone—they’re going to need help.

Timeline of Cyberattacks on Water Systems

New Help Available

The EPA and CISA released a new Water Sector Cybersecurity Toolkit in January with new response guides and a reminder of the free cybersecurity scans the agencies offer, as well as the Known Exploited Vulnerabilities Catalog, a database CISA says is the “authoritative source of vulnerabilities that have been exploited in the wild.”

“Many of the cybersecurity incidents that have occurred over the past decade could have been prevented by simply sharing timely, quality, and actionable information,” CISA Region 4 Director Julius Gamble says. “We encourage critical infrastructure owners and operators to voluntarily share information on cyber incidents to help prevent other organizations from falling victim to similar incidents.”

“To secure themselves, water systems can take basic security measures first,” says Dr. Hasan. “It doesn’t take a lot to protect against very common attacks. Let me give you an example. A lot of these utilities that were hacked back in December and recent months were hacked because they were using a particular pro logic controller which came with a default password of “1-1-1-1-1,” and nobody bothered to change that. That’s why it was really easy for the hackers to break into them without breaking any password – they just tried the default password and it worked.”

The EPA issued a rule in March of 2023 that would require public water systems to include cybersecurity in their mandatory audits, but withdrew that rule in October after a federal appeals court paused the rule in the face of a challenge from a handful of states.

Having been the victim of a recent cyberattack, Provost, TX City Manager Don Provost knows well what resources are out there, even for smaller systems like his.

“There is a lot of grant money out there for cybersecurity, so even if you don’t have the budget for it, there’s a lot of federal and state grants and money that’s out there – pretty much free money to be able to do that,” Provost tells WBRC. “I’ve been able to tap into some of that…I would say having cybersecurity insurance is probably number one behind applying for grants.”

Most of the Alabama water systems we surveyed refused to answer whether they carry cybersecurity insurance, with most saying answering that question could make them a target of hackers.

“A successful year is when I call you once and take your money, because if we’re talking twice – you’re having a bad day,” says Cybersecurity Insurance broker and consultant Joseph Brunsman.

So what does he recommend?

“I think it’s incumbent they (water systems) at least bring in a third party and say ‘hey where do we think our biggest holes are, what’s the biggest bang for the buck?’” And then just start kind of chipping away at this problem over time. Work with the legislature to have this funded over a period of years to spread this out appropriately and start going after this problem,” Brunsman says.

“Now again, that costs money, that costs time, both of those are in short supply when it comes to municipalities. But as the public at large, we all have a vested interest in the lights staying on, the water not poisoning us, so I think there probably should be a demand at the local level for some kind of independent third party to come in and say ‘hey, this is where you’re lacking, this is where you’re good, this is how we fix it’ and have a plan moving forward because this problem – it’s only going to get worse.”

Alabama’s largest water system, the Birmingham Water Works tells WBRC “As part of our ongoing commitment to cybersecurity, we continuously assess and strengthen our defenses against evolving threats,” BWWB Public Relations Director Rick Jackson says. “Our dedicated team works diligently to implement robust security measures, employing state-of-the-art technologies and industry best practices to mitigate risks effectively. Moreover, we understand the importance of education in combating cyber threats. Through regular awareness programs, we ensure that our employees remain vigilant and well-equipped to identify and respond to potential risks, including scams, malware, and other cybersecurity challenges.”

How Can You Help Make Your Water Source More Secure?

Many of the experts we spoke to for this story said water system customers need to take a more active role in securing their own water source by pushing their water systems to spend more time and money on hardening their cyber defenses.

“This impacts everyone,” says UAB’s Dr. Hasan. “So all of us should advocate for more security, more resources being allocated for securing local water systems.”

“Ask them,” PSC Commissioner Oden suggests. “It is our responsibility to ask them and say ‘what are you doing to protect (1) my personal data, (2) what are you doing to protect your grid?’ And they really need to be giving you an answer.”

“Ask their governor to aggressively support water system cybersecurity, and make sure that their water system has at least signed up for CISA’s free weekly vulnerability scanning service and has performed a cybersecurity assessment,” recommends Andrew Hildick-Smith, an advisor and OT Lead at Water ISAC, the international security network created by and for the water sector.

“It’s changing everyday and we’ve got to be ahead of the change,” Oden says. “Is there going to be some breaks? You better believe they’re going to be. Another issue is what happens when that happens, what is the follow up scenario?”

“I’m not overly concerned, I’m not in a panic here,” says Hultquist. “I think, when I see these incidents, I don’t think it’s too late. In fact, I think the opportunity here is for us to act before it’s too late. We have plenty of opportunity to harden our defenses here. The adversary has given us a warning whether they wanted to or not, and we should take that warning and use it.”

Suggestions for Securing Water Systems

What should your water system be doing? Click here for a list of steps the experts we spoke to recommend.

Get news alerts in the Apple App Store and Google Play Store or subscribe to our email newsletter here.

Copyright 2024 WBRC. All rights reserved.