Someone broke into the computer system of a water treatment plant in Florida and tried to poison drinking water for a Florida municipality’s roughly 15,000 residents, officials said on Monday.
The intrusion occurred on Friday evening, when an unknown person remotely accessed the computer interface used to adjust the chemicals that treat drinking water for Oldsmar, a small city that’s about 16 miles northwest of Tampa. The intruder changed the level of sodium hydroxide to 11,100 parts per million, a significant increase from the normal amount of 100 ppm, Pinellas County Sheriff Bob Gualtieri said in a Monday morning press conference.
A press release is here.
Better known as lye, sodium hydroxide is used in small amounts to treat the acidity of water and to remove metals. It’s also the active ingredient in liquid drain cleaners. It higher levels, it’s toxic. Had the change not been reversed almost immediately, it would have raised the amount of chemical to toxic levels.
“This is obviously a significant and potentially dangerous increase,” Gualtieri told reporters. “At no time was there a significant adverse effect on the water being treated. Importantly, the public was never in danger.”
So far, authorities have made no arrests, but they are chasing down several leads. Gualtieri said it’s not clear if the intrusion came from inside or outside the US. Both the FBI and Secret Service are also investigating. The sheriff’s department has alerted area municipalities to the attack and recommended they inspect their water treatment systems and other infrastructure for signs of a breach.
The first signs that anything might be amiss occurred on Friday morning, when a plant operator noticed someone had remotely accessed a system that controls chemicals and other aspects of the water treatment process. Gualtieri said the operator didn’t think much of the incident since his supervisor and co-workers regularly logged into the remote system to monitor operations.
Then, around 1:30 that same day, the operator watched as someone remotely accessed the system again. The operator could see the mouse on his screen being moved to open various functions that controlled the treatment process. The unknown person then opened the function that controls the input of sodium hydroxide and increased it by 111-fold. The intrusion lasted from three to five minutes.
The operator immediately changed the setting back to the normal 100 ppm, the sheriff said. Even if the malicious change hadn’t been reversed, he said the other routine procedures in the plant would have caught the dangerous level before the water became available to residents. It takes 24 to 36 hours for treated water to hit the supply system. No poisonous water was ever released.
The incident is certain to renew the debate over whether processes for utilities and other critical infrastructure should be exposed to the internet. The Pinellas County Sheriff’s Department didn’t immediately respond to a question asking if the utility required personnel to use two-factor authentication to gain remote access to interfaces like the one that was breached in Oldmar. Reuters, citing an interview with Gualtieri, reported that Teamviewer was the application used to gain remote access, but the department didn’t immediately respond to this question either.
Jake Brodsky, an engineer with 31 years experience working in the water industry, said it’s not at all uncommon for water utilities to make such interfaces available remotely. While he frowns on the practice, he said that Gualitieri was probably correct when he said the public was never in danger.
“There’s a bunch of different things [water utilities] look for, and if they see anything out of kilter, they can then isolate the storage water,” he said in an interview. “The danger here is relatively minimal as long as you catch it soon enough and there are multiple checks before that happens.”
Of course, if intruders can remotely tamper with a process, they may also be able to tamper with the safety redundancies in place. If Brodsky were advising Oldsmar officials on better securing their water treatment plant, “the first thing I’d probably do, and this almost doesn’t cost anything, is you disable the remote access,” he said. When remote access is required, as occasionally is the case, connections should be manually allowed by someone physically present and the access should time out after a brief period of time.
“I can’t imagine leaving a connection like that open and exposed to the world,” Brodsky said. “This is cheap and easy. All you do is call the operator and you get the access.”