Hackers Modify Water Treatment Parameters By Accident (softpedia.com) 139
An anonymous reader writes: Verizon's RISK security team has revealed details on a data breach they investigated where some hackers (previously tied to hacktivism campaigns) breached a payments application from an unnamed water treatment and supply company [PDF, page 38], and also escalated their access to reach SCADA equipment responsible for the water treatment process. The hackers modified water treatment chemical levels four different times. The cause of this intrusion seems to be bad network design, since all equipment was interconnected with each other in a star network design, and the payments app contained an INI file with the administrative password for the central router, from where the hackers reached the water treatment SCADA equipment. Of course, the hackers had no clue what they were modifying. Nobody got poisoned or sick in the end.
Re: By "accident" (Score:1)
No one got sick in the end. In the middle, though -- Hoo boy!
now that's some bad network design (Score:1)
Re: (Score:1)
This is heavy.
Re: (Score:2)
something something Earth's gravitational pull something something...
And the worst of it? (Score:5, Insightful)
Re: (Score:2, Insightful)
I just don't get this. I feel bad putting the admin password in a file on our demo VM that runs on a local workstation.
I can't imagine sleeping at night putting it on an actual system somewhere.
Re: (Score:3)
Management very rarely properly estimates damage assessments from IT.
Re: (Score:2, Troll)
I was nodding in agreement to that and then a thought suddenly struck me... what if something like this was left deliberately weak so that a part of the population could be disposed of, should it become necessary, and then hackers are the convenient scapegoat for blame in the eyes of everyone else. Especially if the hackers were associated with parties that the monied interests find inconvenient.
I know they say never attribute to malice that which can be explained by incompetence but maybe sometimes it is m
Re: (Score:2)
You're being silly. Water treatment plants are incapable of producing water that would kill, well, anybody, probably.
Yes, there's enough chlorine (or other disinfectants) in the plant to kill people; but you could open the valves to their greatest extent without jumping the chlorine content up from the usual part-per-million to more than a couple of parts per million...that is, still way less chlorine than your average municipal pool needs to combat all those filthy kids.
I suppose there's somebody so aller
Re:And the worst of it? (Score:4, Insightful)
... you could open the valves to their greatest extent without jumping the chlorine content up from the usual part-per-million to more than a couple of parts per million...that is, still way less chlorine than your average municipal pool needs to combat all those filthy kids.
But what if the bad guys CLOSE the valves? Then live pathogens go straight from the water source into the no-longer-purified water supply. Several million customers are exposed. Many are sickened. Some take permanent damage. Some die. Even after the issue is fixed the whole water system needs decontamination. And the whole set of cities fed by the plant are disrupted (which is what they're really after).
It gets even nastier if the bad guys up the ante by dumping a bit of some particularly virulent bugs upstream of the intakes, during the period where they won't be killed off by the shut-down disinfectant injection.
They use chlorine because its a heck of a lot less damaging to people than the things it is used to kill off.
Re: (Score:2)
It's a shame these hackers didn't attack the Flint water treatment plant two years ago, they could have turned the water treatment back on and saved everyone from lead poisoning.
Re: (Score:3)
Re: (Score:2)
Water treatment plants are incapable of producing water that would kill, well, anybody, probably.
You may want to consider what the first two words of that sentence is and what the purpose of these plants are.
Also you may want to look into what exactly is in your water supply upstream of water treatment.
Water treatment and sewage treatment may not be the same thing, but only due to the level of contamination, not due to a difference in the insane variety of bugs and diseases contained within either liquid. Remember, drink beer because cows are shitting in your water right now.
Killing off the population. (Score:1)
... what if something like this was left deliberately weak so that a part of the population could be disposed of, should it become necessary, and then hackers are the convenient scapegoat for blame in the eyes of everyone else.
They don't need to use a water plant for that. They've got Obamacare.
(bud-a-boom TISH!)
Re: (Score:2)
We must never forget St. Mary's and Three Waters! All good citizens should remember the Articles of Allegiance, and that Chancellor Sutler only wants to keep us safe from degenerates.
Re:And the worst of it? (Score:5, Funny)
Re:And the worst of it? (Score:5, Insightful)
If somebody had have died or gotten sick, the hacking party would be the ones to get in shit, not the asshat that put the admin password in a text file...
They both should get in deep shit for it. Yes, the asshole who left the admin password in a text file should get fired.
However, you should be able to leave an admin password posted on a banner on a 24 hr news station and a good person wouldn't use the password to get in and fuck with a water treatment plant. That's like saying that anyone who leaves their door unlocked deserves to have their house broken into and accidentally burned down while people are trying to steal shit.
So, yeah, the both hackers and the admin should be dealt with severely. This isn't an either/or situation.
Re: (Score:2)
> However, you should be able to leave an admin password posted on a banner on a 24 hr news
> station and a good person wouldn't use the password to get in and fuck with a water treatment plant.
You are not wrong but, there is a question of how much of a risk you are taking. Yes nobody SHOULD do it, but, since you know there are some number of people who WILL do things they should not, maybe the person in charge bears some responsibility for not taking more precautions than the honor system?
Its one thin
Re: (Score:3)
Or if the IT guy/department protested but was told to "do it anyway". Get that stuff in writing folks!
Re: (Score:2)
"Yes, we understand your concerns but sales already told them we could make this work."
Re: (Score:2)
Right but if the house is your chicken coup, and the family is starving because there are no chickens anymore.... whether it was animals who came in through the unlocked door or bad people, either way, youre children are still not eating.
Re:And the worst of it? (Score:4, Informative)
The incompetence here went very deep. If only the NSA were doing something useful like trying to defend this stuff against foreign hostile hackers, instead of trying to spy on citizens.
Who gets fired? (Score:1)
"Yes, the asshole who left the admin password in a text file should get fired"
Assuming, of course, that there were other options and that the application didn't work in such a way as that a password in a text file wasn't required. In the latter case, the blame would be on shitty programmers - often a third party - and not the sysadmin who did the best with what he had.
As somebody who has often had plenty of similar WTF moments with crappy software design, I would not be surprised. In the end though, if one
Your hands were on the wheel. (Score:3)
If somebody had have died or gotten sick, the hacking party would be the ones to get in shit, not the asshat that put the admin password in a text file...
The rules are no different than if you and your gang of adolescent thrill seekers climbed over the fence or found an unlocked gate and began flipping exposed switches or opening valves just for the hell of it.
I disagree (Score:5, Funny)
Re:I disagree (Score:5, Funny)
Yeah, they should have put the admin password in an XML file!
Re: (Score:2)
Everybody knows that an XML file is not safe enough, it has to be in a TXT file.
Re: (Score:1)
Come to think of it, that's a better place than in a TXT file on the server.
Re: (Score:2)
Physical access > network access
Re: (Score:2)
That's one way to protect it from Internet hackers. But then you're vulnerable to the most vicious hackers of them all: the cleaning crews!
Re: (Score:2)
No cleaning crew is allowed in our data center.
Re: (Score:2)
And make sure to base64 encode it. Nobody will figure that one out.
Re: (Score:2)
thisisnotreallyatextfilewiththeadminpasswordinsidejustrenamedtobin.bin
Re: (Score:1)
XML is so 2000's. We put our admin passwords and SQL connection strings in JSON configuration files now.
Re: (Score:3)
XML is so 2000's. We put our admin passwords and SQL connection strings in JSON configuration files now.
This. You pretty much need to ensure that your hosts are not able to be accessed because there's still the stupid plain text or MD5 hashed password in an unencrypted text file somewhere in order to connect your app to your database.
Not that encryption would matter. If someone breaks into a host that has a public key for a database server, then someone can use that same public key for access to the database server as long as they were doing it from the host that they just broke into. Actually securing con
Re: (Score:2)
At least no one would have been able to read it then.
Re: (Score:3)
Re: (Score:1)
Re: (Score:2)
.ini: PASSWORD = ""
Hacker: I guess they don't have a password, what will we do? We'll never guess it now!
:P
Re: (Score:2)
Re: (Score:2)
1) Never Cleartext Passwords (there should be no exceptions, but we all know of some)
2) Networks should be isolated from each other ("Should")
3) IT budgets are often woefully inadequate for proper securty (Cheap Labor/outsourced, low priority upgrade schedules etc,, lowest cost equipment bids).
Good IT is expensive. Bad IT is costly. Dodged a bullet on this one.
Re:cleartext passwords (Score:4, Interesting)
4) IT management rarely has any understanding of risks associated with IT designs/constraints. Even when explained to them.
Shows my main point in Hacktivism! (Score:2)
When these groups try to do their attacks, they don't realize what other fallout which may be happening. Is that Bank using the same data center of a hospital you don't know. Is the budget system going to affect other systems?
Normally the places with the worst security are not that way due to lack of IT Talent, but because the integration of legacy systems is so connected that it becomes a major undertaking to correct.
1980's Mainframes were expensive computers, most organizations could normally afford one
Re: (Score:2)
> So those hackers who think they don't hurt anyone. You are wrong.
Well, that's demonstrably not true- plenty of hackers haven't hurt anyone. More importantly, it perpetuates the downsides of the 80s style hacker mythology- that there's this group of elites who have this power to break in, or not. The problem with this mythology isn't how true it is- it seems to be pretty accurate, and it pretty fun and interesting. The problem is with how people naturally react to it. "Ok, find the bad guys and put
Re: (Score:2)
You know, reading TFA, I refuse to believe "elites" had a damned thing to do with it.
This absolutely screams of being a place which could have been taken down by a couple of bored script kiddies on their first day out.
These guys could have been skilled, but the security reads like it was so bad that it defies explana
"Nobody got poisoned or sick in the end." (Score:5, Insightful)
Problem is, this is a lot more "just the beginning" than "in the end".
How many such systems do you suppose have been penetrated by folks who do know what they're doing, and are just sitting on their access until the next political party convention, or major sporting event, or...?
Re: (Score:1, Interesting)
I distill my tap water before drinking it, using one of these. [amazon.com]
That doesn't solve this problem, of course, but it does give me an extra layer of protection against failings of the water treatment process.
Contrary to strangely-popular belief, distilled water is only barely acidic (thousands of times less acidic than soda pop, slightly less acidic than a banana), and does not leech minerals from your body. It's water. It is perfectly healthy, and it tastes good.
Re: (Score:2)
I distill my tap water before drinking it, using one of these. [amazon.com]
That doesn't solve this problem, of course, but it does give me an extra layer of protection against failings of the water treatment process.
Contrary to strangely-popular belief, distilled water is only barely acidic (thousands of times less acidic than soda pop, slightly less acidic than a banana), and does not leech minerals from your body. It's water. It is perfectly healthy, and it tastes good.
God dammit, not this again. No people, distilled water is not safe to drink. It will try to balance out that PH, it will sap minerals and electrolyres from your water, and it will shorten your lifespan.
Here's a link [livestrong.com]
And here's another. [mercola.com]
Distilled water was a health trend in the 70's, right along with the "don't vaccinate because of autism" trend in the 2000's. It's a clever troll if you want to give someone serious health problems or so, if you really find that funny, but as soon as you crack open a h
Re: (Score:2)
If distilled water actually sapped minerals, consider that once it's going into your mouth it would be combining with saliva, which has a certain percentage of dissolved minerals. Going from drinking regular to distilled water would just mean going from lots of minerals+regular to lots of minerals_distilled. Either way the result from adding the saliva is pretty much the same.
Re: (Score:2)
Except I have a lot of trouble believing that such events haven't already happened, yet there have been no attacks.
Someone is always pissed about something. I'd think that if Trump's business was riddled with holes *this would be about the right time to use those holes before he gets more and more out of control.* Yet we see only one anemic reveal from some Anonymous source which clearly was not some elite hacker who had owned The Donald. And this is a guy who has basically admitted to paying off politic
sounds like classic industrial control networks (Score:3, Interesting)
Re: sounds like classic industrial control network (Score:2)
>no one has the money or incentive
Sounds like the incentives are approaching very quickly.
Re: (Score:2)
While an insecure control system is very undesirable, you should be able to be able to overlay more modern security on top of it in the places where network interconnection is absolutely needed. You should be okay if the only remote access to your 1980's HVAC is through the 2010's firewall and intrusion detection system.
Where interconnection is not required, this is all fixed instantly by air gaps between the control systems and everything else.
I am thinking that this is due to incompetence, not the age or
Re: (Score:1)
Re: (Score:2)
Holy crap ... (Score:5, Insightful)
You know, every time I have encountered anything this moronic I've raised bloody hell over it.
Why the hell would a fscking payment app need the administrative password for the damned router, and what idiot allowed this on their network. On at least three occasions I've said "no way in hell I'm going to put a plaintext password into an INI file, and if you want me to do it you're going to have to send me an email and CC a lot of other people demanding it". (Reading TFA, it wasn't the actual payment app, but they got it off a web server they compromised which had it in an INI file, so bad job in the summary).
I swear, security is often either non-existent or written by idiots.
And that's before you even get to the epic stupidity of having your SCADA stuff to your normal network. I've been in places that had SCADA stuff, and NOTHING was on that network which wasn't fully vetted.
This whole article reads like "what happens when unqualified people run critical systems" -- right down to the fact that they also had access to "2.5 million customer and financial records".
I'd like to say I'm astonished, but that would imply that I keep being surprised at just how bad companies suck at fairly basic security.
Re: Holy crap ... (Score:3)
>Why the hell would a fscking payment app need the administrative password for the damned router,
It's such a pain to have different passwords for everything.
Responsibility (Score:1)
The persons who approved and connected critical infrastructure devices to Internet accessible networks should be hung. Air gap this stuff, people. It doesn't belong on the Internet.
Re: (Score:2)
You never know; some hacker might have been able to help in Flint (where part of the problem was using water from a polluted source, but the other part of the problem was not using enough of some of the chemical treatment).
Re: (Score:2)
You never know; some hacker might have been able to help in Flint (where part of the problem was using water from a polluted source, but the other part of the problem was not using enough of some of the chemical treatment).
You know, I find the potential value of a hacker in that system to be so small as to make it near zero.
Sure, they can hack in. And then what? Do they know what the chemical treatment required for the Flint Water District is? Are they going to re-route the water through a series of tubes away from the lead tubes through the tubes that go to where they make Perrier water? I guess that instead of a V-LAN, they'd configure the water routers to make a new W-LAN for the H20 packets.
Hackers are good at hacking
Airgap (Score:5, Insightful)
Equipment of this sort should be air gapped from the wild wild west of the internet. Frankly anything that is safety related (hospital equipment, elevators, and even HVAC systems) should be unreachable without badging into a building. While there are still ways to propagate things in via USB stick, it would keep clowns from pulling this kind of stuff.
Re: (Score:2)
Yes. Air gap and security guards doing searches should stop most of this stuff.
Some equipment does need network interconnection with the Internet, but the great majority of it does not.
Re: (Score:3)
Bingo. Air gap AND the machines that are on that network should have the USB ports filled with epoxy. When updates are needed, the vendor plugs in a special laptop for the purpose.
It's extremely useful for SCADA to use Wi-Fi, of course; nothing beats being able to haul a tablet right down under the floor where you've just unstuck a valve and then cycle the valve without running up to the console.
But the Wi-Fi of course needs to be locked down to a specific set of MAC addresses, not just with passwords. SC
Re: (Score:2)
The mac address thing is largely obsolete now, with most OSes providing mac spoofing by default, and sniffing the connected devices to copy their mac address is the first step before attempting any passwords anyway.
The thing with scada is most terminals run windows, and its those terminals who are largely targeted.
Also you don't need to have exposed USB ports anyway, something behind (physical) lock could do for occasional updates.
Re: (Score:3)
the SCADA system has to be an appliance, like a Blu-Ray player, only able to run the system programs and no others
Did you know BluRay players will execute arbitrary Java code off of BluRay discs, as part of normal operation? I'm hoping you didn't. BluRay is specifically designed to allow a disc to damage the function of the device (by invalidating keys needed to play other discs).
Re: (Score:2)
Now we pin each death in flint MI on them (Score:2)
Now we pin each death in flint MI on them.
Better plead down to life.
Is this the incident? (Score:2)
Fits the bill
Re: (Score:2)
Nope, looks like that was an accident with manual chlorination, not computerized at all.
This is an "accident?" (Score:2)
So tell ell me why screwing with the process controls in a chemical plant counts only as an "accident."
and also escalated their access to reach SCADA equipment responsible for the water treatment process. The hackers modified water treatment chemical levels four different times
All the security stuff is off-topic (Score:3)
There's only one security that counts with a SCADA system: air gap. Plant-controlling systems must not talk to any other network. ...and then a year or two later you find them trying to quietly slip two network cards into the same machine so they don't have to change chairs to go from corporate-network-with-Internet-access to SCADA.
I recently retired from a much-larger utility and we did struggle with the human factor. The plant guys heard all the lectures from their design consultants that put in the system and the IT people who checked the design over. They understand that they must not interconnect.
Emotionally, it's hard to believe anybody would *want* to break in; it's not like there's money to be made. Hollywood-movie scenarios where "hackers take over" are ludicrous; every device in the plant has an "On/Off/Auto" switch where only "Auto" leaves SCADA in control at all; the most junior operator could run around the plant hitting those switches in five minutes, restoring manual control. (Then we'd have to bring in a dozen folks with cell phones to run the plant manually; no sweat).
And as I posted above, it's not like you can kill anybody with a water treatment plant: the worst water you could put out would either be untreated (please boil water) or absolute max chlorine the system could insert (still less than a swimming pool).
It's going to be the same as "safety"; you can pound safety lectures into people's heads all day, but it seems to take a generation or two for the message to really sink in; hard hats and visibility vests were strenuously avoided as well. We're just going to have to make it a standard, like safety standards: firing for disobedience, regardless of whether anything went wrong.
Re: (Score:3)
There's only one security that counts with a SCADA system: air gap. Plant-controlling systems must not talk to any other network.
And you instantly fail all sorts of control, maintenance, reliability analysis, regulatory requirements for data, optimisation, etc tasks as a result.
Admittedly a water treatment plant is simple and probably should get away with air-gaps, but the words air-gap are the first words that everyone utters when they talk about control systems. This causes two problems.
1. Air-gaps need to be breached to enable a whole world of optimisation and value improving abilities in control systems these days. An air-gapped
Nobody got poisoned or sick in Flint either (Score:1)
How do we know, what the mid- and long-term effects will be? There is no one obviously poisoned by tap water in Flint, Michigan [mlive.com] either.
Should we apply the same spin to people responsible for that, as the submitter applied to hackers because he sympathizes with them?
Re: (Score:1)
Nobody in Flint has symptoms. And there being no symptoms is what led the submitter to acquit the hackers as well.
Re: (Score:1)
Stop shouting, Illiberal asshole... You aren't on an anti-Trump rally complaining about "not being heard".
They did not die on the first day the pipes got contaminated, did they? Not even the first month. Back then, somebody wishing to be sympathetic to the people involved in the mess could've said, the same thing: "Nobody got poisoned or sick in the end".
Today, somebody sympathetic towards the hackers, is making the same mistake making the same
Better call Walter O'Brien (Score:1)
Who was the dipshit engineer? (Score:2)
There is ZERO..... Z E R O reason for the payment system to have any connection to the SCADA system.
Whoever is the manager of that plant and the engineer of that SCADA system and it's network need to be put in jail for their incompetence.
Re: (Score:2)
If we start jailing people for incompetence, we're going to need a lot more jails.
Re: (Score:2)
Considering I worked in the field for 10 years and have extensive experience with it? A lot of thought went into it... What is your background with water filtration and SCADA systems?
I thought so.
Small company with a couple of machines? as a water treatment plant? if the place cant afford another $499 pc to set on someone's desk to keep the systems safe they need to shut down due to even more gross negligence.
Re: (Score:2)
Re: (Score:2)
Not one that is SANE. only those that are utterly insane and want to make it insecure would want to connect remotely.
I see you dont understand a thing about security and know nothing at all about SCADA.
Re: (Score:2)
What happens without regular audits (Score:2)
Re: (Score:1)
"Of Course"? (Score:2)
"Of course, the hackers had no clue what they were modifying."
The report discussed the intruders having little apparent knowledge of what they were doing. The anonymous reader assumes this to mean that the intruders didn't know they were screwing with a water treatment SCADA system.
I think it just as likely that they had figured out they had tapped into a process control system, and were figuring out how to manipulate the system... driving by Braille.
The RISK report report authors could have summarized t
Re: (Score:2)
Clowns to left of me, jokers to the right, Here I am, stuck in the middle with you.
That's Stealers Wheel [youtube.com].
Nobody got poisoned or sick in the end... (Score:2)
Nobody got poisoned or sick in the end. (Score:1)
Re: (Score:2)
I think this was white hat hacking, where a pen testing company got hired and asked "what does this widget on this public app/website do" and modified treatment parameters.
Re: (Score:2)
Not unless Rick Snyder figured out how to run nessus...