Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Security

Breached Water Plant Employees Used the Same TeamViewer Password and No Firewall (arstechnica.com) 80

Posted by BeauHD from the security-shortcomings dept.
An anonymous reader quotes a report from Ars Technica: The Florida water treatment facility whose computer system experienced a potentially hazardous computer breach last week used an unsupported version of Windows with no firewall and shared the same TeamViewer password among its employees, government officials have reported. The computer intrusion happened last Friday in Oldsmar, a Florida city of about 15,000 that's roughly 15 miles northwest of Tampa. After gaining remote access to a computer that controlled equipment inside the Oldsmar water treatment plant, the unknown intruder increased the amount of sodium hydroxide -- a caustic chemical better known as lye -- by a factor of 100. The tampering could have caused severe sickness or death had it not been for safeguards the city has in place.

According to an advisory from the state of Massachusetts, employees with the Oldsmar facility used a computer running Windows 7 to remotely access plant controls known as a SCADA -- short for "supervisory control and data acquisition" -- system. What's more, the computer had no firewall installed and used a password that was shared among employees for remotely logging into city systems with the TeamViewer application. [...] The revelations illustrate the lack of security rigor found inside many critical infrastructure environments. In January, Microsoft ended support for Windows 7, a move that ended security updates for the operating system. Windows 7 also provides fewer security protections than Windows 10. The lack of a firewall and a password that was the same for each employee are also signs that the department's security regimen wasn't as tight as it could have been.
This discussion has been archived. No new comments can be posted.

Breached Water Plant Employees Used the Same TeamViewer Password and No Firewall More

Breached Water Plant Employees Used the Same TeamViewer Password and No Firewall

Comments Filter:

  • How does this keep happening? (Score:5, Insightful)

    by Arthur, KBE ( 6444066 ) on Wednesday February 10, 2021 @10:39PM (#61049496)
    I wouldn't setup an automatic dog feeder like this...

    • Re:How does this keep happening? (Score:5, Informative)

      by dfn5 ( 524972 ) on Wednesday February 10, 2021 @11:06PM (#61049550) Journal

      Re:How does this keep happening?

      In my experience, generally the only people who care about security are security professionals. Sure there are always exceptions, but I work with highly technical developers all day, not the city employees in this case, and they do whatever it takes to get the task at hand completed with no regard for security, let alone the best way to accomplish the task. It is not surprising to me at all that this keeps happening.

      • Re: How does this keep happening? (Score:4, Insightful)

        by e3m4n ( 947977 ) on Wednesday February 10, 2021 @11:16PM (#61049578)
        You ever notice the people with the most access to sensitive data like Department heads and ceos are the first to tell you to override requirements for stronger passwords etc? Its not like all they do is type memos on MS Word. They attend board meetings and share some of the most sensitive insider info the company has.
        • The question is: WHY didn't he add LSD?

        • And when something goes wrong they'll hold a team meeting about how "we all need to be doing better". I know "The Office" was mostly a parody but I swear the parts where Michael Scott would call an office wide meeting on something like racial insensitivity when he's the one that committed the offense was spot on.

      • Even then, security fatigue sets in. You need to be able to use things at the end of the day, and good security is very hard.

        • Re: (Score:1)

          by Anonymous Coward
          No it's not. Password managers are like 30 bucks a year. In this case it would be actually very easy.

          • They presumably shared a password because they did not have a license to TeamViewer, and used the free version.

            As for password managers, they create their own set of risks. They have their place, but are not especially secure.

      • Our IT people make our network so secure that authorized users sometimes have trouble accessing and using it. Then people like me find ways around that to get the job done. My hacks include MAC spoofing, FTP, and cell phone tethering. The MAC spoofing one is funny. Our system only checks for authorized machines on the network. It's up to the machine to check for authorized users. But if I spoof the machine and then plug in a computer I have access to it's like manna from the heavens. Install whatever

        • That your manager let's you do it is the problem.

          If everyone just visibly goes drink some coffee every time there's a hold up because of IT there is some impetus to reduce the overhead to a minimum ... but still a documented and designed minimum, not some ad hoc bullshit.

      • Re:How does this keep happening? (Score:5, Insightful)

        by AmiMoJo ( 196126 ) on Thursday February 11, 2021 @06:39AM (#61050514) Homepage Journal

        It's worse than that. They don't just not care about security, they see it as a barrier to getting their job done. It's literally an obstacle that they have to overcome.

      • In no way I'm trying to excuse the people you're talking about but many times the developers know security should be taken into account, it's just that management doesn't care or thinks it's superfluous and, therefore, something that could be omitted and hence saved money on.
        Of course, there's also developers that don't care and just do the bare minimum to get things working

        • Re:How does this keep happening? (Score:5, Insightful)

          by MBGMorden ( 803437 ) on Thursday February 11, 2021 @11:36AM (#61051738)

          Of course, there's also developers that don't care and just do the bare minimum to get things working

          And it's also a management issue about project load.

          Lets say I'm given 4 projects and a 40 hour week to get them done.

          To write them to a quick and dirty "it'll work for now but needs to be cleaned up later" level it'll take about 8 hours each. To do it properly it'll take 20 hours each. Notice the issue? To do it properly there's not enough time (and next week they'll be throwing a new set of stuff at you).

          When the metric is "get as much done as possible" security suffers.

      • hardware / software stuck on old windows & 500K+ to up grade to software / hardware that will run on better OS's?

      • The only way security gets done right is if every dev is a trained security professional, and that is expensive. And then those developers drive up all your other costs, like insisting on new versions of Windows (and commensurate hardware upgrades). And that is expensive. The short answer to how this keeps happening? Voters are cheap, politicians know that, and so budgets don't account for security costs. Because that is expensive.

      • The guy at the top, needs to deal with a lot of general information, the details on what is going on doesn't concern them.
        Each person under him, and further down, are more and more focused on details on what is going on, but rarely concerned about the details from other, unless it affects their details.

        Security really gets in the way of that model, because it requires a lot of detail on what is going on, across all the details that everyone else is working on. And quite often Security requests make the jo

    • It is the Russian hackers. (Score:4, Funny)

      by Mr. Dollar Ton ( 5495648 ) on Wednesday February 10, 2021 @11:21PM (#61049592)

      Them damned Commies keep re-entering the brains of those poor sods via 5G wireless piggybacking on the Coronavirus and making them do it. It is like a Plague of Putins, if you will.

    • Re:How does this keep happening? (Score:5, Insightful)

      by h33t l4x0r ( 4107715 ) on Wednesday February 10, 2021 @11:33PM (#61049634)
      If you're paying by the project instead of by the hour, and security isn't explicitly in the specs, you won't get it. The contractor's goal is to do the least amount of work possible and still satisfy the project specs.

    • How does this keep happening?

      yea I don't know how the editors keep re-posting stories either

      https://it.slashdot.org/story/... [slashdot.org]

    • Re: (Score:2)

      by nnull ( 1148259 )

      Couple years ago, I had a slashdot user explain to me that this does not happen on systems like this. Experience tells me otherwise when I see industrial techs use teamviewer on a regular basis to access system across the world and most of them use the same passwords for everything. And don't get me going on the open ports directly to PLC's.

      Here we are today, same ol' crap going on. Nobody cares about security.

    • Re:How does this keep happening? (Score:5, Insightful)

      by imidan ( 559239 ) on Thursday February 11, 2021 @02:18AM (#61049886)

      Attrition happened, because everything seemed to be working and there were no disasters, so the city abandoned maintenance on critical infrastructure to pay for more visible projects.

      In a town of less than 15,000 people in Florida? Nobody there is competent to run the system. There probably isn't enough work to hire someone who is competent, especially at the rate they'd have to pay the person. They likely had a contractor install the system, and perhaps (perhaps) at the time it was even reasonably secure. But password management was a pain, and the firewall kept popping up annoying message boxes. It was inconvenient to have to access the system on-site, so why not just plug it in to the network? And the city wanted a new hockey rink.

      The most recent budget the city seems to have on-line is FY2014/2015. They spend 2% of the city budget on IT. They have an IT staff of 3--one of those is IT support, one is a GIS person, and one is the person who actually runs the IT systems. In 2014/2015, one of the city's goals was to replace their Windows XP computers with Windows 7, which came out in 2009. Windows 10 came out in July 2015. They were planning to replace their EOL XP systems with Windows 7, an OS that at that point was on the verge of being obsoleted by 2 versions. And apparently, they never upgraded again.

      They don't prioritize IT, they certainly don't prioritize IT security, they don't pay for adequate or competent staff, they aren't aware of their risk exposure, the people in charge prefer hockey rinks and business corridor improvements over basic infrastructure requirements, and as long as nothing goes wrong, they all think they're doing a heckuva job.

    • Re: (Score:2)

      by sosume ( 680416 )

      Because employees aren't charged with criminal negligence anymore, since they all play the "CYA" game.

    • 1234 is a very popular password
    • They never budgeted for IT security because the people who planned this didn't know or care about IT security. So in a word, ignorance.
    • This was a tiny water plant feeding about 15K customers. They probably didn't even have an IT department or any security policies.. They had internet access on a SCADA node!! Unbelievable At our system we have layers upon layers to get in and the machines have no internet access. Larger systems have bigger budgets and have audits and good policies.
    • It's the water utility for a city of 15,000. They probably have 3 employees and no dedicated IT.
  • Computers interfacing with embedded systems are rarely updated especially by smaller companies or government agencies. Too costly, no one knows how it works, the original contractor is long gone out of business. The embedded system it interfaces with may be in use for decades without a significant update to its software.

    • Re: (Score:1)

      by NFN_NLN ( 633283 )

      They don't even need to update the SCADA system. They just need to front-end it with a bastion host that has an updated firewall and possibly 2FA.

      • You have to realize that the persons in the government setting these things up are not computer people, the contractors want to get it done and get paid and after the fact security holes are someone else's responsibility. The mayor, city council, etc. will be long out of office when the X hits the fan for these systems.

      • Re: (Score:2)

        by nnull ( 1148259 )

        Anyone setting up industrial systems don't really care about that. They'll just install this:

        https://www.ixon.cloud/knowledge-hub/secure-industrial-remote-access-to-any-plc-or-hmi

        Just to punch through the firewall. When there's a market in the industrial sector to bypass IT security, you know things are pretty messed up.

  • I think the biggest problem with Windows is the people who use it.

    So many windows users are sitting on out dated versions of the system, with nearly every security mechanism switched off (to make everything easy). The number of times I've seen people change the permissions on a file share, or file system DACLS to grant full access to Everyone.

    • The laptop I use for PLC programming is outdated.

      I know it will take me about 2 weeks to install everything again on a new win10 laptop. not counting any potential round trip times to manufacturers because their software license enforcing software is likely to throw a fit. So in total about a month probably. So I keep putting it off, because I have "real work" that needs to be done and can't wait a month.

      Tl;Dr: industrial software is utter garbage.

  • Pretty much what I expected.

  • Stop putting stuff online (Score:4, Insightful)

    by bussdriver ( 620565 ) on Wednesday February 10, 2021 @11:07PM (#61049556)

    I don't care if it runs on a Commodore 64 if it does it's job it doesn't need an "upgrade" but never put these things online no matter how modern it is; it all gets OLD. Good luck playing the never ending upgrade game in the real world when eventually you you can no longer run the special software required and are stuck in the past... with an addiction to online integration BS.

    Emulator whatever... if it doesn't really have to be online, DO NOT PUT IT ONLINE!

    This is why everything like this has to be open source; if the company drops support they must be forced to open the source. Not that it protects you from the online addiction but it sure would help if systems would disable EVERYTHING that isn't necessary.

    Windows? WTF? I wouldn't run a toaster on windows and certainly not ever online!

    USERS have zero grasp of security. Stop letting them choose passwords! ASSIGN THEM! They'll write down whatever stupid shit they come up with anyhow.
    Stop letting users put dangerous systems online or making an idiotic phone controller.

    • My money is on Covid-19 work from home measures. What if teamviewer was installed to ensure remote access during a time when they were 50/50 rotating who showed up at work. I think the mention of outdated windows was irrelevant and the community login password for TeamViewer was all that was needed. Firewall, no firewall. TeamViewer is remote support software. Its not even required to be installed to do remote work. The local pc just has to execute it in runtime. It is only needed to be installed to a) con
    • There needs to be jail time for sharing passwords. As soon as one person or more goes to jail for sharing people will learnquick smart.

      • Rock vs. hard place (Score:4, Interesting)

        by k2r ( 255754 ) on Thursday February 11, 2021 @05:29AM (#61050372)

        I do understand that the jail industry needs more revenue but people are doing this because they need to do their work. They are not responsible for the budget and they will be considered troublemakers if they can’t do their work because security get’s in their way but somebody else can because they work around security measures.

        The solution to password sharing is some form of two factor authentication with a hardware token, preferably with the same personal device they use for physical entry.

        Security is a process and getting your work done is a process, too. These problems (password sharing) often are a result of implicit (unrecorded) and/or changing processes for the “getting your work done” part contradicting the security part.
        If everyone at every time has their individual access with all the rights they need to do the work expected, people will not share passwords.
        If people are (even implicitly) expected to work remotely but don’t have the access (yet), people will share passwords. So you have to make that impossible in a way management can see and understand. Using the personal card they use to get on site (and to track their time and - if possible - to pay in the canteen) is a good way.

        • > I do understand that the jail industry needs more revenue but people are doing this because they need to do their work.
          Oh the we were only following orders excuse. Where have we heard that before ?

          > They are not responsible for the budget and they will be considered troublemakers if they can’t do their work because security get’s in their way but somebody else can because they work around security measures.
          WHy didnt you think perhaps their bosses should be criminally responsible

        • Can you have different usernames or passwords and still log into the same account? Because logging in as a different windows user will in best case deny the user access to the controls, and in worst case the scada software freaks out and does something "undefined".

        • Why industry find it easier to maintain LOTO discipline but rarely IT security?

          I think the problem is the lack of good standards and conviction in IT (not just management) that security needs to be build around physical artefacts (smartcard, dedicated laptops, airgaps, physical paperwork to justify exceptions).

      • "There needs to be jail time for sharing passwords."

        Like Netflix passwords? It has been for a decade.

        https://www.businessinsider.co... [businessinsider.com]

    • For a small water treatment district there just aren’t enough people for 24x7 on-site supervision. It gets set up so you have maybe 16x7 or 16x6 or 16x5+8x2 and a pager. (Oh, and then someone quits and you are short-staffed so you end up with less than that.) Some facilities like this only actually have on-site staff 8x5, and do batches those five days a week and draw from them on off days.

      The problem is the remote access system was done on the cheap, apparently not even paying for a Team Viewer lic

      • Most people remember a time before internet; even after internet it took many years for upgraded systems with support to exist and be purchased.

        How did the world function without internet? The main reason for many of the computer controlled systems was to save money by automating jobs away so fewer human monitors were required. They had computers so they didn't need human monitoring 24x7 and there are stll some people alive who remember pre-computer control...

        Many of these systems have no serious need for

    • We refuse to pay for offline monitoring, and we refuse to pay for securing the online monitoring.

      We get what we pay for.

      • So we buy human monitoring to monitor our automated monitoring which exists solely to replace human manual monitoring?

        They really do not need this; they have forgotten the recent history when the great new computer system made everything better but the golden goose was NOT enough after a while so then they shoved an ethernet cable up its ass and nearly cooked their goose!

        People did this before the computer age and before the internet age.

  • >used an unsupported version of Windows

    Would that be Win7, which Microsoft _does_ support as far as they can make money on it? They gladly support their Edge browser on Win7, There's a reason W10 "upgrades" were "free". You are the profit.

    • Re: What does that mean? (Score:5, Informative)

      by e3m4n ( 947977 ) on Wednesday February 10, 2021 @11:29PM (#61049618)
      Wasnt relevant to the story anyway. TeamViewer was runnin in the background. Someone gained access by logging in through TeamViewer. Did not matter if it was windows 7, windows 10, Ubuntu 18.04, or Os X. They got in via a legitimate means of access by guessing or knowing the TeamViewer machine ID amd password.
      • THANK YOU.

        TeamViewer was the fault point. More specifically a widely available login was the culprit. They didn't use some Windows 7 exploit to compromise the system.

        FYI Windows 10 has security holes too, some from Windows 7. So stop the carping on using old operating system.

  • That's hardly a unique case (Score:5, Informative)

    by Rosco P. Coltrane ( 209368 ) on Wednesday February 10, 2021 @11:29PM (#61049622)

    I worked in the cinema industry, and boy do I have a nice list of passwords that will get you in 3/4 of the word's cinemas' servers and live streaming boxes. The DCI [wikipedia.org] standards are such a fucking joke it's not even funny.

    I have no beef against the cinema industry so I have no reason to use them. But I can tell you the whole infrastructure is staggeringly insecure. Others have though, and have played minor havoc in certain cinemas. Specifically, I remember an incident where a cinema reported someone logging into the (open, public, password known to everybody and his dog) wifi intended for audio devices for the hard of hearing, and changed the wifi's AP's SSID to something offensive. Obviously done by someone in the know.

    Sadly, that particular industry has other things to worry about, since COVID-19 essentially destroyed it, and it wasn't in such a great shape before the pandemic to begin with.

  • Seems that everything is working as designed. The hacker did the ethical thing and alerted the people responsible of the need for a change to operating procedures. I mean, it's only water, right? One of the simplest molecules around. It's not like you're gonna drink it.
    • You are not far off. He could have gassed the whole neighbourhood instead. The chlorination equipment in water plants is usually controlled by SCADA and in this case was probably accessible via the same team viewer. Wonder what happens if you open all the valves on it...

      • What happens is the water quality sensors further down the line trip and the whole plant shuts down while an engineer investigates.

        • Re: (Score:2)

          by k2r ( 255754 )

          What happens is the water quality sensors further down the line trip and the whole plant shuts down while an engineer investigates.

          But the water quality sensors had been turned off years ago because they were inconvenient. And they were some cheap model installed 15 years ago by the nephew of the major.
          And we didn’t sent an engineer to investigate but the operator has sensors as a hobby so they did have a look at it and turned it off and on again.

          Why don’t these stories happen? Why is IT different?

  • Technological Debt (Score:5, Insightful)

    by gweihir ( 88907 ) on Wednesday February 10, 2021 @11:53PM (#61049672)

    Also known as "IT rot". IT is expensive. Some people think they can do it on the cheap and just set up something, connect it to the Internet and let it run. These days that must be regarded as gross negligence.

    • Re: (Score:2)

      by tflf ( 4410717 )

      Also known as "IT rot". IT is expensive. Some people think they can do it on the cheap and just set up something, connect it to the Internet and let it run. These days that must be regarded as gross negligence.

      We can expect to see more and more as various levels of government struggle with insufficient revenue, and rapidly growing infrastructure deficits,
      Few politicians want to make the hard decisions. Given the choice of spending scarce financial resources to make the IT infrastructure safe, or fixing potholes, IT repairs get deferred 99 times out of a 100. Politicians know few if any taxpayers care about IT upgrades, while the torches and pitchforks come out if potholes don't get filled.

    • I think people donâ(TM)t realize how poorly funded the water treatment industry is in general. I donâ(TM)t pretend to know why. I work in the process automation industry and most of those facilities are running on decades old technology. Almost all would like to upgrade to a modern control system with the capability for secure remote access but canâ(TM)t get the funding so they make due with stop gap efforts that are not supported and susceptible to attack. I saw one plant that was trying t

      • Re: (Score:2)

        by gweihir ( 88907 )

        But why? Any Linux/xBSD system comes with sshd and x-forwarding. And long-term updates. The real problem these people have is using tech that is cheap in the short run, but very expensive in the long run.

  • Its pretty obvious the entire plant are pidgeons planted by a joint task force by the North Koreans, Chinese and Russians going back to well 2000.

  • Does Florida use the same password on all of it's systems that are connected to the internet?

  • Somehow, it always seems to be TeamViewer that's responsible for leaks like this. Of course, TV security is pretty bad, and it always seems to be getting hacked, but shared passwords written on stickynotes under the keyboard are a problem as well.

  • - windows 10 would not have prevented this
    - a firewall would not have prevented this

    • - windows 10 would not have prevented this
      - a firewall would not have prevented this

      Except that it was (likely) a known windows 7 security flaw that allowed files to be dropped through team viewer so that the team viewer interface itself was invisible to the physical user of the machine.

      Also, this was a waste water treatment plant, not a drinking water treatment plant.

  • You get what you pay for - if you don’t pay for safe water supply you will not get it. The city does have a yearly budget for NaOH, why don’t they have a yearly budget for IT updates and security? The city does not allow working around safety equipment measuring PH, why do they allow working around IT security measures?

    And I bet that “mouse cursor moved slowly and then ‘1’ key was pressed multiple times in a form field” is either a case of “stack of papers on desk c

  • Firewalls? The automation needs to be behind a VPN. Passwords? The key for the VPN needs to be on a smartcard, in a laptop made unusable for anything else but logging into the automation. Or at the very least a laptop with a VM dedicated to the automation, still with the smartcard.

    Maybe for emergencies have some one time use logins for remote use on an unsecured computer ... with logging and the absolute requirement to justify the use in writing afterwards, it needs to be extremely annoying to deal with the

  • Having worked in water quality for a few years, I am surprised they had a system as sophisticated as Windows 7. A lot of the smaller wastewater plants were still running Windows XP and even DOS to connect to their SCADA and PLC systems. Security is considered a big hassle as it takes a lot of training. Upgrades costs money that most municipalities simply do not have. When there is no budget, things don't get done, or done with a patchwork of parts. It is sad to see. I am not advocating throwing money at the

  • As others have stated but it's worth repeating:

    Windows 10 would not have prevented this.

    A firewall would not have prevented this.

    The culprit was TEAMVIEWER. The login for TeamViewer was the culprit. Too bad reporting is now a matter of sensationalism.

    • Re: (Score:2)

      by techmage ( 72232 )

      Training would have prevented this whole thing from exploding. Keeping critical infrastructure off the internet would have prevented this. What the hell is is Teamviewer doing on a system that is linked to their SCADA? Air Gap anyone?

      • Because security is the last concern if they are given the mandate to just get the job done and aren't given an IT budget, let alone an IT professional that provides consultation. If they had, then TeamViewer would have been off the system and remote access would only happen over a VPN. But to do this, there has to be some money to pay the professional who knows how to set this up and maintain it. As my brother likes to put it, their level of "I Care" wasn't high enough.
    • By which they mean "doing things a bit better" would have prevented this. Well, that's obvious isn't it.

  • Auditors need to join the 21st century (Score:4, Interesting)

    by sbjornda ( 199447 ) <sbjornda&hotmail,com> on Thursday February 11, 2021 @11:22AM (#61051652)
    Auditors, especially public sector (in my experience), are mostly still only looking for financial fraud like embezzlement, not enterprise risk. They absolutely could obtain the expertise needed to audit this kind of blatant, simple, security failure, and bring it to the attention of senior management. It boggled my mind that the auditors would be concerned about a hundred dollar transaction while ignoring the millions of dollars worth of equipment (IT and other technology) whose failure could shut down the entire company for 6 months. Oh yes, we put it on the "departmental" risk register, but in the absence of enterprise risk management you could talk to senior execs until you were blue in the face and all they would do is nod sagely and say they would put it on an agenda. Way to miss the big picture, auditors and senior execs.

    --
    .nosig

  • I'm guessing it's also the free-to-download "only for personal use" version as well, which is probably another reason they were all sharing one password.

Slashdot Top Deals

We are not a loved organization, but we are a respected one. -- John Fisher

Close