Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
×
The Military Open Source Programming United States

DARPA Is Worried About How Well Open-Source Code Can Be Trusted (technologyreview.com) 85

An anonymous reader quotes a report from MIT Technology Review: "People are realizing now: wait a minute, literally everything we do is underpinned by Linux," says Dave Aitel, a cybersecurity researcher and former NSA computer security scientist. "This is a core technology to our society. Not understanding kernel security means we can't secure critical infrastructure." Now DARPA, the US military's research arm, wants to understand the collision of code and community that makes these open-source projects work, in order to better understand the risks they face. The goal is to be able to effectively recognize malicious actors and prevent them from disrupting or corrupting crucially important open-source code before it's too late. DARPA's "SocialCyber" program is an 18-month-long, multimillion-dollar project that will combine sociology with recent technological advances in artificial intelligence to map, understand, and protect these massive open-source communities and the code they create. It's different from most previous research because it combines automated analysis of both the code and the social dimensions of open-source software.

Here's how the SocialCyber program works. DARPA has contracted with multiple teams of what it calls "performers," including small, boutique cybersecurity research shops with deep technical chops. One such performer is New York -- based Margin Research, which has put together a team of well-respected researchers for the task. Margin Research is focused on the Linux kernel in part because it's so big and critical that succeeding here, at this scale, means you can make it anywhere else. The plan is to analyze both the code and the community in order to visualize and finally understand the whole ecosystem.

Margin's work maps out who is working on what specific parts of open-source projects. For example, Huawei is currently the biggest contributor to the Linux kernel. Another contributor works for Positive Technologies, a Russian cybersecurity firm that -- like Huawei -- has been sanctioned by the US government, says Aitel. Margin has also mapped code written by NSA employees, many of whom participate in different open-source projects. "This subject kills me," says d'Antoine of the quest to better understand the open-source movement, "because, honestly, even the most simple things seem so novel to so many important people. The government is only just realizing that our critical infrastructure is running code that could be literally being written by sanctioned entities. Right now." This kind of research also aims to find underinvestment -- that is critical software run entirely by one or two volunteers. It's more common than you might think -- so common that one common way software projects currently measure risk is the "bus factor": Does this whole project fall apart if just one person gets hit by a bus?
SocialCyber will also tackle other open-source projects too, such as Python which is "used in a huge number of artificial-intelligence and machine-learning projects," notes the report. "The hope is that greater understanding will make it easier to prevent a future disaster, whether it's caused by malicious activity or not."
This discussion has been archived. No new comments can be posted.

DARPA Is Worried About How Well Open-Source Code Can Be Trusted

Comments Filter:
  • So lets say they are scanning the kernel and they find something they don't like. What next?
    • I predict this research will be used to cause the damage it is supposed to prevent.

      • This just seems like the gubmints public-facing reason for running surveillance on the Linux community, because China.
        • Re:Actions (Score:5, Insightful)

          by timeOday ( 582209 ) on Friday July 15, 2022 @08:55AM (#62705078)
          Does it matter that their public-facing reason clearly does make complete sense?
          • I guess I don't see the sense in basically running a dragnet with unknown parameters across the entire Linux community because Chinese people write source code too.
            • Do you honestly not? You don't believe that people try to do nefarious stuff? Or don't believe that China is a particular offender at hacking? Because they do, and they are.
              • I don't subscribe to the "one bad apple spoils the bunch" line of thinking, especially when it comes to matters of public surveillance/data privacy.
                • What are they doing that constitutes public surveillance, or an invasion of what should remain private data?
    • by Immerman ( 2627577 ) on Friday July 15, 2022 @08:54AM (#62705076)

      Presumably they fix it?

      It's not like they have a lot of other options. What are they going to do, run Windows? Then they can't even scan to detect the back doors other governments have hired programmers to install. (And if you think all Microsoft programmers only work for Microsoft, I have a bridge to sell you. That's way too tempting a target.)

    • by sjames ( 1099 )

      How about submit a bug report, or better yet a patch?

    • "People are realizing now: wait a minute, literally everything we do is underpinned by Linux,"

      "People are realizing now: wait a minute, literally everything we do is underpinned by Windows."

      One of these is a cause for serious concern, the other is a cause for relief. I'll let you decide which is which.

  • How well can it be trusted? Are we supposed to trust whatever the owner of such code claims, just because they say so?
    • by Anonymous Coward

      I think they're taking for granted that the closed source code can't be trused so they're concentrating on the open source code that they might be able to help with. Makes sense to me.

    • by fazig ( 2909523 )
      You haven't read past the title, or have you?

      In your defense, it's a shitty title that does not encompass what the article is about. DARPA is worried about the major contributors being from countries that the US sanctions, namely China (Huawei), and Russia (Positive Technologies).

      Anyway, that's what the article is about. Whether there's anything to those worries I don't know.
      As far as I'm concerned, given how many security backdoors have been found in Huawei 5G equipment, (for those who can't sense the
      • by splutty ( 43475 )

        You could be charitable and say that this research is meant to see if there is actual evidence at all.

        • by fazig ( 2909523 )
          If the article didn't mention China and Russia specifically within the context of "sanctioned by the US government", I might have.
          But from the phrasing I can say that at least the authors of that particular article decided to put a spin on it by quoting Dave Aitel out of context. But in any way, that's how they decided to put it.

          The "bus factor", that d'Antoine was quoted on is a quite valid point to make, though that appears to be mostly just a foot note about what else Margin Research does.
  • They are starting with the Linux kernel of all places.. The Linux kernel is one of the most analyzed pieces of software on the planet. A smarter move would be to scan some of the libraries that don't get nearly as much attention as they should

    • by Anonymous Coward on Friday July 15, 2022 @08:22AM (#62704982)

      The Linux kernel is one of the most analyzed pieces of software on the planet

      "I've been tested more than any cyclist ever." - Lance Armstrong

    • They are starting with the Linux kernel of all places.. The Linux kernel is one of the most analyzed pieces of software on the planet. A smarter move would be to scan some of the libraries that don't get nearly as much attention as they should

      The law of relativity can apply to software. The more energy that goes in to it the less likely it will do what you want it to do. And it is just good practice to analyze the energy going in to such a vital and large project. So "many eyes" is well and good but you cannot assume all eyes can or want to see what they should.

      • by gmack ( 197796 )

        My issue is that there are a LOT of security professionals looking at that code, plus several university projects working on code analyzers. Meanwhile, we have a bunch of libraries that we depend on that just don't get anywhere near the attention we need. A good example of this was the ssl libs a few years ago that ended up with crazy exploitable bugs because no one was really paying attention. I'm sure the state of security for some of the desktop libs are even worse.

        If you are going to throw governmen

        • My issue is that there are a LOT of security professionals looking at that code, plus several university projects working on code analyzers.

          Yes, so a lot of opposing opinions that get in to things that are sure to cause conflicts eventually. Therefore bring those opinions closer together before they commit to them? The fastest and and most unrecoverable way to break upstream is to break the downstream base they depend on.

        • This comes to mind: https://xkcd.com/2347/ [xkcd.com]
          • by gmack ( 197796 )

            Exactly. Although if I drew it, there would be 10 of those tiny vertical pieces all stacked on each other.

  • Eyes (Score:5, Insightful)

    by J-1000 ( 869558 ) on Friday July 15, 2022 @08:13AM (#62704958)

    The key security feature of Linux has always been eyeballs. Yes anyone can contribute, but you also have many, many people looking at the changes.

    So you choose who to trust: a) a planet of fellow users, any one of whom could raise an alarm the moment they saw something amiss, or b) a small, secretive group.

    How many people have seen the Windows source code?

    • How many people have seen the Windows source code?

      ...and neither went blind nor died laughing?

    • by HiThere ( 15173 )

      Perhaps they don't use the MSWind code on anything "sensitive". I don't know about this decade, but a couple of decades ago that would have been the smart decision. (I stopped both using and reading about details of MSWind software a couple of decades ago.)

      A couple of decades ago, my choice for anything sensitive would have been Unix. Now it would probably be Linux. So it's a good thing if they're trying to make it more secure. Python is another good place.

      OTOH, the argument that there are lots of libr

      • Yeah, I imagine a lot of sensitive uses are in things like weapon control systems where they're using a stripped-down OS that's only a bit more than the kernel to begin with.

    • Re: Eyes (Score:5, Insightful)

      by djp2204 ( 713741 ) on Friday July 15, 2022 @08:39AM (#62705022)

      Are you sure there are many many eyeballs looking at the code? Maybe everyone thinks thats the case, and because of that no one is.

      • Re: Eyes (Score:5, Insightful)

        by scamper_22 ( 1073470 ) on Friday July 15, 2022 @09:31AM (#62705174)

        People don't even look at published documentation, much less the code.

        The recent log4Shell vulnerability really highlighted this. How many people knew that log4j automatically resolved LDAP values? I sure didn't. No one in the office did. You didn't need to look at the code for this. Just looking at the documentation of Log4j, you could have found this out. Yet, who looked at this documentation before reading it? Knowing this, it would have caused red flags on my end.

        Ultimately, we do need more of something on Open Source that underpins huge parts of our infrastructure. Not just from a security point of view, but even reliability. We have things like building codes and electrical codes... to make our physical infrastructure safe.

        I don't quite know the solution, but there's a bunch of options.
        1. Have government/industry fund projects to make sure they are staffed (probably based on the most used components)

        2. Rely on vendors like Red Hat to 'take responsibility' for the components. Government/big players would not use the components directly. They'd use approved versions from a red hat repository and redhat would have people on staff to maintain those projects.

        3. Have government/industry/non profits fund things like security scans/audits of components

        4. Make things like a bill of materials (SBOM) mandatory. This work is already in progress. ...

      • by e3m4n ( 947977 )
        that's exactly how the o-ring issue for the challenger space shuttle happened. 20 different independent checks results in so many people assuming its already been checked or going to be checked 19 more times, they took the lazy approach. If DARPA wants to scrutinize open-source code, nobody is going to stop them. That's literally the point of open source. Given the scope of their project, I suspect it is going to be quite the feat. I think this is more about the sociology of code contribution more so than
        • by HiThere ( 15173 )

          Not "the one task", but one of a few. (It's already better than humans as *several* specialized tasks.)

          • by e3m4n ( 947977 )
            well Im not completely ready to fully trust AI yet. Code review and making suggestion is fine. I would rather not let it write its own code. I am also against revoking freedoms based on the results of a pre-crime algorithm. Letting that cat out of the bag is as bad as the joke 'when black babies are born, are they already on parole?'. There is too many variables that can change the outcome up until the last second. Motive Means and Opportunity work fine for a - did he do this crime - question. Not nearly en
      • by J-1000 ( 869558 )

        Are you sure there are many many eyeballs looking at the code? Maybe everyone thinks thats the case, and because of that no one is.

        I think you make a good point.

    • by znrt ( 2424692 )

      well, windows code was not written by "sanctioned entities", likely even with some help from the nsa! (and i probably should point out that was ironic.)

      of course these "sanctioned entities" have never ever done anything against the usa, but simply invaded a border country after clearly warning the usa for years they would do so if they kept creeping their military pact in there, which of course the usa gleefully did. makes one wonder who the real threat is here ...

      but, yeah, we have to be all terribly scare

      • Hey dude. Russia isn't gonna suck your dick.
      • Even if the USA did what you claim (hint: it's a lie), that would have changed nothing, because Putin wants to go into the history books as a modern day Peter the Great. Everything else is smoke, diversion and cover. And you're a troll adding to the latter.
        • by znrt ( 2424692 )

          Everything else is smoke, diversion and cover. And you're a troll adding to the latter.

          coming from someone making wild claims with no evidence whatsoever that's a quite comical assessment + ad-hominem combination.

          • You seriously want to dispute that Putin has aspirations to be the next great Russian leader? Have no been paying so little attention? Perhaps you're too dumb to understand that you're trolling...
            • by znrt ( 2424692 )

              yes. how can you even possibly determine what anyone's aspirations are? if you would just think for a second about what evidence there actually for what our media has been parroting non stop you will realize there is none.

              the facts don't accompany either: the big imperialist conqueror couldn't even cross the dnieper river. the narrative is just laughable.

              however: the history of nato expansion, the steady deployment of nato military units in ukraine, the coup that ousted the existing regime, its links with

              • Crimea means exactly nothing to you? Funny, that. Instead, you repeat the narrative that Western media are limited..? Sure, I know. But as you refuse to believe Putin wants to be the next Peter the Great, because I can't prove that (analysts agree on that, perhaps not on those words, but with that idea), please you prove that Putin hadn't attacked had NATO and the US handled things differently. It has never been stated before the attack afaict. Any official message of the Kremlin will do. Thanks.
                • by znrt ( 2424692 )

                  Crimea means exactly nothing to you? Funny, that.

                  while the annexation of crimea wasn't precisely backed by any international law (yes, that's ironic) it hardly counts as a proof of russian imperialism/expansionism.

                  first of all, crimea is a critically geostrategic russian interest, second it didn't happen in isolation, since rusia already had control over it thanks to agreement with the existing ukrainian "state" ... which got clearly invalidated by that coup. third, crimean population is overwhelmingly ethnic/culturally russian, and wanted to stay that wa

                  • Thank you for taking the time to write up your position. Since I followed the situation after the MH17 missile attack, I realise that more had been going on, since a long time. Please realise that the separatists are a local minority, to begin with....

                    Aside that, the pretext of the invasion in the official words of the Kremlin are that Ukraine has to be denazified and the Russian speaking Ukrainians need to be liberated. Not a word about NATO in there, and a lot sounding like expansion wishes. You think t

                    • by znrt ( 2424692 )

                      Thank you for taking the time to write up your position.

                      glad to be able to discuss such sensible topics in a calm and rational way.

                      Please realise that the separatists are a local minority, to begin with....

                      even if they were, they are still waging war. they really aren't a minority, that's just a blanket statement that i guess goes to uphold the myth of ukrainian unity which is now in danger etc. i'm not questioning ukrainian identity at all, but that unity never existed and is just another myth from western media. ukraine has been highly polarized for many years. here https://en.wikipedia.org/wiki/... [wikipedia.org] you can see a few maps of electora

                    • Wow, you have more time than I do, and once again thank you for your write up. Your initial messages came across to me as much less differentiated and trolling, now I see your points, though I don't necessarily agree with all of them. Regarding Russian speaking Ukrainians, one of my former colleagues here in Switzerland was one such person, and after the annexation of Crimea he shouted loudly that he wished Russia would take all of Ukraine. He's left way before the war, so I'm not sure if he still thinks th
          • Also, talking about wild claims, claiming that any different behaviour by the USA wouldn't have lead to Russia invading Ukraine, or undertaking a special military operation, as you may call it, has no basis in reality. This has been clear since Russia annexed Crimea. Without that pesky Corona, it might well have happened way earlier too.
    • by SirSlud ( 67381 )

      The consideration is that of security and national interest. "Many eyeballs" isn't inherently a good thing if all those eyeballs are in the heads of adversaries. Just as important is "whose eyeballs".

      Your implication is "don't worry, it's like, lots of different people all over the world" but given the high stakes, that's not a sufficiently precise answer, and may not be entirely accurate given the size of the kernel. The answer to "who and how many are looking at what" is likely to be quite different depen

    • You are making the wrong argument. Seems like DARPA is working in good faith, what here is not to like?
    • by Ichijo ( 607641 )

      The key security feature of Linux has always been eyeballs.

      That's true in theory but Linux is written in C, and C is a write-only language [c2.com].

      "In theory, there is no difference between theory and practice, while in practice, there is." --Benjamin Brewster [wikiquote.org]

    • The key security feature of Linux has always been eyeballs. Yes anyone can contribute, but you also have many, many people looking at the changes.

      So you choose who to trust: a) a planet of fellow users, any one of whom could raise an alarm the moment they saw something amiss, or b) a small, secretive group.

      How many people have seen the Windows source code?

      That's true.

      But if I want to contribute a backdoor into Windows I either need to compromise a developer of the relevant windows subsystem (very hard) or plant a skilled dev as an agent and get them hired by MS and put onto an appropriate development team, probably harder. And then I still need to get the code through review.

      There's a surprising amount of security that comes from the fact that MS is a single organization where devs are placed there by the organization and they are all fairly well known by th

    • by kmoser ( 1469707 )
      For an idea of how Windows source code was written, look at any entry from the IOCCC [ioccc.org].
  • by Opportunist ( 166417 ) on Friday July 15, 2022 @08:13AM (#62704964)

    That is, as well as you can do a source audit of it.

    • So can DARPA, in that they're negotiating licensing that lets them do that.

      The feds have long had the Windows sources, and all the other ones too. If they choose do to code analysis, they can actually make a direct code-to-code comparison.

      • by HiThere ( 15173 )

        Having the "sources" doesn't help that much unless you also have the build chain, and ways to ensure that the version you checked matches the version you got.

        And with all software you need to worry about the recent updates and patches to things you've validated.

  • So, after decades of using Free Software and not supporting the folks who write the software, they realize hey, maybe we should give something back.

    Took you guys long enough.

  • by Tony Isaac ( 1301187 ) on Friday July 15, 2022 @08:26AM (#62704986) Homepage

    Automated analysis can find the low-hanging fruit. This is important, because so many developers are unaware of even the simplest security risks, like SQL injection. If that's the goal, this effort has potential value.

    We have a lot of experience using automated analysis tools, known as antivirus software. Basically, antivirus software is only useful for detecting known threats. They have very little ability to detect previously unknown or newly developed threats. This DARPA analysis will suffer from the same shortcoming.

    • by HiThere ( 15173 )

      Mmmmh...that's not quite right. The simple tools you're talking about that may be true of, but if they're talking about code scanning AIs you could probably detect potential buffer overflows that nobody's yet targeted and various other well characterized coding flaws.

      The thing is, running those tests isn't easy or cheap. So if the government wants to do it and recommend fixes, that's great. And the source code is out there, so they CAN do it. (Of course, they could also run those tests, and then keep th

      • Oh yeah I forgot they were using AI! Well then everything will work splendidly then I'm sure!

        AI is only as good as the data set it was trained on.

        And existing antivirus tools aren't so "simple" any more. They too use "AI" and heuristic analysis to try to find patterns that don't exist in newly developed threats. The problem is, these analytics have a lot of false positives, so most organizations turn them off.

      • by Anonymous Coward

        The simple tools you're talking about that may be true of, but if they're talking about code scanning AIs you could probably detect potential buffer overflows that nobody's yet targeted and various other well characterized coding flaws.

        I think you're misunderstanding the GP. We know how to write tools to detect potential buffer overflows, although balancing that detection with false positives and programmer effort continues to be an active area of research.

        What we don't know is how to write tools that detect security issues no one has defined yet. No matter how good a buffer overflow detector you make, it will never detect timing side channel attacks, for instance. At best, your concept of checking for security is limited by how accuratel

  • Linux kernel might have some China in it. Better make sure we don't have to nuke it and anyone who touched it.
  • by The Evil Atheist ( 2484676 ) on Friday July 15, 2022 @08:51AM (#62705072)
    So you want code that can be trusted, but you also want to force backdoors to be put in for the "good guys"?

    Well every government thinks they're the good guys. But what we end up with is just backdoors.

    If you're concerned about how well open source, or ANY code, can be trusted, then fund multiple independent research groups to analyse and harden the code against ALL backdoors, whether or not they're for the "good guys".
  • by JBMcB ( 73720 ) on Friday July 15, 2022 @08:57AM (#62705092)

    Looks like DARPA is really interested in Linux kernel security now. If only there was a governmental agency who was interested in Linux kernel security and has been for decades.

    https://en.wikipedia.org/wiki/... [wikipedia.org]

    • If only there was a governmental agency who was interested in Linux kernel security and has been for decades.

      selinux is good, but it is a bit hamstrung by the lack of good tools to go with it. As well, Capabilities is only one kind of security. There's also just making sure you haven't left a hole someplace that lets an attacker just go around that system.

  • by Anonymous Coward

    Neither are perfect.
    Closed - You have to trust a vendor, who in turn, needs to trust their employees. However, that's a finite chain of trust that can be developed, documented, and maintained. In the end you have a single entity you can drag into court for their failures. Implying, the vendor has a vested interest to perform the due diligence in trying to get it right. I am agreeing that not all of them succeed. Heck, it can even fail at times within our own spook agencies. Rosenbergs, Snowden, etc.

    Op

  • Because they are working for money.
  • As many other have said, if they want an OS with no code written by developers from China or Russia or any other place DARPA doesn't like they're very welcome to use Windows or develop their own OS. Instead they still want to use Linux because it's far more secure than either of these options but somehow want it to only use code from approved countries.

    If they genuinely want to pay for an extra audit of Linux kernel code then by all means, as long as they reveal anything that audit finds with complete trans

    • by nagora ( 177841 )

      As many other have said, if they want an OS with no code written by developers from China or Russia or any other place DARPA doesn't like they're very welcome to use Windows or develop their own OS.

      If you think Russia and China have never managed to plant agents within Microsoft to write code for them then you're dreaming. This is espionage 101 level stuff.

  • TL;DR: Government shouldn't and doesn't really worry about open source. It is a cash cow for them, but they don't understand it.

    I find it funny that the former NSA guy is nervous about the Linux kernel, while NSA itself were major contributors under MDA904-01-C-0926. Their contributions brought about enhancements to Security Enhanced Linux (SELinux) as well as some very helpful guidelines in how to lock down Linux and other server operating systems. Did they try to backdoor Linux? Of course they did,

C'est magnifique, mais ce n'est pas l'Informatique. -- Bosquet [on seeing the IBM 4341]

Working...