Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
AI Technology

Dozens of Top Scientists Sign Effort To Prevent AI Bioweapons (nytimes.com) 53

An anonymous reader shares a report: Dario Amodei, chief executive of the high-profile A.I. start-up Anthropic, told Congress last year that new A.I. technology could soon help unskilled but malevolent people create large-scale biological attacks, such as the release of viruses or toxic substances that cause widespread disease and death. Senators from both parties were alarmed, while A.I. researchers in industry and academia debated how serious the threat might be. Now, over 90 biologists and other scientists who specialize in A.I. technologies used to design new proteins -- the microscopic mechanisms that drive all creations in biology -- have signed an agreement that seeks to ensure that their A.I.-aided research will move forward without exposing the world to serious harm.

The biologists, who include the Nobel laureate Frances Arnold and represent labs in the United States and other countries, also argued that the latest technologies would have far more benefits than negatives, including new vaccines and medicines. "As scientists engaged in this work, we believe the benefits of current A.I. technologies for protein design far outweigh the potential for harm, and we would like to ensure our research remains beneficial for all going forward," the agreement reads. The agreement does not seek to suppress the development or distribution of A.I. technologies. Instead, the biologists aim to regulate the use of equipment needed to manufacture new genetic material. This DNA manufacturing equipment is ultimately what allows for the development of bioweapons, said David Baker, the director of the Institute for Protein Design at the University of Washington, who helped shepherd the agreement.

This discussion has been archived. No new comments can be posted.

Dozens of Top Scientists Sign Effort To Prevent AI Bioweapons

Comments Filter:
  • by FudRucker ( 866063 ) on Friday March 08, 2024 @03:47PM (#64301095)
    So it's not a matter of "if" but when and how bad it can get, there's plenty of people that think the world is over populated that would gladly unleash a global pendemic to cull the herd, (was covid an alpha test run?)
    • by gweihir ( 88907 )

      Actually, it is not really. The large models can only used as a service. Hence they can be shut down.

      • The primary attack vector wouldn't be asking ChatGPT to play a fun game of biological doomsday attack planning. A Nation State would have the hardware and resources to train their own purpose-built model.
        • Though having huge publicly accessible machines capable of taking a shot at questions like this which are also vulnerable to social engineering attacks is... really something.
          • I would think the biggest risk is a foundation model of chemical or protein interactions [ibm.com] (as are used in drug design), rather than an LLM.

            (Sorry if I am underestimating you, GPT4).

            • Right there's no question there are more dangerous assets for biological weapons, and I'm begging the bigger problem is going to be model theft by nation state actors or black market resellers. It's more of a general observation about the... progression? of security vulnerabilities in our technological infrastructure.
        • by gweihir ( 88907 )

          A nation state does not need it. I take it you have not read the story?

          • I did and I don't see how it supports your comment. What did I miss?
            • by gweihir ( 88907 )

              ... "could soon help unskilled but malevolent people create large-scale biological attacks"...

              Obviously, nation states can get or already have _skilled_ people.

              • Uh ok So do you think all world governments have the knowledge and skill to create novel proteins? Or do you think that no world government is malevolent?
      • Here's a chemical weapons example: Dual use of artificial-intelligence-powered drug discovery [climate-science.press]. You take a model designed to minimize harm and then flip the sign on the objective function. So it's not too implausible.

        It wouldn't be an LLM, but there's lots of machine learning stuff that isn't based on LLMs.
    • by gtall ( 79522 )

      "was covid an alpha test run?"

      Was JFK assassinated by LBJ?

      Is the former alleged president a Russian agent?

      Are UFO's real and the government is covering up that we've been infiltrated?

      Is the government keeping herds of pink unicorns?

      Hey this fun, anyone can do it....just rattle off whatever zephyrs in your brain.

    • I wonder how many of those signatories have also signed NDAs/secrecy agreements with govt agencies for work they've done or been consulted about for the US military & spy agencies. My guess is that they're positioning themselves on this issue out of well-informed concerns that they're not at liberty to divulge to the public.

      Just like other germ & chemical warfare projects in previous decades, we can be sure that our governments are working on weaponising this stuff, despite public declarations to
  • Then it's totally going to work.
    • Maybe it's more like nuclear physics.
      • by Roger W Moore ( 538166 ) on Friday March 08, 2024 @05:46PM (#64301393) Journal
        Stopping nuclear proliferation is only possible because it is insanely hard to separate two isotopes of the same element and radioactivity is extremely easy to detect. The result is that it takes a massive industrial effort to create nuclear weapons meaning that only nations can do it and, when they do, it is basically impossible to hide.

        I'm no expert in biology but I suspect it will eventually become much, much easier to create a deadly bioweapon that it is to create a nuclear weapon. However, as dangerous as such a possibility might be, in the past the solution to technological horrors has always been more technology. If you cut off certain areas of advancement because you are scared at what they may enable that will not stop others with far less noble intentions exploring them and then they, and not you, will have the benefits of that knowledge and often that knowledge can contain the best solution to prevent those potential horrors.

        The best example of this is nuclear weapons: as I said above part of the reason nuclear proliferation can be controlled is because it is so easy to detect radioactivity and part of the reason they have not been used for so long is that multiple countries have them. Had we never studied nuclear physics over concerns that it might lead to nuclear weapons, then we would never have developed the sensors we have today. Leaving a nation like North Korea free to develop nuclear weapons without anyone knowing about it or being able to deliver an equally horrific response should they ever choose to use them.
        • It occurs to me that the development of an effective Covid vaccine was record-breakingly fast due to this general line of technology. Although this is somewhat less enthralling if the disease itself was also the result of genetic engineering.
        • Yes it's super easy, I created genetically engineered antibiotic resistant bacteria, many years ago as a noob biology college student (the rest of the class did too). Took a couple hours to do. The only hard part would be getting the right DNA, which we ordered from another lab.

          As for bioweapons, there's been various experiments to try to make them super dangerous but not come back to bite you (with a good chance it skips the "come back"). But the only way to do that seems to be if it doesn't spread, which

        • by mjwx ( 966435 )

          Stopping nuclear proliferation is only possible because it is insanely hard to separate two isotopes of the same element and radioactivity is extremely easy to detect. The result is that it takes a massive industrial effort to create nuclear weapons meaning that only nations can do it and, when they do, it is basically impossible to hide.

          I'm no expert in biology but I suspect it will eventually become much, much easier to create a deadly bioweapon that it is to create a nuclear weapon. However, as dangerous as such a possibility might be, in the past the solution to technological horrors has always been more technology. If you cut off certain areas of advancement because you are scared at what they may enable that will not stop others with far less noble intentions exploring them and then they, and not you, will have the benefits of that knowledge and often that knowledge can contain the best solution to prevent those potential horrors.

          The best example of this is nuclear weapons: as I said above part of the reason nuclear proliferation can be controlled is because it is so easy to detect radioactivity and part of the reason they have not been used for so long is that multiple countries have them. Had we never studied nuclear physics over concerns that it might lead to nuclear weapons, then we would never have developed the sensors we have today. Leaving a nation like North Korea free to develop nuclear weapons without anyone knowing about it or being able to deliver an equally horrific response should they ever choose to use them.

          We've also done pretty well with biological and chemical weapon proliferation. To hear of a gas attack is pretty rare, even amongst organisations that have almost no regard for human lives. If Hamas could get their hands on a chemical weapon, would they not have unleased it on Tel Aviv by now?

          Proper biological or chemical weapons are also fairly difficult to make, also quite difficult to deliver. It takes skill and equipment to develop a viable strain of anthrax and even that won't guarantee a high fatal

    • It's more like the Paris accord and climate change. CO2 levels are still going up despite any promises from most countries on the globe.
      • I love how the implication is that its always the fault of the humans.

        Welcome to the dawn of the misanthropic era.
        • I love how the implication is that its always the fault of the humans.

          PT Barnum proved long ago that our species is dumb enough to believe it. Every time.

          The fate of humanity was never in question, because we only pretend to evolve. Just a matter of time.

    • Gun control works pretty well in countries that have implemented it after mass-shootings, e.g. the UK & Australia. Mass shootings & deaths by firearm are relatively rare, violent crime & burglaries are relatively low, & they haven't been overtaken by tyrannical dictators.

      Are people in the USA so violent, murderous, & despotic that you need more guns than people to keep them at bay?
      • by piojo ( 995934 )

        Are people in the USA so violent, murderous, & despotic that you need more guns than people to keep them at bay?

        In short, yes. You have gotten to one part of the problem. At least if you don't take guns away from the gangs but do prevent law abiding citizens from getting guns, you make the law abiding citizens defenseless--this isn't the worst thing ever, as there are tons of countries where the bad guys can get guns but normal people can't--but American criminals are too violent and already have lots of guns.

        (Of course there are other self defense tools like pepper spray, but guns are more effective, and are also go

        • I remember seeing an interview with a US police officer stating that it was common for victims to be shot by their own firearm after criminals taking it from them.

          Whatever the contorted, torturous rationales gun nuts can come up with for not putting adequate gun controls in place, the facts are that guns are a huge problem that outweigh any possible benefits by a large margin.

          The USA has already left the rest of the civilised world aghast at its overt willingness to accept massacring its own school ch
  • AI will usher in the new dark ages, it will drive knowledge deeper into protected silos that will be protected as anyone could train. The issue is , you won’t need to train one up specifically for making bombs or bio weapons, just the chemistry and engineering behind it the future won’t be open or transparent..
  • by TwistedGreen ( 80055 ) on Friday March 08, 2024 @04:06PM (#64301151)

    Wow I thought this was crazy but then I found sites like this: https://www.genscript.com/gene... [genscript.com]

    The standard delivery includes:
    * 4 g of lyophilized plasmid containing your gene insert (1 g for low-copy plasmid) *
    * Sequence chromatograms or NGS read depth plot covering your gene (electronic)
    * Construct map for the plasmid (electronic)
    * Quality assurance certificate

    • Where my mind goes: Someone coming up with a custom genetic bioweapon that targets and only affects a specific sub-group of humans. Spooky.

  • For the first time in human history, it seems probable that a nuclear-scale weapon of mass destruction could be designed in a glorified garage lab by somebody filled with enough hate and despair not to give a damn how many innocent people they kill.

    If you were a Palestinian doctor whose entire family had been wiped out by Netanyahu's thugs, aided and abetted by the US, Canada, the UK and a number of other countries whose citizens actually voted for those governments, what might you do?

    • Solution to the Fermi Paradox, right here.

      • I hate to say it, but you might be onto something.

      • Don't need that. We're squandering the easily reached iron. Once it's gone to rust then it's much harder to redevelop industrial technology for the next species. Plate tectonics don't bring it up as fast as we can fuck it up.

        But sure, that could do it.

        • Don't need that. We're squandering the easily reached iron.

          All those sensible Grandmas out there cooking on cast iron all those years to save future generations from the cancer of non-stick sales gimmicks, being held responsible for harm via mining deficiencies.

          Save the iron, or the kids. Oh the moral dilemmas we manufacture on behalf of Greed.

  • by Joe_Dragon ( 2206452 ) on Friday March 08, 2024 @04:54PM (#64301293)

    falken's maze
    black jack
    gin rummy
    hearts
    bridge
    checkers
    chess
    poker
    fighter combat
    guerrilla engagement
    desert warfare
    air-to-ground actions
    theaterwide tactical warfare
    theaterwide biotoxic and chemical warfare
    global thermonuclear war

The best defense against logic is ignorance.

Working...