The US Government's Anthropic Models Ban Was Never About an AI Jailbreak (techcrunch.com) 58
TechCrunch's Zack Whittaker argues that the U.S. government's abrupt export-control order forcing Anthropic to pull its Fable 5 and Mythos 5 models offline was "never about an AI jailbreak" threat. Instead, it was driven more by "personality differences" between the AI company and Trump administration. Security experts say the reported guardrail bypass did not justify the order and warn that the move sets a troubling precedent: the government can unilaterally disrupt American software products without court approval, potentially undermining trust in U.S. AI providers. From the report: Katie Moussouris, a cybersecurity veteran and researcher who founded Luta Security, said in a blog post that Anthropic recently shared with her a private copy of a paper written by security researchers describing an alleged guardrail bypass in Fable 5. (The Wall Street Journal reports that the paper's authors are security researchers at Amazon.) Moussouris said that Anthropic reached out to ask for her take on the paper. Moussouris' blog post described how the researchers triggered the guardrail bypass, but said that the bypass itself "should never have triggered an export control." The difference is largely between asking an AI model to "review code for security issues" versus asking it to "fix this code."
The end result is largely the same, even if the questions are posed slightly differently. "The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense," said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. Moussouris and dozens of other top security researchers and experts have since called on the Trump administration to revoke the export control order, calling the move to pull advanced cybersecurity capabilities from network defenders in the U.S. as "dangerous."
Past administrations have made sweeping decisions on knowledge gaps. For instance, language used by the U.S. government during the 2010s to fix export law covering cybersecurity tools that could also be used for cyberattacks was so broad that inadvertently, it nearly outlawed legitimate security and vulnerability research. However, the Trump administration's directive appears retaliatory. Justin Hendrix, the editor of Tech Policy Press, said the Trump administration's move is "likely to raise alarms in foreign capitals about the reliability of American AI for critical applications." The message is that AI companies in the United States can't be trusted to operate without interference from the U.S. government.
The Trump administration hasn't confirmed why it invoked its export control directive. Did the officials misread the report and freak out? Did Amazon CEO Andy Jassy say something to senior government officials that prompted the reaction, out of caution or spite? Was something lost in translation, or was this a way to pressure Anthropic, with whom the administration already has a fractious relationship? It's possible that the White House was unaware of the far-reaching consequences of the letter's demand and officials are scrambling to undo the damage of their own making. To quote Hendrix, "the climate is one of a cloud of suspicion that senior officials are picking favorites based on personal and political factors." The aftermath is that the government has set a dangerous precedent about how much control it intends to wield over the release of American-made software. This time the government took issue with Anthropic; tomorrow it could be with anyone else.
The end result is largely the same, even if the questions are posed slightly differently. "The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense," said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. Moussouris and dozens of other top security researchers and experts have since called on the Trump administration to revoke the export control order, calling the move to pull advanced cybersecurity capabilities from network defenders in the U.S. as "dangerous."
Past administrations have made sweeping decisions on knowledge gaps. For instance, language used by the U.S. government during the 2010s to fix export law covering cybersecurity tools that could also be used for cyberattacks was so broad that inadvertently, it nearly outlawed legitimate security and vulnerability research. However, the Trump administration's directive appears retaliatory. Justin Hendrix, the editor of Tech Policy Press, said the Trump administration's move is "likely to raise alarms in foreign capitals about the reliability of American AI for critical applications." The message is that AI companies in the United States can't be trusted to operate without interference from the U.S. government.
The Trump administration hasn't confirmed why it invoked its export control directive. Did the officials misread the report and freak out? Did Amazon CEO Andy Jassy say something to senior government officials that prompted the reaction, out of caution or spite? Was something lost in translation, or was this a way to pressure Anthropic, with whom the administration already has a fractious relationship? It's possible that the White House was unaware of the far-reaching consequences of the letter's demand and officials are scrambling to undo the damage of their own making. To quote Hendrix, "the climate is one of a cloud of suspicion that senior officials are picking favorites based on personal and political factors." The aftermath is that the government has set a dangerous precedent about how much control it intends to wield over the release of American-made software. This time the government took issue with Anthropic; tomorrow it could be with anyone else.
Why Didn't Anthropic Sue? (Score:3)
Re: (Score:2)
adding to the hype, can trump be pissed off at meta next
Re: (Score:2)
Alternately, they may have been serve a gag order, which makes a lawsuit rather tricky.
Or perhaps they are, for once, trying not to aggravate the government further? It's not like there's a 24 hour expiration date on the legal route, nor does it tend to move very quickly.
The legal route is also rather more complicated since this is a national security issue, which means federal courts, which means judges that are a lot more friendly to Trump and still haven't resolved the previous / ongoing case of whether
Re:Why Didn't Anthropic Sue? (Score:5, Informative)
This is one of those things where the courts give deference to the administration. It is an emergency action. You can litigate it later, but due to the claim of imminent harm the courts will not block the government's action.
Think about when Trump activated the National Guard and ordered them into action in California. The courts ruled against him after the fact, but the courts refused to block the action because the governments claim was imminent harm could occur.
The court MUST give deference to the government in emergencies. Just as you MUST give way for an ambulance with lights and siren going -they get the benefit of the doubt, even if it later turns out they were just sick of sitting in traffic.
Re: (Score:3)
Think about when Trump activated the National Guard and ordered them into action in California. The courts ruled against him after the fact, but the courts refused to block the action because the governments claim was imminent harm could occur.
Which was the wrong call. The Republicans ordered the National Guard into California as a political act, Trump happened to be the stooge with the button. This was all obvious from the start.
Re: (Score:2)
1. this is great marketing for them. "We're the only AI company with a model so powerful the US government held it back for national security reasons"
2. they would rather work through the objections and get it back online as fast as possible. Bringing in the lawyers, especially around national security, is just a good way to drop an anchor and go nowhere for 6 months while the court figures out how to review all the classified documents and such.
Re: (Score:2)
F the Trump administration getting in the way of a private company.
TFA is shit. (Score:2)
TFT and the thrust of TFS and TFA are trash.
The jailbreak, if real, is still trivially executed and still allows using the full power of Fable 5/Mythos model. This 'logic' is fully backwards:
"Moussouris' blog post described how the researchers triggered the guardrail bypass, but said that the bypass itself "should never have triggered an export control." The difference is largely between asking an AI model to "review code for security issues" versus asking it to "fix this code." "
It is backwards because the
Re: (Score:1)
> The jailbreak, if real, is still trivially executed and still allows using the full power of Fable 5/Mythos model.
Can you provide any evidence at all for that claim? It seems rather amazing that you have to caveat "if" it's real, but you can still confidently claim that it's both "trivial" and completely bypasses every safeguard to grant you the "full power" of Mythos.
Re: (Score:3)
"If real [as described by TFA]". The onus of providing evidence for their claim does not lie on me; I claim something else given the same premises TFA provides.
TFA: The sky is purple, thus things will have a yellowish hue without other illumination.
Me: If the sky is indeed purple, then it still makes more sense for things to have a purplish hue without other illumination, because that's how reflection generally works.
You: Do you have evidence that the sky is purple?
Re:TFA is shit. (Score:4, Insightful)
Name it after Trump and it will not only be back on the market tomorrow it will be fully endorsed by him.
Re: TFA is shit. (Score:4, Interesting)
TFA from Techcrunch is basically "creative editorializing" the original reporting of other sources (axios mainly) to justify the clickbait headline. But it you click through to the original sources the story is more nuanced and more interesting.
The surface dynamic is anthropic is in a delicate position and struggling to manage a "temperamental" regulatory power *and* strategic customer shortly before their IPO.
The background dynamic is multiple sources close enough to the matter in the exec branch felt so strongly this was an unnecessary escalation and that anthropic was the party who could and had to fix it, that they're talking to axios reporters the next day.
In an administration that is proudly punitive of leaks, don't assume multiple people are spilling the tea to reporters this quickly out of civic interest or a strong belief in the role of free press.
Re:Must've all been a big mistake. (Score:4, Interesting)
Take a look at Nightmare-Eclipse (6 Windows zero days, 3 of them still unpatched), the Oracle Peoplesoft zero day, the Cisco zero day, and on top of that a remote DoS against OpenBSD, hundreds of CVEs against linux and Firefox. This is a lot of crap raining down on corporate America, possibly more than they can handle. They can especially not handle this, if zero days keep hailing in as they did. They are not geared for this.
One can all blame this on poor coding practices, maybe the bugs were found by some other means, but we do know, how the OpenBSD remote DoS was found. I would have shut down the new Anthropic stuff just as well, bribes or not. We can't allow most of corporate America to implode, just to show, that software wants to be free.
Either China is a threat or it isnt (Score:2)
We need to beat China at AI by bulldozing our environment and violating property rights of our citizens to build it
But we also need to allow China to use our AI without limits because reasons
Make that make sense. Both cannot be true
Re: (Score:2)
China is a threat when it allows them to game the stock market to get rich quick. China isn't a threat, when it allows them to game the stock market to get rich quick. American companies are a threat when it allows them to game the stock market to get rich quick. Getting the idea here?
Re: (Score:2)
We need to beat China at AI by bulldozing our environment
We need to beat China at green energy by bulldozing our undeveloped land for solar panels.
Re: (Score:3)
NOW th issues for the USA is to prove they do not have spyware and a kill switch in any of their tech
The EU pushing for digital sovereignty is the smart move.
I would also be pointing out to TMSC that production in the EU will also be beneficial and they should open a plant there too.
TechCrunch also reports... (Score:2)
... that water is wet.
Bribes (Score:5, Insightful)
Re:Bribes (Score:4, Interesting)
Re: (Score:2)
That's possible. My guess was that Glasswing detected a backdoor that the feds had insisted be inserted into some, or much, of the popular commercial software.
Re:Bribes (Score:4, Insightful)
Sometimes the Trump regime has reasons besides personal wealth - in this case setting an example to corporate America of what will happen to your business if you refuse to cooperate.
Re: (Score:3)
The Trump administration has shown that there are NO reasons they do anything other than to make Trump and/or his friends more money. This administration is 100% about being corrupt.
Re: (Score:2)
Right, sometimes it's about personal power instead of personal wealth. Corruption, either way.
it's so tiring... (Score:5, Insightful)
Once again, America's economic interests and foreign relations have suffered because we elected an emotionally fragile boy-king who can only think of himself. What's it going to be next month?
Left or right... At this point I'd just be happy for some adult leadership in the room.
Re: (Score:2, Interesting)
Re:it's so tiring... (Score:5, Insightful)
Problem is, the damage is done - he's shown the world how unreliable the US can be, and they're not gonna forget. People loved to complain about America, but up until now they could typically count on America being willing (even eager) to lead... even if it was in a heavy-handed or tone-deaf manner.
"Make America Great Again" has, ironically, accelerated the country's decline towards irrelevance in the eyes of the rest of the world.
Re: (Score:3, Insightful)
Billions of around the world are all still eagerly awaiting the most anticipated obit in human history. If someone could nudge nature along a bit, that would be fantastic.
As nice as it will be to see his end, the chaos in the US is only going to increase when he passes away. The powers are entrenched, and Trump's chaotic stupidity right now actually slows their progress toward complete dominance. Once he's out of the way, the behind the scenes string pullers will be free to manipulate Vance, who will have zero backbone and less desire to be placated than Trump.
It's gonna get a *LOT* uglier before it turns around. Unless Trump dies right as the mid-term results sweep through.
Re:it's so tiring... (Score:4, Insightful)
You forgot one critical thing:
Somehow Trump holds uneducated manly-men in thrall despite his Ivy-league unearned diploma and having [b|m]illions of dollars his entire life.
Vance screams "Yale Law" from every pore. That's why he has to keep reminding us all that he comes from Middletown, Ohio - because you would never know it any other way. He's socially repellant and awkward. There's no way that guy can keep this grift together.
We've seen several others try to claim the crown and miserably fail for the same reasons - what country do you suppose Ron DeSantis will be offered an ambassadorship to when he's term limited out, knowing that ambassadorships are where inconvenient people are sent to quietly go away by this administration?
It's not like they have a need for a State Department, after all.
Re:it's so tiring... (Score:4, Insightful)
We tried for adult, but she had a funny laugh and was a "she" so no dice. Some number of millions of Americans were either convinced that this guy was the better choice somehow, or are just plain morons that vote against their own interests because the guy on the TV said to.
Re: (Score:2)
Harris could have maintained the facade, this is true; however, the facade was covering extreme corruption, so it is better that the facade was not maintained. Or are you thinking Trump is a cause rather than a symptom?
Re: (Score:2)
Right, did you really expect Americans to elect a black woman?! We're in this situation because we have enough racists and misogynists that we're not able to elect a smart competent black woman over an exceptionally incompetent, corrupt buffoon. Anyone who says we're not a racist country is ignoring the plain facts — many would rather see the downfall of our nation than to hand over control to a black woman.
Musk and David Sacks (Score:1)
I think it's engineered by those two, and certainly supported by them. Proof: I haven't seen Musk ask for the ban to be lifted, if he did that Trump would lift the ban.
If the Democrats win the presidency, the Trump administration just offered a legit pathway for shutting down X and x AI.
Re: (Score:2)
Re: (Score:2)
They're in charge. Who else could it be?
They should leave (Score:2)
Anthropic should just quietly go to another country and then announce their new headquarters.
Re: (Score:2)
Yeah, because this vindictive President would never have his puppet acting AG direct the Department of Justice crawl all the way up their ass on the way out, saying that the code was written in the US, and is subject to US export controls (it was, and it is. Source: I work for an avionics company where everyone has to go through US export law familiarization every single year). Then watch them control the export of it specifically against the country you tried to relocate to so you literally can't even wo
Re: (Score:3)
Then sell it and take all the talent with you. Or scorch the earth and delete all of it, then rewrite it elsewhere, which would take time but could produce a better product. My point being that the Trumpets are playing a dirge for Anthropic and all of its top employees. They need to go elsewhere for their own good. The only way the company will survive is if they sell it to Musk, Bezos, or another billionaire kiss-ass, and even that is a crap shoot given how fickle this administration is.
Congratulations, yo
Can't care either way... (Score:3)
I fully expect this administration to just be an extension of Trump's ego at this point. Yes, going after companies that he disagrees with is a major issue (see how the FCC pressured major networks and how quickly some of them folded).
I can't care about Anthropic getting in a spat with Trump. Anthropic were the ones that started this "our model is so good at finding security exploits, it's too dangerous" marketing campaign and people are surprised that the current administration took their word for it and banned even their "safe" version after a simple jailbreak was found? There is a good argument that there are other models that are about as good as Anthropic's models when it comes to finding potential security flaws, but none of them when around saying "it was too dangerous to release to the general public".
They made their bed and somebody came along and lit it on fire. Seems par for the course these days.
The miss-anthropic principle (Score:2)
Re: (Score:2)
This administration is already doing this, because that is what fascists do. They want government to control the means of production in as many ways as they can.
Easy.. (Score:1)
Model Censorship is bullshit (Score:3)
It's not only that there will always be jailbreaks, but there are also already enough competing models that have jailbreaks and/or decencensored versions. All harmful content comes also from training sets that contain public data. With a few web searches you can find the same stuff. You just have to type search queries instead of natural language questions.
Stevie Wonder (Score:2)
how much cocaine (Score:2)
It would be interesting to know how much cocaine, ketamine and alcohol are being consumed in the oval office. Anyone who hurts the fat baby's feelings gets nuked (metaphorically). Waiting for the announcements that Scotland is being invaded after locals dirtbike a few golf courses (literally).
Most likely scenario (Score:2)