Amazon CEO's Talks with U.S. Officials Triggered Crackdown on Anthropic Models (msn.com) 40
The Wall Street Journal reports:
The Trump administration's decision to halt all foreign use of Anthropic's most capable AI models was prompted by conversations between Amazon Chief Executive Andy Jassy and U.S. officials including Treasury Secretary Scott Bessent, people familiar with the matter said.
Researchers at Amazon had used a series of prompts to get Anthropic's Fable 5 model to provide them with information that could be used to aid cyberattacks and was supposed to be off limits, Jassy told the officials, according to people familiar with the matter. Tech industry executives have been in regular touch with the administration about the power of cutting-edge AI tools. Shortly afterward, White House officials held a meeting to discuss how to respond and security researchers began testing Amazon's claims. The officials asked Anthropic to fix the vulnerabilities or take down the model, according to administration officials. The officials decided that the most direct way to address that risk was by preventing foreign governments, companies and individuals from accessing the tool, the people said. President Trump later signed off on the action despite reservations about it hindering innovation, a senior White House official said.
The administration had long felt that Anthropic, one of the leaders in America's AI race, couldn't be trusted to manage the security risks its new model presented. Friday's call between some administration officials and Anthropic Chief Executive Dario Amodei reinforced that feeling, the people said...
Anthropic has said that the vulnerabilities like those flagged by Amazon are relatively basic. The company has said that other publicly available models are capable of discovering them and that they don't represent a full so-called jailbreak, a point of view shared by some security researchers familiar with Amazon's research.
The article points out that Amazon is "a big investor in Anthropic, supply Anthropic with chips for data centers.
Researchers at Amazon had used a series of prompts to get Anthropic's Fable 5 model to provide them with information that could be used to aid cyberattacks and was supposed to be off limits, Jassy told the officials, according to people familiar with the matter. Tech industry executives have been in regular touch with the administration about the power of cutting-edge AI tools. Shortly afterward, White House officials held a meeting to discuss how to respond and security researchers began testing Amazon's claims. The officials asked Anthropic to fix the vulnerabilities or take down the model, according to administration officials. The officials decided that the most direct way to address that risk was by preventing foreign governments, companies and individuals from accessing the tool, the people said. President Trump later signed off on the action despite reservations about it hindering innovation, a senior White House official said.
The administration had long felt that Anthropic, one of the leaders in America's AI race, couldn't be trusted to manage the security risks its new model presented. Friday's call between some administration officials and Anthropic Chief Executive Dario Amodei reinforced that feeling, the people said...
Anthropic has said that the vulnerabilities like those flagged by Amazon are relatively basic. The company has said that other publicly available models are capable of discovering them and that they don't represent a full so-called jailbreak, a point of view shared by some security researchers familiar with Amazon's research.
The article points out that Amazon is "a big investor in Anthropic, supply Anthropic with chips for data centers.
What about domestic use? (Score:3)
Re: (Score:3)
Re: (Score:2)
How hard may it be to someone to resell the API to other nations? As soon as there is a domestic use model, there is a huge gray market reselling the API.
Random ass "decisions" (Score:1)
Yep, definitely some tech (and country) to rely on.
Re:Random ass "decisions" (Score:5, Informative)
The story is pure spin. If there was any truth in what's being said then all LLMs would be banned. The real story is one of pick-on-Anthropic.
Re: (Score:1)
I mean, it is a pretty impressive model. And not just from the benchmarks. Relentlessly proactive and resourceful. It's a shame it's cut off.
Re: (Score:2)
some one got scared, some one got paid to shut it down other got paid to make all look good.
Re:Random ass "decisions" (Score:5, Insightful)
Re: (Score:2)
Probably. But in the larger scheme of things, this just makes it abundantly clear that, besides its numerous other severe problems, LLM type AI is not even reliably available. This bribe will likely backfire massively.
Good. The sooner the hallucination collapses, the better.
Missing from the article (Score:2)
Re: (Score:2)
The risk equation is:
risk = offensive_capability * jailbreakability
According to Anthropic's own doom-saying, Mythos could literally end the whole-wide world overnight with a series of catastrophic, hypercane level cyberattacks. Currently, Fable 5 and Mythos are the only Mythos-class models available publicly. Thus, even if other models have high jailbreakability, the risk level is down-regulated by having lesser offensive_capability. Again, this is according to *Anthropic.* Obviously, jail-breaking GPT 3.5
Re: (Score:3, Insightful)
Re: (Score:1)
Wait, are you saying OpenAI is no longer a hype stock since Uncle Elon sued them?
It is not clear why Claude and kin are being singled out above and beyond Copilot, Grok, Gemini, ChatGPT et cetera.
Ask the cabal in the eggshell-beige house...
Re: (Score:1)
Well considering how bad most software is, it might be able to do some damage. But competent humans are much more of a threat.
Re: (Score:2)
I believe you make a valid point here. But you seem to assume that the actions of the administration are based on facts and not done out of vengeance. Given that Trump is a mafia boss, is that assumption correct?
In any case, restricting its use to American citizens only is likely difficult - people entering social security numbers into Claude to get access perhaps? Or requiring national security clearance? And we're just at the beginning of the AI boom, this kind of ban will only be effective for a short t
Re: (Score:1)
> Anthropic made the claims, not me
> According to Anthropic, a jailbroken Fable 5 or Mythos is the apocalypse.
I'm pretty sure you're lying and they never said anything like that. They may have suggested it would be a bad outcome, but "apocalypse" is your own editorializing.
> literally end the whole-wide world overnight with a series of catastrophic, hypercane level cyberattacks.
I don't even know what level of spin this is on, but you're clearly not making the slightest bit of a commitment to the tr
Re: (Score:2)
According to OpenAI in 2019, GPT-2 could be used for all kind of dangerous things. Today people are training GPT-2 models in 2 minutes for fun. Don't believe such arguments, half they are marketing, half they are arguing why the competition should be regulated.
Re: (Score:3)
Eh... I think the "used for all kind of dangerous things" started with GPT-3. I used GPT-2 a good bit back in the day, and it couldn't pass as a person, let alone reason out as a cyberattack.
Since it wasn't instruction fine-tuned, you couldn't trust that GPT-2 would ever do what you wanted. The best you could do is "cute it" with leading text and hoping that it doesn't go off on a tangent. That said, you could kinda use it for things with decent reliability. For example, you could write a long paragraph
Re: (Score:2)
They said they only release small, then only medium, then only large, before they finally released XL (1.5B, very little by today's standards) because it could create fake news. GPT-3 had its own hype, but the whole "too dangerous for you" thing started already with GPT-2.
Re: (Score:1)
You didn't even check the summary...
Anthropic has said that the vulnerabilities like those flagged by Amazon are relatively basic. The company has said that other publicly available models are capable of discovering them
no wonder AI is taking away all the "thinking" jobs.
None of you lazy fuckwits are capable of independent thought any longer.
Brides (Score:5, Insightful)
It's going to be weird if we ever become a democracy again not seeing a headline involving an obvious quid pro quo bribe every day. There are Latin American dictatorships that would take a look at what Trump's doing and say hey buddy, tone it down a bit.
I would take this admins (Score:4, Insightful)
Re: (Score:2)
If some AI can hack your power plant, a clever human can too. Secure your fucking power plant or use at least a firewall.
Re: (Score:2)
Seek help.
Well, Bessent is an idiot, so (Score:2, Interesting)
If Bessent is involved, and taken at all seriously,
then here's no truth or insight to be found with any of them.
Just more banditry.
And???? (Score:1)
nslookup will give you information useful for cyberattacks. lets see Bessent try to make that US only.
Antropic literally asked for this (Score:2)
Anthropic is led by AI doomers and are always asking for ways to save the world from their own products. Anthropic have been campaigning for years to get government regulation in place in the full knowledge that the head of the US state is Donald J. Trump. Only last week, Anthropic were putting out statements that they want a pause on AI development. Now they have the regulation they wanted and they have the pause they wanted. Let's see how they like it.
Re: (Score:2)
I'm personally convinced they're a cutout. They're being used to intentionally push this fear agenda so that the government can crack down and further implement the panopticon.
It's the only reasonable response to them repeatedly saying, "our products are dangerous". They're pushing for government intervention, and for whatever reason, want industry regulation or governmental industry control.
Amazon cut down Anthropic... (Score:3)
...in its Prime.
Public vs Private Agents (Score:2)
If folks are losing their minds over what the public version of the Agent can do,
try to imagine what the non-public ( in house only ) version is capable of.
In a way, the race to AGI is a lot like the race to develop the first working nuclear weapon.
Whomever gets there first will have a significant advantage over those who don't.
Thus, the " damn the torpedoes " attitude most Nations are exhibiting.
I'm curious what unforseen catastrophe this will set into motion :P