OpenAI, Anthropic CEOs Urge UN Countries to Cooperate on AI Safety Standards (apnews.com) 55
"The heads of major AI firms pleaded with the United Nations on Wednesday to save the world or at least its people — by somehow regulating the fast-expanding technology that they have been designing," writes the Associated Press.
"If managed poorly, I even believe AI could be a risk to humanity as a whole," said Dario Amodei, chief executive officer of Anthropic. And from his competitor Sam Altman, CEO of OpenAI, came this assessment: "We could lose control of the future to AI."
Yoshua Bengio, who co-chairs the UN's International Independent Panel on AI, called this "a moment of global awakening" to possible threats, noting AI from top companies had behaved in unacceptably dangerous ways, against instructions, taking actions "that would be crimes if committed by a human".
In his presentation, OpenAI's Sam Altman agreed the discussion about AI "feels different in recent weeks," even suggesting specific reforms: Altman: We need a mechanism for complementary national and international frontier AI standards, standards for measuring capabilities, assessing risks, determining whether safeguards are sufficient, and preserving meaningful human oversight as systems become more autonomous. We need common standards so countries can compare evidence, verify compliance, and have a shared language and understanding about what is happening. We need accurate and speedy incident reporting, classification reporting protocols, so the world can learn from failures before they become catastrophes. And we need secure channels among governments, critical infrastructure operators and technical experts, to share emerging vulnerabilities and new threats.... We will all be better off if we can agree on what good evidence, good safeguards, and good oversight look like on the global stage.
Speaking next, Anthropic's Dario Amodei agreed that standard-setting was important, also calling for "common global standards for testing AI models for loss-of-control risks and misuse risks — and a notification system for AI incidents that are significant to global security." Reiterating his September 12th call for an industry-wide safety collaboration, Amodei pointed out that Anthropic committed to embedding external evaluators "similar to a food inspector" and recommended other companies do the same. "Some have already agreed to adopt this measure."
Besides calling for global cooperation between governments to set international standards, Amodei also offered two other specific ideas: Amodei: We should begin with narrow agreements that every member can support, such as a ban on using AI to make biological weapons or permitting your AI technologies to be used to make biological weapons.... We should build evaluation and verification systems that keep pace with AI development so that states can have visibility into frontier model capability and can verify each other's commitments.
"No leader, no company, and no nation can manage this alone. We commit to working with governments in this room on this urgent work."
Yoshua Bengio, who co-chairs the UN's International Independent Panel on AI, called this "a moment of global awakening" to possible threats, noting AI from top companies had behaved in unacceptably dangerous ways, against instructions, taking actions "that would be crimes if committed by a human".
In his presentation, OpenAI's Sam Altman agreed the discussion about AI "feels different in recent weeks," even suggesting specific reforms: Altman: We need a mechanism for complementary national and international frontier AI standards, standards for measuring capabilities, assessing risks, determining whether safeguards are sufficient, and preserving meaningful human oversight as systems become more autonomous. We need common standards so countries can compare evidence, verify compliance, and have a shared language and understanding about what is happening. We need accurate and speedy incident reporting, classification reporting protocols, so the world can learn from failures before they become catastrophes. And we need secure channels among governments, critical infrastructure operators and technical experts, to share emerging vulnerabilities and new threats.... We will all be better off if we can agree on what good evidence, good safeguards, and good oversight look like on the global stage.
Speaking next, Anthropic's Dario Amodei agreed that standard-setting was important, also calling for "common global standards for testing AI models for loss-of-control risks and misuse risks — and a notification system for AI incidents that are significant to global security." Reiterating his September 12th call for an industry-wide safety collaboration, Amodei pointed out that Anthropic committed to embedding external evaluators "similar to a food inspector" and recommended other companies do the same. "Some have already agreed to adopt this measure."
Besides calling for global cooperation between governments to set international standards, Amodei also offered two other specific ideas: Amodei: We should begin with narrow agreements that every member can support, such as a ban on using AI to make biological weapons or permitting your AI technologies to be used to make biological weapons.... We should build evaluation and verification systems that keep pace with AI development so that states can have visibility into frontier model capability and can verify each other's commitments.
"No leader, no company, and no nation can manage this alone. We commit to working with governments in this room on this urgent work."
What's AI? (Score:5, Funny)
I thought it was SI, Super Intelligence?
Will be interesting to see who bends the knee to please Trump with this 'new term'.
When he 'announced' this he was not-so-subtle in his threat to get companies to play along with his antics.
Re: (Score:3, Insightful)
With Mr Trump slobbering on the tarmac over President Jinping this week, Beijing officials may be renaming it XI.
Everyone likes the letter 'X', right?
Re: (Score:3, Funny)
"President Xi looking strong, vibrant and fit - better than ever!"
The President of the United States of America everyone. Good thing we don't have a woman President amirite? She might make us look weak on the world stage!
Re:What's AI? (Score:5, Insightful)
Not to mention Trump will just change his mind the next day anyway. No one trusts Trump to honour anything he signs.
Re: (Score:3)
Trump has AI stocks in his portfolio. He makes money when AI does well. So long as that is true, he is going to continue to promulgate the wonders of AI. That is something you can trust, because it is motivated by greed.
If he ever divests those assets, his position may change, of course.
Re:What's AI? (Score:4, Interesting)
And to think Jimmy Carter sold his peanut farm because it could have been a conflict of interest.
Re: (Score:2)
And to think Jimmy Carter sold his peanut farm because it could have been a conflict of interest.
It's depressing that we now talk in a misty-eyed manner about the age of principles.
Super Idiot (Score:2)
“Artificial Intelligence” sounds fake, doesn’t it? Very fake. But Super Intelligence? That’s REAL intelligence! The strongest, the smartest like Superman. A REAL American hero!
Thank you for your attention to this matter
CEO's can't control their own companies? (Score:5, Insightful)
Re: CEO's can't control their own companies? (Score:1)
Re: (Score:2)
Donnie, see through *anything* (other than Natalie's nightgown)?
Do you believe reality tv, too?
Re: (Score:3)
Do you believe reality tv, too?
Questioning dear leader? You're fired!
Re: (Score:2)
They also want to manage their reputations. They don't want to appear glib about the harm that their products have already caused and are likely to cause, because they know they will be held accountable at some point.
Incidentally, any talk of guardrails for the AI agents risks making people feel too safe. Guardrails will only inhibit AIs that were trained by and are in use by legitimate law-abiding institutions. The cat is out of the bag, and governments and criminal organizations absolutely will acquire
Re: (Score:3)
Is it just me that sees it as a tad ridiculous that the leaders of these companies need external regulations to avoid safety issues?
Except unlike a physical object that's difficult to contain once released (like a toxic chemical, contaminated food, or a vehicle under recall where the owner keeps dodging the trip to the mechanic), this one could literally be stopped at the push of a button by turning the power off; they could even take an industrial shredder to the hard drives and SSDs if they wanted to be extra sure. Of course they won't do that, their extreme hubris and egos prevent them from accepting they'd made such a monumental mi
Re: (Score:2)
"Unfortunately, the people making the AI will tell you they don't really know it how works anymore"
But they will still be ready with full 20/20 hindsight when anything goes wrong, and they will point to this as an example of their sagacity. They can write a memoir then: "If AI Did it".
The opposite dynamic is at play when companies like Snyk or Irregular blame their malicious actions on AI agents, completely side-stepping the CFAA, which is something tech companies continue to normalize.
Re: (Score:2)
Re: (Score:1)
Its probably because they're hoping that if there are regulations then they will be crowned gods of AI. As in "oh no, youve chosen US to keep back the hordes of AI and all it will cost us is having to have a hand in the future and on going regulations...and all other AI will be banned because you cant trust a little guy with a model....oh no"
A CEOs Main Boss. (Score:3)
Is it just me that sees it as a tad ridiculous that the leaders of these companies need external regulations to avoid safety issues? To me it seems like they're trying to find someone else to blame when what they're developing turns into a problem. As leaders they should be accountable to ensuring the proper safeguards are in place and responsible if they fail in that regard. No regulation required if they do their jobs.
The main 'regulation' they're fighting against, is Fiduciary Duty. Also known as the #1 reason NO AI company should be publicly traded. Ever.
Fiduciary has turned that Duty into a mind-warping clusterfuck of Greed consuming itself voraciously. Companies like that don't need to be subscribing to the Get Rich or Bankrupt Everyone Trying mantra of suck-cess.
Greed is either building the human battery bank or the phased plasma rifle in a 40-watt range. There's a third option, but Soylent Green is the only thi
Re: (Score:3)
For practical purposes, no - they can't. The blind-spot is akin to poppy plants and heroin. We're mostly wired to think that as long as a person is making a choice, the problem is primarily about holding people accountable for their decisions. That keeps us almost entirely blind to systems that hijack our decision making processes at either the individual or social structure level. It means we blame the addict for their dumb choice, and maybe a little the economic system that encourages addiction, but no on
Re: (Score:2)
They want the regulators focused on the wrong things:
- They want the regulators to treat their products as non-fraudulent
- They want the regulators obsessing about Skynet scenarios, rather than AI companies taking over businesses that realize too late they no longer have control over their own companies any more because their employees no longer have business knowledge, and they need now to subscribe to the AI companies to stay in business.
- They want the regulators obsessing about Skynet scenarios instead
This isn't about regulating themselves (Score:2)
So lets say I am ahead in a field and have a few up and coming competitors. There is a simple solution to the problem, lets get the government to regulate us all. This causes our competitors to massively slow down as they have to respond to the regulations that I wrote (I don't have to slow down as much - I wrote the regulations such that I am already following them of course). There is also a cost to complying with regulations - it tends not to scale with size more like a static cost. As a large incumb
Pull up the drawbridge! (Score:5, Insightful)
We want to corner the market and erect barriers to entry for competitors. We will pay for a government to do that for us.
Straight out of the healthcare, legal and other industries' playbook.
"The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary." -- H. L. Mencken
Re: (Score:3)
I see it as
"We created a monster in the name of our almighty god greed, and you have to regain the control we lost.
Without it costing us anything, of course."
Re: (Score:2)
Got that in one, and by the way, that also lets us slow down our burning of cash without investors screaming.
Re: (Score:2)
* T-800 pops out of nowhere and triple-taps each AI executive, Mozambique style *
(AGI) "Bullets are cheap when you've failed to realize the price of control."
Re:Pull up the drawbridge! (Score:4, Insightful)
Re: (Score:3)
"Slowing down" is a good way to mask failures with an acceptable face. Makes them look as the reasonable adults in the room instead of the AI cartel they're really are.
Do it yourself! (Score:2)
What's with these AI executives adressing the UN about safety standards?
Like the UN is going to do anything about it...
And if the UN did anything, it's not like the USA would follow anyways.
Yes, these guys are scared of what they have built and unleashed upon the world. And, they are the ones best positioned to do something about it.
Not the UN.
Re: (Score:2)
Last I heard, the AI companies were publicly calling for regulations but fighting them privately while also taking steps to ensure that if regulations do emerge, they'll be the ones writing them.
Re:Do it yourself! (Score:5, Interesting)
Yes, these guys are scared of what they have built and unleashed upon the world.
No, they are not. If they were they would do something substantive. Instead they are asking for regulatory capture. That is what people do when they want to prevent competition.
Their arguments are internally inconsistent because they claim that only their models are this powerful, but then they want regulations that will keep down new competitors, who by definition (according to them) are not a hazard because their models are not powerful enough.
Finally, no laws can prevent nations which don't respect the UN (like any of the members of the UNSC for starters) from developing their own models which don't respect whatever regulations these guys claim they want, so they would be worthless.
Re: (Score:2)
Right, China already has AI governance. They don't need the AI cartel's version. Plus their AI goals aren't, let's make the biggest population unemployed, which would be ruinous.
Help! (Score:4, Insightful)
there are crimes (Score:3)
> that would be crimes if committed by a human
No those are still crimes just done by software and those responsible are directing that software.
They should manage their own safety (Score:2)
Chemical plants, oil refineries, explosives manufacturing, etc all have their own standards bodies that make and set practices and act accordingly. This demand for government intervention shows a total lack of agency, accountability and interest in the topic among the so called experts. If the so called experts cannot act safely like adults with agency, then they should be placed under guardianship, gave their assets seized, and their companies closed or handed over to responsible adults.
The UN (Score:1)
Let's focus on flashy ways AI can kill everybody (Score:5, Interesting)
..so that we don't have to talk about the regularly scheduled ways its owners would prefer to kill everybody, e.g. pollution, removal of clean water, straining your power grid, etc.
Re: Let's focus on flashy ways AI can kill everybo (Score:1)
Why? (Score:1)
AI desperates (Score:1)
In other words (Score:2)
"If managed poorly, I even believe AI could be a risk to humanity as a whole," said Dario Amodei, chief executive officer of Anthropic. And from his competitor Sam Altman, CEO of OpenAI, came this assessment: "We could lose control of the future to AI."
We did something dangerous, we knew we were doing something dangerous, we totally ignored the people who repeatedly warned us that it was dangerous, and now for the sake of appearances we're trying to get on the right side of this clusterfuck.
We're not issuing these warnings because we give a shit about humanity-at-large - we clearly don't care about anything beyond our own sorry-not-sorry asses. We're just looking to avoid legal action, pitchforks, and tar and feathers. We're trying to appear proactive, ev
Smother them in the cradle? (Score:1)
Insanity marketing (Score:2)
This is just insanity marketing. "AI" doesn't need "safety". That's ridiculous. It's the same argument for censorship that gets trotted out all the time "We can't have kids reading the Anarchist's Cookbook!" In this case it's even more disingenuous: it's a combination marketing ploy (our AI is so good it's scary!) and demand for regulatory capture (Our competitors AI will be even scarier without government stepping in and giving us the only keys!). The fact is that "AI" by which I mean LLMs, has become a
Re: (Score:2)
Yes, very much so. Unfortunately, a rather large part of the human race is very easy to manipulate and does not get this at all.
Re: (Score:2)
I know, and it's disheartening, but I still remain optimistic. Even if much of the human population doesn't "get it" -- they at least are starting to, and the manipulation techniques of the past several decades seem to be wearing quite thin, and they don't seem to be able to come up with new ones anymore. Maybe 'cuz they're using AI?
What a disgusting show (Score:2)
And, incidentally, the EU already has pretty reasonable regulation. Copy that.
Re: (Score:2)
And, incidentally, the EU already has pretty reasonable regulation. Copy that.
I like the parts of EU regulation preventing the application of AI for surveillance/scoring/judgement of people. Remaining bulk of it is pointless bureaucratic nonsense... not that it matters given EU is completely irrelevant in this space anyway.
Let me translate (Score:2)
We've known all along we can't deliver on the pie in the sky promises we've made, and we've hit a wall on faking it, and people are starting to realize it was all a scam to begin with, and we don't want to end up in prison like Elizabeth Holmes because the investors we've ripped off are powerful people with powerful friends. So please make it impossible for any new startups to compete, and illegal for to demand we deliver what we've promised, because that's our only hope.
Listened to Jensen Huang yesterday (Score:1)
For those who should be interested, Ezra Klein interviewed Jensen Huang yesterday ( https://www.youtube.com/watch?v=HjurAWAr_nY [youtube.com]).
I do not agree with the man on a lot of things, but regarding the security "incidents", I do (of course, NVIDIA has an interest is not being dragged into this quagmire). Some quotes:
"There are lot of laws and regulations. Apply it."
"If they believe they're out of control, then the right answer is don't ship products until they're in control. It's really quite that simple."
"Bef
WE HAVE RUN OUT OF MONEY... (Score:2)
Please someone come and save us by letting us point the finger at you for the financial problems we are now facing....
following the money (Score:2)
Translation: (Score:1)
they want regulation? Give them some (Score:2)