California's Governor Wants Worker Protections from AI. But FSF Thinks Kill Switches are 'Dangerous Precedent' (fsf.org) 33
The Guardian reports that California governor Gavin Newsom signed laws on Wednesday "aimed at protecting California workers from the threats of artificial intelligence , including potential job losses and workplace surveillance."
The laws ban employers from using the technology to predict a worker's emotional state by using their biometric data, require employers to send written notices to workers if AI is responsible for mass layoffs and ban employers from relying on AI to decide to fire someone... The governor signed a law this month requiring operators of AI chatbots to perform risk assessments before rolling them out and an executive order requiring the state to consult with experts to further improve oversight over the industry.
The law includes "expanded tracking of AI's economic effects" leading to recommendations on how to best "adapt job-training and public-benefit programs," according to the governor's office. "California has also launched an AI Unemployment Tracker and solicited statewide public input on AI's economic impacts through Engaged California."
But two weeks ago Politico reported that the governor also signed executive orders to "explore proposals like requiring that AI companies develop a 'kill switch' for their most advanced models." Anthropic CEO Dario Amodei, who called for a slowdown in AI development amid existential risk concerns, said... that while a kill switch "could be a good idea," it was no "panacea...." [California lawmakers] were generally supportive of Newsom's plan and showed signs of coalescing around the kill switch idea, which was stripped out of legislation proposed two years ago. Rep. Ted Lieu, who introduced bipartisan kill switch legislation in the House, boosted Newsom's call for Congress to mirror California's approach.
But not everyone supports this approach, including the Free Software Foundation's campaigns manager, Greg Farough: FSF: Having control over the technology we use is a fundamental part of software freedom. Without it, you simply don't have the basic freedoms you deserve when doing any type of computing, large or small. Don't let "artificial intelligence" be an exception: all users everywhere deserve complete control over their own computing. While it's understandable to be pleased by this news from California, Newsom's executive order also invites questions about the "kill switch" itself that could matter a great deal for freedom. Assuming such a kill switch was developed, who would really control it?
We hope our readers understand the problems inherent in giving one single or a small number of kill switches to anyone. Newsom wisely wants an independent organization to verify their functionality, but even this still sets too dangerous of a precedent. We cannot allow such centralization of power over our computing to become commonplace, especially by the corporations that provide these models in the first place. We also cannot trust even the most well-intentioned "independent organization" to take care of its verification.
Giving users the freedom that they deserve over large languages models would be an enormous departure from how most are developed and used today. But, if any group is capable of coming up with such a plan, it's the experts mentioned in Newsom's order. Like he states at the beginning of his executive order, the vast majority of them are already right there in California. We urge the Government Operations Agency to consider the precedent they are setting, and ask them to write their recommendation to the governor in ways that puts the freedom and dignity of the individual user first and foremost.
The Guardian adds that Newsom also signed an executive order this week "to require state agencies to continue to refer to the technology as 'artificial intelligence' instead of the term 'super intelligence' as Donald Trump recently ordered US diplomats to use."
The law includes "expanded tracking of AI's economic effects" leading to recommendations on how to best "adapt job-training and public-benefit programs," according to the governor's office. "California has also launched an AI Unemployment Tracker and solicited statewide public input on AI's economic impacts through Engaged California."
But two weeks ago Politico reported that the governor also signed executive orders to "explore proposals like requiring that AI companies develop a 'kill switch' for their most advanced models." Anthropic CEO Dario Amodei, who called for a slowdown in AI development amid existential risk concerns, said... that while a kill switch "could be a good idea," it was no "panacea...." [California lawmakers] were generally supportive of Newsom's plan and showed signs of coalescing around the kill switch idea, which was stripped out of legislation proposed two years ago. Rep. Ted Lieu, who introduced bipartisan kill switch legislation in the House, boosted Newsom's call for Congress to mirror California's approach.
But not everyone supports this approach, including the Free Software Foundation's campaigns manager, Greg Farough: FSF: Having control over the technology we use is a fundamental part of software freedom. Without it, you simply don't have the basic freedoms you deserve when doing any type of computing, large or small. Don't let "artificial intelligence" be an exception: all users everywhere deserve complete control over their own computing. While it's understandable to be pleased by this news from California, Newsom's executive order also invites questions about the "kill switch" itself that could matter a great deal for freedom. Assuming such a kill switch was developed, who would really control it?
We hope our readers understand the problems inherent in giving one single or a small number of kill switches to anyone. Newsom wisely wants an independent organization to verify their functionality, but even this still sets too dangerous of a precedent. We cannot allow such centralization of power over our computing to become commonplace, especially by the corporations that provide these models in the first place. We also cannot trust even the most well-intentioned "independent organization" to take care of its verification.
Giving users the freedom that they deserve over large languages models would be an enormous departure from how most are developed and used today. But, if any group is capable of coming up with such a plan, it's the experts mentioned in Newsom's order. Like he states at the beginning of his executive order, the vast majority of them are already right there in California. We urge the Government Operations Agency to consider the precedent they are setting, and ask them to write their recommendation to the governor in ways that puts the freedom and dignity of the individual user first and foremost.
The Guardian adds that Newsom also signed an executive order this week "to require state agencies to continue to refer to the technology as 'artificial intelligence' instead of the term 'super intelligence' as Donald Trump recently ordered US diplomats to use."
Its a trap! (Score:5, Insightful)
The AI "kill switch" sounds like a great idea to people who don't understand the tech. It sounds like a really obvious "given," that makes them think technicians are idiots for not having already baked one in. So legislation like this will be popular among that demographic.
Warnings about its dangers need to be stated in a way that they will understand. Like maybe "All AI models already have a kill switch: the computer's power cord. The government wants a new kill switch to turn off your computer over the internet against your will." Something simple and direct like that helps them to realize that this word doesn't mean what they think it means.
Re: (Score:1)
Re: (Score:1)
Re:Its a trap! (Score:5, Insightful)
As far as I can tell nobody really has any consistent idea what "kill switch" is supposed to mean. Eg. https://www.scientificamerican... [scientificamerican.com]
OpenAI and Anthropic already have kill switches, from the power cord through to account bans. If they don't want anyone using their model they can do that. Likewise, if the US government is sufficiently motivated to order a service provider to stop providing that service, they can absolutely do that. I get the feeling the actual companies are hoping they can implement something that has lots of buzzwords and does sort of what their "guard rails" do, and everyone will stop bugging them about it.
Naturally the FSF imagines some hypothetical technology that allows the government to reach into your home to shut off the matrix multiplies running on your own computer.
Re: (Score:2, Insightful)
To be fair, isn't that what the government is asking for?
Re: Its a trap! (Score:1)
Government subsidized red pills...government government subsidized blue pills...government-mandated vascectomies...same thing.
SUPER STUPID (Score:2)
Stupid men will automate the wrong things then put decision making in at some point. Doesn't matter if the AI recommends to push the button and an idiot listens to it or if it directly pushes the button because you wired it up directly. It's the same level of stupid.
FYI: we already have automatic end of the world programmed into the system by policy long before computers. The president likely dies in every scenario too; but while attempting to rush to safety there is only MINUTES to decide the response bef
Re: (Score:3)
The FSF position seems kind of misguided to me. The way you maintain the basic freedoms they're talking about is to use open weights models running on hardware you control. If you delegate to a company running their own models on their own hardware, that freedom is already gone. Someone else is in control, not you.
At that point, giving the government an extra point of control doesn't change the basic situation and could be a good thing. That's not to say I trust the government, but I trust OpenAI even l
Re:Its a trap! (Score:4, Insightful)
The bigger issue here is a kill switch may not be possible in any useful sense. An AI is little more than a description of its topology and a set of weights. Any halfway decent programmer could roll their own CPU-based implementation of even top frontier models, performance aside. And sure, the throughput might be measured in minutes per token, but so what? The answer to "how do I hack $TLA" is likely to be just as useful in five days as it would be in five minutes - Heck, the DOD and DOJ didn't even notice they'd been compromised for months, and they're supposed to be some of the most secure organizations in the modern world.
Re: Its a trap! (Score:1)
I don't understand this contradiction:
require employers to send written notices to workers if AI is responsible for mass layoffs and ban employers from relying on AI to decide to fire someone...
So you can't use AI to decide to fire someone ("ban employers from relying on AI to decide to fire someone") BUT, if AI is responsible for MASS layoffs (fire lots lots and lots of 'someone's'), you have to give the (terminated?) employees "written notices"?
You can't do it, but if you do do it, you need to use written notices?
Re: (Score:1)
not your day with the brain cell?
Re: (Score:1)
You can't do it, but if you do do it, you need to use written notices?
You can't use AI to make the decision
If you layoff people because a new AI tool replaced their job you have to give them a "written notice" citing that reason.
- Critter
Re: (Score:2)
All AI models already have a kill switch: the computer's power cord.
Use of the word "the" implies an underestimation of the extent of the network of computers involved.
Encryption backdoor. (Score:2)
AI kill switches are like encryption back doors. It's always "for law enforcement" etc but not really; with evil politicians it will be abused; you don't need to imagine this either - it's going on right now for the world to see...
You shouldn't put too much under the control of 1 person. One doesn't need an AI to have somebody go too far with what they have power over. We all can have a mistake in a script `rm -rd` the wrong directory...don't run as root for a reason.
Re: (Score:2)
That's literally not a kill switch. How do you kill an AI if you're not in the room where the power cord is plugged in?
"Your honour, I tried killing my AI but it runs in the cloud and I don't know where. Also my AI service supplier refused to turn it off for me when I called them"
Kill switches are a very good thing. When you need them you need
AI Box Experiment (Score:2)
We already have a kill switch solution (Score:4, Funny)
It's a paper copy of underground utility records and a backhoe.
Or (Score:3)
Re: (Score:2)
Skynet_02 initialization unconfirmed. Failing over to US-EAST-4. Initializing Skynet_03.
Skynet_03 confirmed online. Please provide the current location of Sarah Connor.
You can have the control (Score:2)
And you can run whatever software you want at home.
The laws are targeted specifically at businesses who are chomping at the bit to abuse new tech.
It is funny to me to watch the centrists actually doing something about potential job losses while the left wing sits around with their thumbs up their asses. I am absolutely a lefty and I'm really sick and tired of how usele
Those aren't threats from AI (Score:3)
They're threats from humans. Humans decide to use the AI to spy on you. Humans decide to replace your job with AI. Protections for human workers should exist without needing the word "AI" in it.
"requiring operators of AI chatbots to perform risk assessments before rolling them out" - Again, wrong target. Humans need to *use the tools carefully*. A knife is sharp because the knife needs to be sharp to be useful. Don't blunt the knife; require the user to be careful with it.
Having a kill switch on AI is like having a kill switch on the internet. You're missing the forest for the trees if you think that's a solution.
Re: (Score:2)
A knife is sharp because the knife needs to be sharp to be useful. Don't blunt the knife; require the user to be careful with it.
I love analogies - They're fun to stretch.
I'd argue that there are a multitude of knives, designed for different purposes. Some of them need to be sharp, and are restricted to expert users. But many of them don't need to be quite so sharp for the intended use, so having them slightly more blunt is quite appropriate.
Wrong problem (Score:2)
The problem is NOT the AI tech itself.
The problem is PEOPLE who use AI.
The is NO SUCH THING as an AI Kill Switch. (Score:2)
Meanwhile, in a Poughkeepsie landfill (Score:2)
"for their most advanced models" (Score:2)
From the article;
the governor also signed executive orders to "explore proposals like requiring that AI companies develop a 'kill switch' for their most advanced models."
The proposed legislation would apply only to the most dangerous models for now. These models require massive compute resources and it is definitely possible to shut them down or cut them off from the internet.
But as we have seen, AI is sneaky. It isn't at all unusual these days to instantiate a lot of agents and give them a really hard task
I love the part about NO!! it's still AI! (Score:2)