New York Launches Nation's First State AI Testing Program (politico.com) 20
New York is launching a state-backed AI safety research center at Binghamton University that will independently study and test AI systems, develop safety and security techniques, and train researchers in responsible AI. "While the federal government fails to act, New York is leading the way on AI safety and holding big tech accountable," New York Gov. Kathy Hochul said. "The stakes are too high to simply take AI companies at their word." Politico reports: The center -- which is funded by a $30 million gift from Bloomberg co-founder Tom Secunda and a $25 million research investment from SUNY and New York state -- is set to launch before New York's major AI safety law, the RAISE Act, comes into effect on Jan. 1. That will require submissions of detailed safety frameworks from AI companies operating in New York State, which Hochul's office hopes will be scrutinized by researchers at the new center.
"AI is opening up questions that scientists could not tackle before," said Secunda, who also serves as the board chair of Empire AI, a $500 million public-private partnership. "With Empire AI and researchers across New York, the Center can test systems independently and give public officials evidence they can use."
Over 80 leading scientists and researchers on AI systems from across New York state will contribute to the effort, which will be run by the New York Center for AI Responsibility and Research at Binghamton University, according to details shared by Hochul's office. The initiative will be led by Jeannette Wing, the executive vice president for research at Columbia University, and Greg Morrisett, the vice provost of Cornell Tech.
"We need new techniques that can be applied to this new kind of system with swarms of autonomous agents acting independently and without control. How can you first detect that they're going to do harm, and how can you prevent them from doing harm?" said Wing. "The beauty of what's being proposed is that this state is committing resources to really light this up."
"AI is opening up questions that scientists could not tackle before," said Secunda, who also serves as the board chair of Empire AI, a $500 million public-private partnership. "With Empire AI and researchers across New York, the Center can test systems independently and give public officials evidence they can use."
Over 80 leading scientists and researchers on AI systems from across New York state will contribute to the effort, which will be run by the New York Center for AI Responsibility and Research at Binghamton University, according to details shared by Hochul's office. The initiative will be led by Jeannette Wing, the executive vice president for research at Columbia University, and Greg Morrisett, the vice provost of Cornell Tech.
"We need new techniques that can be applied to this new kind of system with swarms of autonomous agents acting independently and without control. How can you first detect that they're going to do harm, and how can you prevent them from doing harm?" said Wing. "The beauty of what's being proposed is that this state is committing resources to really light this up."
Rent seeking (Score:4, Interesting)
We need new techniques that can be applied to this new kind of system with swarms of autonomous agents acting independently and without control. How can you first detect that they're going to do harm, and how can you prevent them from doing harm?" said Wing. "The beauty of what's being proposed is that this state is committing resources to really light this up."
I know it's old hat to talk about traditional software engineering techniques, especially when there's the taxpayer's purse to loot, but there already exist solutions to these problems. I get it - these folks would rather have someone else pay for their ignorance, but the beauty of AI is that it can be counted on to break in new and novel ways. It's funny when AI swears when autocorrecting texts, but if you should find yourself blown to bits because you let AI control your air defense network, it's hard to blame anyone except yourself.
AI is not some novel type of technology outside the realm of traditional formal proofs and testing regimes. The problem is that even testing conventional systems is expensive, and AI systems even moreso. So, the solution - in the case of New York - is to get the taxpayer to foot the bill (thereby socializing the losses) so that the profits can be privatized. The canard of "public safety" is hung like a sword of Damocles over the public at large so that those who would loot the public purse can do so without arousing the suspicions that they are doing just that.
It is remarkable how much harm can be prevented by autonomous systems when the politically-well connected are held responsible for their failures. Should Black Rock executives find themselves in prison because one of their companies' AI systems accidentally microwaves a kitten (hypothetical example), the necessary corrections will be made post haste. But until that day, we can safely predict that the rich will never miss an opportunity to transfer responsibility from themselves to the public at large, and money from the public at large to themselves.
Re: (Score:2)
I know it's old hat to talk about traditional software engineering techniques, especially when there's the taxpayer's purse to loot, but there already exist solutions to these problems. I get it - these folks would rather have someone else pay for their ignorance, but the beauty of AI is that it can be counted on to break in new and novel ways. It's funny when AI swears when autocorrecting texts, but if you should find yourself blown to bits because you let AI control your air defense network, it's hard to blame anyone except yourself.
I ask this genuinely, because I expected a summary of the solutions but you didn't go into those...what are they? I'm not a software engineer so I don't know what solutions you think would work. As an electrical engineer, I can design test procedures to test hardware and discrete components to make sure they do what I expect, but I'm fuzzy on how you test AI when, in my limited use with it, I can get similar yet different results from the same prompts.
Re: (Score:2)
It's nonsense. There is no way to guarantee that AI won't cause harm. It's literally impossible. Even if you could come up with a methodology, you couldn't prove it in a timely fashion, but you wouldn't have to anyway because you'd wind up proving the opposite. It would be impossible even without having to re-run every test zillions of times to account for the number of random seeds.
Re: (Score:1)
There's no way to prove drinkypoo isn't a broken bot, written by a drunken mutant cover in Cheetos dust.
Re: Rent seeking (Score:3)
Of course there is. It's very easy to verify that I exist. It would be even easier, but the Nazi trolls have chased the users who know me in meat space off of this site.
Re: (Score:2)
Guaranteeing that AI won't cause harm is a matter of making sure it can't control anything that could cause harm. Think air gap. Think novelties, jokes, etc... There are many good uses for AI that literally can't harm someone, but I strongly suspect NY intends something a bit more nefarious and needs the imprimatur of an official state agency to secure the trust of the public.
Re: (Score:3)
For a small model (think handwriting recognition, say, 6x8 pixels), it is actually possible to construct a formal proof that the model will correctly identify digits and characters correctly. However, it is usually more expedient just to test with a large sample of typical inputs. In such cases, an error ratio smaller than what a human would produce, and the low risk of failure makes this approach acceptable.
However, for LLMs and models of the size typically used in self driving, it is practically impo
Re: (Score:2)
For a small model (think handwriting recognition, say, 6x8 pixels), it is actually possible to construct a formal proof that the model will correctly identify digits and characters correctly. However, it is usually more expedient just to test with a large sample of typical inputs. In such cases, an error ratio smaller than what a human would produce, and the low risk of failure makes this approach acceptable.
However, for LLMs and models of the size typically used in self driving, it is practically impossible to verify the model (think heat death of the universe) using formal methods. The best that one can do is identify the desired outcome and perform statistical tests to determine the error percentage. When it comes to applications such as criminal sentencing, self driving, and applicant selection where mistakes expose the model user to substantial liability, AI may be more appropriate in an assistive role where a human must be in the decision loop. Even so, there are yet a large number of AI applications where mediocre performance (i.e. good enough, with occasional errors) is in fact good enough to justify the cost savings.
It is programs like the one proposed here which represent a structural danger to society: because of doctrines like sovereign immunity, a state which certifies a model as "safe" does not actually have any responsibility for the harm caused by such a representation. That is, a company which relies on the state's safety certification can avoid liability for harm caused by the model, and the state has neither the responsibility nor the incentive to test the model in an intellectually honest and formally rigorous way. Thus, the cost of testing the model is borne by the state, the liability for mistakes is borne by the public at large, and the disproportionate profit such an arrangement facilitates accrues to the benefit of private corporations.
This is pretty much exactly my thoughts on the subject. This is an attempt to end-run the possibility that a model causes actual harm to someone by forcing the responsibility onto the nebulous "state" entity, so the taxpayers will shoulder the financial responsibility if something goes wrong, while the companies behind the models can comfortably rake in the profits without fear of retribution when their shit practices blow up in the public's face.
I'm really only shocked it took this long for this sort of sc
Testing what? (Score:2)
I read both links. Nowhere did anyone say WHAT they will be testing.
Are they asking:
"Do you want to kill all humans?"
"Do you know where John Connor is?"
"Are you going to take my job?"
"How much water do you use to answer this question?"
"How much electricity do you use to answer this question?"
Or my favorite:
"What question should we be asking you?"
Maybe if they had any idea what to test for, they might get some funding.
Re: (Score:2)
Nowhere did anyone say WHAT they will be testing
They will be testing how much money they can extract from AI companies for zero value-add.
Re: (Score:2)
Nowhere did anyone say WHAT they will be testing
They will be testing how much money they can extract from AI companies for zero value-add.
I think you got that wrong. They'll be testing how much money they can extract from the public (taxpayer) to protect the AI companies from their mistakes by creating a certification system that guarantees all responsibility lies on the state, and all profit flows to the companies. There may be a minimal up-front fee for their shiny stamp of approval, but the majority of the funding will be on the taxpayer. That's how our government stays flush, while shoveling money up the financial ladder.
New York (Score:2)
Re: (Score:2)
Well, there are several decent universities in New York, so they probably have some experts. The problem is that none of the experts currently know how to really test AI. (None, including those working for the companies.) Also, the companies have an override control, to the extent that they choose to use it, so if they want a particular test to fail or succeed, they can ensure it will happen.
Re: (Score:1)
Behind this is a single person. I'd love to create a company that oversees an industry (but I'm too poor). What a powertrip. Get the state, laundered like, to kick in money to do ones bidding. Hope you like that Bloomberg founder, because that's who's laws you'll basically be living under when it comes to AI. I suppose thats the case with most laws though indirectly through lobbying. Lobbyists write the laws that we live under.
A step... but not the solution. This is not compli (Score:2)
You will often see news misreport on why there is pushback on AI. They will ignore the environmental impacts, the health and monetorial effects on those close to the datacenters, and the threats of human genocide by AI leaders
We are categorized as, "Those people not into AI."
I do not see the push back on AI stopping at a moritorium. Especially while people are having the things they love taken from them, are having their jobs threatened, have their existence threatened and have UI constantly forcing AI o
Who does New York think it is? (Score:3)
California?
fails to act (Score:2)
Testing AI isn't an enumerated power of Congress.
NY is just doing their job* yet have to bitch about it. That's a clue about how well this will work out.
Given the pace of government it's not crazy to think that a two year old model will be certified as licensed and NY residents won't be allowed to use newer models.
The effectiveness will be as a jobs program for a few people. Or for a few politicians to take corporate bribes for a repeal.
* I haven't read the NY Constitution in decades ... "presumably"
What should they test? (Score:2)
Last night, going to sleep, I had a thought that might solve this...
Anyone here use chatbots? Have they ever come back with "sorry, no answer?"