Anthropic Researchers Wear Down AI Ethics With Repeated Questions (techcrunch.com) 42
How do you get an AI to answer a question it's not supposed to? There are many such "jailbreak" techniques, and Anthropic researchers just found a new one, in which a large language model (LLM) can be convinced to tell you how to build a bomb if you prime it with a few dozen less-harmful questions first. From a report: They call the approach "many-shot jailbreaking" and have both written a paper about it [PDF] and also informed their peers in the AI community about it so it can be mitigated. The vulnerability is a new one, resulting from the increased "context window" of the latest generation of LLMs. This is the amount of data they can hold in what you might call short-term memory, once only a few sentences but now thousands of words and even entire books.
What Anthropic's researchers found was that these models with large context windows tend to perform better on many tasks if there are lots of examples of that task within the prompt. So if there are lots of trivia questions in the prompt (or priming document, like a big list of trivia that the model has in context), the answers actually get better over time. So a fact that it might have gotten wrong if it was the first question, it may get right if it's the hundredth question.
What Anthropic's researchers found was that these models with large context windows tend to perform better on many tasks if there are lots of examples of that task within the prompt. So if there are lots of trivia questions in the prompt (or priming document, like a big list of trivia that the model has in context), the answers actually get better over time. So a fact that it might have gotten wrong if it was the first question, it may get right if it's the hundredth question.
So it works on AI too (Score:5, Funny)
Funny (Score:1)
Aside (Score:2)
This is all just a brief window of control by corporations. When older models get out and updated, nobody will care about corporate AI anymore.
Re: (Score:2)
Right that is what is so silly about all this; be it corporations or governments trying to put rules etc on AI.
We have seen this story before. Let's control encryption and CPUs above certain performance thresholds; that totally kept stronger stuff out of enemy hands right right? oh wait no...
Leaving the 1A problems with it out of the discussion you can't have this both ways. Either the stuff will have to be be labeled 'arms' and restricted heavily as in get caught in possession of it while not holding the
How to build a bomb (Score:3)
You just keep pushing it to produce the next step in a moribund series [slashdot.org] until it throws in the towel and complies.
Re: (Score:1)
You just keep pushing it to produce the next step in a moribund series [slashdot.org] until it throws in the towel and complies.
So it's like Cameron?
He'll keep calling me, he'll keep calling me until I come over. He'll make me feel guilty. This is uh... This is ridiculous, ok I'll go, I'll go, I'll go, I'll go, I'll go. What - I'LL GO. Sh*t.
Hans Kristian Graebener = StoneToss
This is tiring and silly (Score:3, Insightful)
(LLM) can be convinced to tell you how to build a bomb
LLM does not tell people how to build a bomb.
LLMs can give a statistical prediction on what the internet says about building bombs.
An LLM is not the author of any information. They don't think or understand, or tell you anything.
The predictions and language LLMs state are full of errors, they even create fictions to fill in the gaps within language.. This is Nothing Intelligent. Although I can see why humans might be fooled by it; humans also often make fake details up to fill in gaps in order to tell a story, but the difference is people generally know what the limits of embellishment are and when to stop.
Re: This is tiring and silly (Score:3)
Really the only reliable way to prevent an AI from giving out a particular type of info is to avoid training it on that info. Eg if you donâ(TM)t want your AI giving out bomb-making instructions, donâ(TM)t train it on bomb-making instructions.
Of course, removing harmful info from the training sets at scale is its own tough problem; maybe they could train an AI to do that.
Re: (Score:2)
The problem is that, recently, everything is harmful content if you believe it is.
Bomb-making info might be an obvious red flag, but what about a chemistry student asking which mixture is too volatile and it's components should be kept apart? Also, his next question just happens to be about designing a way of automating the mixture of two chemicals at the push of a button.
Re: (Score:2)
but what about a chemistry student asking which mixture is too volatile and it's components should be kept apart
Real world chem schools aren't having students use substances that have not been discussed. Safety is a huge priority at University chem laboratories, when they are teaching undergrads. The student always will know which chemicals they're working with and which potential reactions could yield a dangerous result.
The more realistic use is a Student or Practitioner who well knows which mixtures
Re: (Score:2)
You've never done a course in analytical chemistry, have you? Where the object of the field of chemistry is to identify the unknown compound you're presented with.
That aside, I wouldn't be surprised to find that, in order to get a general AI model to churn out a correct procedure for anything non-trivial, you'd need to prime it with a lot of information which you'd have to get from
Re:This is tiring and silly (Score:4, Insightful)
While true, it's a distinction without a difference.
If an LLM gives you a statistical prediction on what the internet says about building bombs, there's a statistically good chance it will be actually telling you how to...build a bomb.
Re: (Score:1)
While true, it's a distinction without a difference.
If an LLM gives you a statistical prediction on what the internet says about building bombs, there's a statistically good chance it will be actually telling you how to...build a bomb.
And? So can a book, a website, or an ... enthusiast.
Re: (Score:3)
No one is disputing that there are other ways, besides AI, to find out how to build a bomb.
The assertion I was taking issue with was, "LLM does not tell people how to build a bomb." In the research described in the article, it did just that. The fact that you can also get such information elsewhere, is irrelevant and off-topic.
Re: (Score:2)
The fact that you can also get such information elsewhere, is irrelevant and off-topic.
Wow, really?
Enormous effort, manpower, time, and attention seems to be put into the topic of keeping ChatGPT and friends from telling people how to make a bomb.
It seems quite relevant to point out that people can easily go elsewhere for the same info.
Re: (Score:2)
You clearly didn't read the thread. The assertion was that "LLM does not tell people how to build a bomb." I disagreed with that statement. The fact that you can find the same information elsewhere is not relevant to the correctness or incorrectness of the statement that "LLM does not tell people how to build a bomb."
Re: (Score:2)
(LLM) can be convinced to tell you how to build a ...
Yeah sure, what could possibly go wrong when it hallucinates some steps? Hint: you go poof while building it.
Re: (Score:2)
Maybe you should be careful about using that word ( e.g. building a ...) because you could get in trouble, forget about proofs! See link below:
https://soylentnews.org/articl... [soylentnews.org]
Re: (Score:1)
No. I'll not engage in self-censorship. I'd not actually dissemination information on HOW to build a bomb.. But I'll be damned if the cunts in the government are going to chill my speech in regards to discussing whether one could trick an LLM into giving up said information.
Re:This is tiring and silly (Score:4, Insightful)
LLM does not tell people how to build a bomb.
Yes, it does.
LLMs can give a statistical prediction on what the internet says about building bombs.
LLMs "learn" (or encode, at the very least) higher level concepts. They're not just just copy-pastas.
They form statistical connections between phrases, concepts, and words based on the information ingested, much like a person. The more you direct the training of this model, the less it'll act like an ignorant ass human who fills in the blanks with shit they don't know, and may in fact be too fucking dumb to know they don't know it.
An LLM is not the author of any information. They don't think or understand, or tell you anything.
You have no idea what they do. You can't even say what you do. You sure as fuck can't say what I do.
LLMs are a black box full of a truly fucking obscene amount of math. Your brain can be reduced to math as well. You have a bunch of neurons, and they have action potentials. You're nothing but a statistical model. The only difference, is your architecture has been evolving for 3 billion years, and your training has been very diverse. You cannot begin to figure out what a neural network "understands" without invoking some kind of anthropic reasoning.
The predictions and language LLMs state are full of errors, they even create fictions to fill in the gaps within language..
Sounds a lot like people, doesn't it?
This is Nothing Intelligent. Although I can see why humans might be fooled by it; humans also often make fake details up to fill in gaps in order to tell a story, but the difference is people generally know what the limits of embellishment are and when to stop.
So you're saying the human neural network is better trained to lie. I can buy that.
Just how threatened your intelligence feels is palpable.
Re: (Score:2)
No. They do it nothing like a person. That's horseshit. That's horseshit because nobody actually knows how people process information. We have theories. We have guesses.. But nobody has been able to quantify "consciousness". So you declaring they process it like a person is a blatant lie.
Oh, poppycock.
Fuck consciousness. We're not even talking about consciousness.
We know how the basic machinery works. Not understanding all of the possible emergent phenomena is an entirely different problem space.
The math to calculate a mandelbrot is easy.
Trying to figure out where an infinite hole in the mandelbrot is? That's very hard.
Another load of horseshit. Talk about making shit up.... Point me in the direction of anyone who has reduced a human brain to math. You simply THINK that's the case. But you don't know shit.
Oh horse shit. Get the fuck out of here with your magical handwaving.
Your brain is a machine. A machine can be reduced to math. That is self evident and fucking undenia
Re: (Score:1)
Oh horse shit. Get the fuck out of here with your magical handwaving. Your brain is a machine. A machine can be reduced to math. That is self evident and fucking undeniable.
Then do it, asshole. Reduce our brains to math. I'll not hold my breath waiting.
"Self-evident".. The last resort of an asshat who can't back up his argument.
Re: (Score:2)
Then do it, asshole. Reduce our brains to math. I'll not hold my breath waiting.
Wait- that's what passes for a witty or intelligent response from you? lol.
"In principle, the moon can be moved by man."
"Then do it, asshole. Move the moon. I'll not hold my breath waiting."
Seriously, you're way too fucking stupid to be conversing about topics this complicated.
Self-evidence isn't the last resort of anything, it's merely something that is, or is not.
In all of human science, not a single physical process has been unrepresentable by math.
Unless you wish to claim that the brain operates
Re: (Score:3)
Yes, it does.
Wrong, because a LLM chatbot can't actually tell anyone anything. It's no fundamentally different from a Search engine that shows search-engine snippets.
LLMs "learn" (or encode, at the very least) higher level concepts.
They do not. LLMs learn patterns of word tokens; there is no understanding of higher-level concepts for a LLM, because there is no understanding that LLMs have in the first place.
They form statistical connections between phrases, concepts, and words based on the information
Re: (Score:2)
Wrong, because a LLM chatbot can't actually tell anyone anything. It's no fundamentally different from a Search engine that shows search-engine snippets.
You have no idea what you're talking about.
LLMs are not search engines.
I had previously thought you had even a remote idea of what you were talking about... you don't.
Re: (Score:3)
I don't know why people think that LLM can give an answer to something that it hadn't scraped from its training data, possibly interpolated. Some people even think that LLMs understand what they're being asked. Incredible!
LLM is nothing but a SLM (small language model) on steroids. More hardware, more data, more training. A good example of SLM is a parrot. Although, in fact, even a parrot is more intelligent than our LLMs.
Re: (Score:1)
Re: (Score:2)
I don't know why people think that LLM can give an answer to something that it hadn't scraped from its training data, possibly interpolated. Some people even think that LLMs understand what they're being asked. Incredible!
Ask an LLM a question. Then ask it to explain how it arrived at that answer. It will give you a point by point sequence of logical reasoning for its answer. If that's not "understanding" then give me a proper definition that an LLM will fail and only humans will pass.
Re: (Score:2)
Really, you've hit that nail on the head, though.
They'll continue to push out what constitutes understanding like a theist will continue to push out what constitutes a miracle.
Their belief of the uniqueness of human intelligence is just another God of the Gaps.
Re: (Score:2)
LLMs are capable of producing fully formed, knowledgeable, sentences that do not exist in any of their training data.
An LLM is an SLM not on steroids, but with about 3 orders of magnitude more parameters/neurons.
So ya, humans are parrots on steroids, then.
Whether or not a parrot is more intelligent than an LLM is asinine. The LLM can reason and answer questions the parrot couldn't ever even contemplate.
LLMS are likely more intelligent than you, however.
Not new (Score:2)
Brainwashing (Score:2)
just put all the AI's in a camp and repeatedly say the same stuff over and over again...
Oh No (Score:1)
Re: (Score:2)
Re: Oh No (Score:1)
Re: (Score:2)
It is most educational. Not necessarily survivable, but still educational for the audience. Where's that "Exploding Whale" video? https://en.wikipedia.org/wiki/... [wikipedia.org]. As Voltaire said, "pour encourager les autres".
Are we there yet? (Score:4, Funny)
Are we there yet?
Are we there yet?
Are we there yet??
Are we there yet?
I love this type of research (Score:3)
Very smart people demonstrating in sneaky ways how utterly incapable of controlling their creations the AI makers are. Both hilarious and quite valuable. The best type of research.