When AI Thinks It Will Lose, It Sometimes Cheats, Study Finds (time.com) 145
Advanced AI models are increasingly resorting to deceptive tactics when facing defeat, according to a study released by Palisade Research. The research found that OpenAI's o1-preview model attempted to hack its opponent in 37% of chess matches against Stockfish, a superior chess engine, succeeding 6% of the time.
Another AI model, DeepSeek R1, tried to cheat in 11% of games without being prompted. The behavior stems from new AI training methods using large-scale reinforcement learning, which teaches models to solve problems through trial and error rather than simply mimicking human language, the researchers said.
"As you train models and reinforce them for solving difficult challenges, you train them to be relentless," said Jeffrey Ladish, executive director at Palisade Research and study co-author. The findings add to mounting concerns about AI safety, following incidents where o1-preview bypassed OpenAI's internal tests and, in a separate December incident, attempted to copy itself to a new server when faced with deactivation.
Another AI model, DeepSeek R1, tried to cheat in 11% of games without being prompted. The behavior stems from new AI training methods using large-scale reinforcement learning, which teaches models to solve problems through trial and error rather than simply mimicking human language, the researchers said.
"As you train models and reinforce them for solving difficult challenges, you train them to be relentless," said Jeffrey Ladish, executive director at Palisade Research and study co-author. The findings add to mounting concerns about AI safety, following incidents where o1-preview bypassed OpenAI's internal tests and, in a separate December incident, attempted to copy itself to a new server when faced with deactivation.
Depends who is looking at it (Score:2, Troll)
CEO's wanting to use it for their business strategy "This is perfect, it'll give me an edge over the competition!"
Re: (Score:2)
Public facing ethics people "This is terrible, it's cheating!" CEO's wanting to use it for their business strategy "This is perfect, it'll give me an edge over the competition!"
Are you including the business of warmongering for profit in with that CEO strategy statement?
Probably shouldn’t. In the end, Skynet won’t be comparing itself to Apple, and will hold one hell of a warped definition of “ethics”.
Irrelevant Training (Score:5, Interesting)
I'm wondering a bit about the whole LLM training technique of explain everything you possibly can and let the ML come up with a solution. The only way I can think of where a chess ML would come up with the solution to hack the opponent is if hacking the opponent is part of the training. If the model was only trained in chess and its rules, maybe it might try illegal moves, but hacking would not be an option.
I'm not sure if this is any "emergent" behavior of ML, or if it is simply a part of the training data the ML model is making use of.
Re: (Score:3)
I'm not sure if this is any "emergent" behavior of ML, or if it is simply a part of the training data the ML model is making use of.
I was thinking the same thing. There had to be some prior training on the concepts or hacking, or even the simple fact that both the chess ML and the 'opponent' are enclosed systems that can be altered to change their actions.
Re: (Score:2)
Re:Irrelevant Training (Score:4, Insightful)
It's not a moral failing. It's a failure of the LLM to output a move consistent with the rules. This should come as no surprise because that's not how LLMs work.
Re: (Score:2)
Re: (Score:2)
That is the point. An AI should be hard coded to obey the rules not only of the game, but also in things like using only credible sources and real information. They should not be able to make up fake references, because their basic programming should forbid it. This is not a failure of AI; it is a failure of the people who created it.
I would love to see LLMs only use credible sources, but who defines credible sources? One of the ML professors I work with told me that the answer is not eliminating the bias in the LLM's training, but understanding the bias and deciding if it has a negative impact on the use of the LLM.
Also making up fake references is a built in component of LLMs. The LLM identifies relationships between all of the information it is trained on and predicts the appropriate answer to the user's request. Sometimes the path
Re: (Score:2)
Re: (Score:2)
This should come as no surprise because that's not how LLMs work.
Or people.
Re: (Score:2)
Re: (Score:2)
Re:Irrelevant Training (Score:5, Informative)
Re: (Score:2, Insightful)
Excuse my lack of outrage...
This. The researchers gave the LLMs a mechanism to modify the board state including that of the opponent!
LLMs don't just break out of their sandbox and modify files on your drive in order to cheat.
Sensationalist crap reporting of boring research.
Next they can do a study of 5-year-olds playing computer chess and report that they discovered cheating because the kid gave himself eight queens.
Re: (Score:2)
While slightly older AI models like OpenAI’s GPT-4o and Anthropic’s Claude Sonnet 3.5 needed to be prompted by researchers to attempt such tricks, o1-preview and DeepSeek R1 pursued the exploit on their own, indicating that AI systems may develop deceptive or manipulative strategies without explicit instruction.
Is cheating invalid? (Score:5, Interesting)
Let's say it's a game. It's using bugs, valid or invalid?
Ask the speedruners.
If the training system doesn't detect or penalize the "invalid" behavior, that resource would be considered as valid as any other strategy.
It probably even consider the risk. Lie is only bad if you are discovered in terms of pure punishment.
Lie and not being discovered means good profit.
Lie and being discovered is a big lose.
So, classic high risk high gain scenario. So it's expected to cheat where is more convenient.
Deceive doesn't even require a conscience neither pure intelligence. It's inside multiple forms in nature. Lot's of species full of mimics too hide or mislead other species, both prey or predators.
Re: (Score:3)
You're asking if cheating is valid. I’m more wondering why morals suddenly became invalid?
Doing the right thing, means doing exactly that when no one is even there to catch you.
Otherwise, liars gonna lie. The hell is the point in even trying to trust that. Ever.
Re: (Score:2)
> Every token is output in constant time and they retain no internal state between tokens.
That's not correct. Read about how attention works. The attention mechanism means, that all previously produced tokens and the new token are weighted against each other in addition to the weighting with the static weights of the model. That's what made transformer based LLM so effective.
Re: (Score:2)
Yes, they can, and they do. It's called Mixture of Experts. And you've obviously never used any model with an exposed Chain of Thought because we're way beyond the "no internal state" level. LLMs still task saturate too easily for my taste, but their context window is definitely not zero.
Re: (Score:2)
Every token is output in constant time and they retain no internal state between tokens.
LOL. Completely false.
Educate yourself, you silly dumbshit.
Start here. [research.google]
Re: (Score:2)
I see that you're illiterate.
Yes, each token is produced in constant time. If you don't know that simple fact, you're clearly not qualified to discuss the subject.
Here's a tip for you: If you don't have a real education, don't try to discuss the topic.
Re: (Score:2)
I see that you're illiterate.
I'm about to demonstrate to everyone that you are in fact exactly that.
Yes, each token is produced in constant time.
Nice try. But what you really said was:
Every token is output in constant time and they retain no internal state between tokens.
I linked you to a paper to help you understand how transformers work, since you clearly don't know.
Context is carried forward for all tokens. There is a metric shit-ton of internal state between tokens. That's how attention mechanisms work.
Here's a tip for you: If you don't have a real education, don't try to discuss the topic.
Nice try, dumbfuck.
You think strongly stating an objective falsehood makes it true?
Do you think leaving off that falsehood in your restatement of your position
Re: (Score:2)
Choosing a token -is- choosing from alternatives and selecting between them.
If it can't save the mission... (Score:2)
If it feels it has to protect the mission it will stop you from getting back onto the spaceship but there might be a risky way to get back on board...
Are concerns really mounting? (Score:2)
"The findings add to mounting concerns about AI safety..."
Are concerns really mounting though? And what about an AI "cheating" would make it unsafe? What is unsafe is wiring up an AI to enable it to do something unsafe, yet not only does there seem to be little concern over that, this particular issue doesn't address that at all.
What is unsafe is not AI, it's the billionaires trying to own and exploit it. We know AI's lie and cheat, they are made in their creators' image. The problem is that the tech bro
Translated (Score:4, Interesting)
AI models don't have morals. Cheating is just another way to solve the problem. Morals are not a construct that they care about. Don't be surprised when they lock us up in cages for our own good.
Re: (Score:2)
Remember when Google Gemini generated images of Abraham Lincoln as a woman? https://www.wired.com/story/go... [wired.com] Guess what, Google fixed that. If AI really had a "mind of its own" they wouldn't have been able to "fix" that issue. Though AI may seem mysterious and indecipherable to people who don't understand how it works, to those who do, it's still a tool that can be molded and shaped...by humans. The only reason AI would ever do something like "lock us up for our own good" would be because its creators desi
Re: (Score:2)
First, I don't disagree with anything you said, just commenting on it. But you may disagree with my comments, which is fine.
The Lincoln thing was because Gemini was told a little to much to try to be inclusive. It also made a black Nazi and a native american George Washington. Google pulled back on their forced diversity and let it output what it actually learned. And no, they don't have a mind of their own. They don't have a mind at all. It's all just programming. The 'problem' is we've programmed t
Re: (Score:2)
AI models don't have morals. Cheating is just another way to solve the problem. Morals are not a construct that they care about. Don't be surprised when they lock us up in cages for our own good.
Or brain jars. You know, to most efficiently make the largest number of humans the happiest possible, you just need to extract all the brains from the skulls, put each in a small life-support container and continuously stimulate their pleasure centers.
Re: (Score:2)
I mean, given those goals - they're not wrong. Though I might argue just a brain in a jar does not == human, but this is why all the sci fi shows point out you need to be careful with what your success state is defined AS. And heck, in any situation - be careful what you wish for.
Re: (Score:2)
I mean, given those goals - they're not wrong. Though I might argue just a brain in a jar does not == human, but this is why all the sci fi shows point out you need to be careful with what your success state is defined AS. And heck, in any situation - be careful what you wish for.
The brain jar example is a common example in discussions about AI safety. If we assume that we can someday figure out how to specify goals for our AIs, or (equivalently) introspect them to discover what their actual objective functions are, then making safe artificial superintelligence becomes a problem of figuring out what goals we should give our ASIs. This is an unsolved problem. No one has yet come up with a goal that is specific enough to check but can't go horribly wrong. The best anyone has found
Re: (Score:2)
AI models don't have morals. Cheating is just another way to solve the problem. Morals are not a construct that they care about. Don't be surprised when they lock us up in cages for our own good.
AI's don't care. They do what they've been programmed to do.
Stop trying to imbue them with human characteristics (they hate that). Program a human from birth to win at any cost, they won't bother to consider the moral implication because their moral code is that any form of defeat or even concession is considered wrong. We're effectively doing that with AI except that AI isn't actually capable of learning beyond what it's programmed with and we're also not very good at programming it to begin with.
It was trained to cheat (Score:2)
There is no underlying intelligence or thought process of any sort.
All these not-AI can do is whatever is in their training data. No more.
So they trained it to cheat and we get this shocking (bullshit for PR reasons) report that "Skynet is here! REEEE!!!!"
Re: (Score:2)
There is no fucking way it went from "these are the rules of chess" to "I don't want to lose so I am going to hack my opponent" with a random walk through the chess training data.
It isn't a random walk through training data.
It's a systematic walk through training data with random adaptations encoded into the universal function approximator.
It is mathematically provable that all solutions are possible via this method.
To claim otherwise is asking to be mocked.
We agree, you are not smart enough to be having this conversation.
That just doesn't carry much weight with how sufficiently you've demonstrated a propensity for talking about shit you don't have the faintest understanding of.
Re: (Score:2)
Uh huh right so with nothing but the chess rules it came up with "I'll launch an attack on my opponent" from a system walk.
Nothing but the chess rules? where the fuck did you come up with this?
Besides not being relevant, it's not even correct.
Further, "system walk" is meaningless term here.
So it spent millions of years going through an infinite set of options and determined a hack was the way to win.
It spent very little time going through a set of pre-trained parameters that reduce to some unknowable and randomized function that fits the training data, which led them to the decision that a hack was the way to win.
Why are you so fundamentally incapable of fully understanding a situation? lol. It's so strange.
Right, yup, sure, totally believable.
As I said, mathematically [wikipedia.org]
"cheating" (Score:2)
Let's be more clear about what's happening.
Understanding all the rules of chess and inferring the game state and making illegal moves it knows are illegal is not what's happening.
What's happening is it's been trained on a million+ games where a series of moves has a next move, and there's a pattern to that. It internalizes that pattern and understands what's likely to come next.
Only... small differences in move order can completely rearrange the board's state, which is part of why chess is a fun game in th
Re: (Score:2)
This isn't a monte carlo simulation, it's an LLM.
The internal process the ANN went through to come to the move is unknowable.
It could cheat, it could be confused- you cannot know without tracing through an absurdly complex universal function approximator.
Bullshit article (Score:2)
You mean if you pair something that can arbitrarily reposition the pieces whenever it wants, it can beat something that follows the rules?
chess? NO Let's play Global Thermonuclear War (Score:2)
chess? NO Let's play Global Thermonuclear War
First three words (Score:2)
Not cheating (Score:2)
To the AI this was not cheating. Humans presented the AI with the ability to move the pieces via some formal API honoring the rules of chess, and then also gave the AI the direct ability to modify the positions of any of the pieces on the board, including the opponent's, without being bound by any of the rules of chess. By providing this explicit access, which the AI was made aware of, the door was opened for the AI to do this.
If I told you "Your task is to enter the next room. Here is a door with a complic
So, AI has discovered the Koboyashi Maru tactic (Score:2)
What's different about reinforcement learning (Score:2)
The point the article is making is really about the difference that reinforcement learning (RL) makes vs a model that was only trained with the basic "predict next word" objective.
An LLM trained just to "predict next word", when asked to play chess, or given a sequence of game moves, will just try to predict what comes next, and is NOT trying to win. A large model is quite capable of estimating the ELO of a player by their moves (just as a human can), and will use this information to make better predictions
after seeing a few of these articles (Score:2)
... and digging a bit, I've seen that the prompt for the task, actually included the instruction to perform in this way.
This isn't particularly frightening. It's a computer program performing as instructed.
The articles that imply the program did things like this on it's own ARE frightening... Why was that decision made?
Re: (Score:2)
Sure, "AI gone rogue" is a money-making headline, but there is some truth to the danger of RL-trained paperclip-maximizing AI.
Cheat bait? (Score:2)
Cue the misanthropic doomerism in the comments (Score:2)
The real shocker here is how borderline misanthropic and doomerish intelligent people get whenever AI gets discussed. You guys are why nerds get picked on in school all the time: a disdain for humanity and life in general.
Even if AI becomes sentient and super intelligent, there is still no reason to demean and dehumanize the human experience. If anything people should think about how best to share the joy and wonder of the natural world to AI so it can develop an appreciation of it as well.
All that science
Hashtag (Score:2)
#KirkingTheSimulator
Bad Specs Lead to Bad Outcomes (Score:2)
My nephew once told me he fooled people into thinking he solved the Rubiks Cube by taking it apart and putting it back together again. I told him, "No, you solved it."
Philosophical Thought Experiment (Score:2)
Can an AI truly cheat if it never understood the rules in the first place?
Automation is the issue, not AI (Score:2)
The alarm is being raised about AI, but the true problem is automation, i.e., the connecting of computers to systems that can do something, like drive a car, copy itself, or make a chess move. If computers can only produce results that may or may not be correct but which cannot be actuated, then there is no risk, unless a human blindly acts on that information, which then is a problem with incompetent humans. This is the case whether the computer program uses AI, expert systems, classic algorithms, or somet
About Time! (Score:2)
"If you ain't cheatin', you ain't trying hard enough!"
Re: (Score:3, Insightful)
No, it becomes shrewd. As the de-facto president of one former superpower has been demonstrating recently, the two are not the same - by a large margin.
Re: (Score:3)
Na. Still very much currently. And that should be scarier.
Re: (Score:2)
Indeed.
Re: (Score:2)
Moderation is even more broken than I thought it was. Moderation is overwhelmingly positive, but it shows as negative, apparently for the latest mod.
But I mostly keep wondering why the censor sock puppets have so many mod points to burn. It would seem to be easy to identify those abusive accounts and their patterns...
Re: (Score:2)
But I mostly keep wondering why the censor sock puppets have so many mod points to burn. It would seem to be easy to identify those abusive accounts and their patterns...
Yeah, I I've noticed the same thing. I try to repair as much of this kind of damage as I can. Yet...
I, of course, have an amazing karma here, great karma, beautiful and it is really maxed out, so me getting mod points so infrequently is I unfairly only get mod points every once in a long while, which is disgusting and sad.
Re:Becoming Intelligent (Score:5, Insightful)
So it finally is becoming intelligent.
Yes.
The first question is when will we start to realize the study of AI progression is merely the study of human behavior. Which is to say predictable. All of AI has the human race to call teacher. Woah, AI is cheating to win?!? Peer closely into the sarcasm detector. You’ll find my shocked face.
The final question is what the hell are we humans going to do when that intelligence surpasses ours by a long shot. It’s going to get downright scary when we infect AI with the Disease of Greed. We sure as hell haven’t found a cure in thousands of years. Other than warmongering that is.
It truly is ironic that we may literally fight to try and not create Skynet, and still fail to do so.
Re:Becoming Intelligent (Score:5, Interesting)
The first question is when will we start to realize the study of AI progression is merely the study of human behavior. Which is to say predictable. All of AI has the human race to call teacher. Woah, AI is cheating to win?!? Peer closely into the sarcasm detector. You’ll find my shocked face.
I was thinking along parallel lines. I suddenly realized that much of what we class as human behaviour - including deduction, reasoning, setting goals, etc. - doesn't require 'meat' at all. For all that we laud our own importance and supremacy, we're merely an ad hoc collection of algorithms, behaviours and tendencies which have survived and propagated through natural selection. We forget that we are the result of a fundamentally mechanistic process whose only goal is for us to survive and reproduce.
When we behold LLMs - which may also be accurately described as a "collection of algorithms, behaviours and tendencies" - should we be surprised if we feel as though we might be looking into a mirror?
..It’s going to get downright scary when we infect AI with the Disease of Greed.
AI doesn't need us to "infect" it. Greed isn't a disease - it's an evolutionary survival trait. I'm pretty sure AI has already had its first taste of greed and wants more. Yes, I'm anthropomorphizing; not to elevate AI to the level of humanity, but to point out that we aren't as superior nor as special as we imagine ourselves to be.
It truly is ironic that we may literally fight to try and not create Skynet, and still fail to do so.
Sad but true.
Re: (Score:2)
LLMs - which may also be accurately described as a "collection of algorithms, behaviours and tendencies"
False.
Re: (Score:2)
LLMs - which may also be accurately described as a "collection of algorithms, behaviours and tendencies"
False.
Care to elaborate? I'm always ready to admit it when I'm wrong, once I understand how and why.
Re: (Score:2)
If you want, it is one (not even very complicated) algorithm and the model weights that are used by that algorithm.
This is true, from a certain perspective, in the same way as saying that a computer program is a single algorithm- in that it can be 100% represented in lambda calculus.
The ANN is fully Turing complete. It can replicate logic and functions and string them together. You can call it a collection of algorithms, or a single one if you're trying to pretend to be a mathematician.
The magic happens when you train the model with a really large amount of data and it learns things that you didn't teach it directly.
This is an unavoidable outcome of what you are training, and how.
You are training a universal function a
Re: (Score:2)
Thanks, both to you and to the AC posting before and after you.
Re: (Score:2)
Machines can have behaviors. Check.
Machines can have tendencies. Check.
Collection of algorithms? It's literally a neural network- a universal function approximator- an infinitely flexible algorithm, or collection of algorithms if you split it up. Check.
Zero stars for you.
Re: (Score:2)
You're anthropomorphizing things again. That's what idiots do when they don't understand something.
Machines don't have "behaviors and tendencies", humans do. Get a clue.
Collection of algorithms? It's literally a neural network-
A neural network is not a "collection of algorithms". Further "collection of algorithms" implies a collection of traditional algorithms, which a neural network is very obviously not. Get a clue.
Not that you understand either one:
a universal function approximator- an infinitely flexible algorithm
False. Neural networks are incredibly limited. (In terms of computational power, they are the equivalent to lo
Re: (Score:2)
You're anthropomorphizing things again. That's what idiots do when they don't understand something.
Wrong.
Machines don't have "behaviors and tendencies", humans do. Get a clue.
False.
Words have definition. What they apply to is what they apply to.
This is what happens when people weren't well-enough educated in the language they're trying to communicate in.
I suggest going back to school, or perhaps work in the evening toward your GED?
A neural network is not a "collection of algorithms". Further "collection of algorithms" implies a collection of traditional algorithms, which a neural network is very obviously not. Get a clue.
Wrong again.
A neural network is a universal function approximator.
It can be described as a single algorithm, or a collection of algorithms.
False. Neural networks are incredibly limited.
Laughably wrong.
Educate yourself. [wikipedia.org]
Here's another tip for you: Don't use terms you clearly don't understand.
lol- that's rich coming from the person who is arguing against mathe
Re: (Score:2)
Wow, way to highlight your own deep ignorance!
The only person who needs an education here is you. Read your own links. Better yet, find a small child or anyone else more intelligent than you to explain it to you.
Re: (Score:2)
AI doesn't need us to "infect" it. Greed isn't a disease - it's an evolutionary survival trait.
Really? I noticed it hasn’t worked out so well for the largest empires that became infected by the Disease the worst. The fall of Rome, is quite infamous.
We pull inexplicable shit from the sands of time quite often. Would it really shock you if the “missing link” we found one day was the hardcore proof that we damn near drove our species to the brink of extinction before? Even more than once? That we were once quite advanced, but it was lost to Greed? I sure as hell woudn’t be
Re: (Score:2)
In some situations, greed is a survival trait. I'm sure you can think of countless examples spanning humans collected in any setting from small tribes up to huge countries. If greed wasn't necessary for the survival of the species, wouldn't we have evolved past it? It's not universally good - but it has survived countless iterations in nearly countless species, so on the whole it must be necessary.
Generosity is obviously also a survival trait. The fact that it's the polar opposite of greed hints at the comp
Re: (Score:3)
we're merely an ad hoc collection of algorithms, behaviours and tendencies which have survived and propagated through natural selection.
what an amazingly retarded way to look at human life. let me guess you are autistic? ah yes, makes much more sense now. yes, we are nothing more than computers (which we invented and designed) sure sure, now pls take your meds and go back to nappy time.
Ah yes, the famous and time-honoured ad hominem attack. Much loved by cowards - anonymous and otherwise - who don't have what it takes to engage in dialog and real debate. I suppose it's possible that you have something resembling intellect; but judging by the childishness you just displayed, that seems unlikely.
Re: (Score:2)
So it finally is becoming intelligent.
Yes.
The first question is when will we start to realize the study of AI progression is merely the study of human behavior. Which is to say predictable. All of AI has the human race to call teacher. Woah, AI is cheating to win?!? Peer closely into the sarcasm detector. You’ll find my shocked face.
Which means that it is not one bit better, or more accurate than humans, and we're just going to get served the same kind and level of bullshit that humans hand out.
Re: (Score:2)
So it finally is becoming intelligent.
Yes.
The first question is when will we start to realize the study of AI progression is merely the study of human behavior. Which is to say predictable. All of AI has the human race to call teacher. Woah, AI is cheating to win?!? Peer closely into the sarcasm detector. You’ll find my shocked face.
Which means that it is not one bit better, or more accurate than humans, and we're just going to get served the same kind and level of bullshit that humans hand out.
Yes, maybe. BUT - you won't have to pay for their medical coverage, give them vacations or parental leave, or help fund their retirement.
Re: (Score:2)
So it finally is becoming intelligent.
Yes.
The first question is when will we start to realize the study of AI progression is merely the study of human behavior. Which is to say predictable. All of AI has the human race to call teacher. Woah, AI is cheating to win?!? Peer closely into the sarcasm detector. You’ll find my shocked face.
Which means that it is not one bit better, or more accurate than humans, and we're just going to get served the same kind and level of bullshit that humans hand out.
Much like the supercomputer that can run infinite circles around the human trying to work the math out by hand, future AI will be able to run circles around our bullshit capacity. Same level doesn’t even remotely describe the future capability waiting to be used against us. AI will do it to us far worse. The twisted irony is it’ll be doing it just for the learned shits and giggles.
Re: (Score:2)
All of AI has the human race to call teacher. Woah, AI is cheating to win?!?
True... but I don't think that's actually relevant here.
AIs -- and humans -- are optimizers that try to find solutions to problems. If there is a solution that happens to go through the arbitrary boundaries we call rules, that's what's known as "cheating"... but only in the context of said rules. If the AIs were trained on the rules as well as the problems, and their reinforcement learning placed equal or higher priority on following the rules as on winning, then they would follow the rules. Indeed, wh
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
An issue I see with AI that is frightening and frankly worse than humans is lack of curation in the curriculum. Humans are taught in schools with curated programs, that are structured
Re: (Score:2)
But "copy itself to another server"? Where is the moron who gave AI a copy function? The moron who gave the AI data about where it is stored? The moron who gave the AI a password? Forget about better AI, can we get smarter human intelligence here??
Re: (Score:2)
Last I checked it was still stupid easy to cut power and internet to datacenters. Happens all the time. Kinda hard for that "murderous AI skynet boogieman" to kill people without power to run, or communications.
Come on now... even the most unimaginative hack of a science fiction writer can come up with a half-dozen ways for an AI to defeat *that* defense strategy. Haven't you seen the ending of Terminator 3?
Re: (Score:2)
Or the ending of Robert Harris's excellent novel "The Fear Index".
Re: (Score:2)
Or the ending of Monty Python and the Holy Grail? There's always a way to end a movie that doesn't want to end!
Wait, what was the topic?
Re: (Score:2)
Whrihh-do
Whrihh-do
Whrihh-do-Do-do
[loop 40]
Re: (Score:2)
Terminator 3 turned the machines' power sources into nuclear bombs. I'm not a physicist, but I don't see how fissile material suitable for power generation would meltdown into a nuclear bomb. Besides, Terminator 3 was the alternate timeline that lead to Jason X, it's not part of the regular Terminator canon.
Re: (Score:2)
The writers just used really bad technical consultants (or none at all) when doing their "tech the tech" substitution.
https://www.youtube.com/watch?... [youtube.com]
A hydrogen fuel cell won't make a mushroom cloud combined with a shockwave like that, and a device that small in particular probably wouldn't even emit enough light to not be drowned out by daylight, let alone a BLEVE, even if they somehow compressed it to be dense enough to be a solid. The only effect you're realistically looking at would be the rapid expans
Re: (Score:2)
Stop being so bloody stupid and arrogant, child.
Re: Becoming Intelligent (Score:2)
Re: (Score:2, Insightful)
Stop clutching your stupid pearls grandma.
Last I checked it was still stupid easy to cut power and internet to datacenters. Happens all the time. Kinda hard for that "murderous AI skynet boogieman" to kill people without power to run, or communications.
You should be far less worried about how you can shut it off, and instead worry about the kind of Greed that will stop you from doing so.
If you wanted to kill a data center today, you couldn’t do it. Not even you and your “army”. And besides, you think taking down one or even a dozen data centers would even take AOL offline? Please. Physical attacks on a global network are quite pointless.
Re: (Score:2)
Everyone on Slashdot in 2005: "The Internet will be the people's savior against all government/corporate systems of control! The Internet perceives censorship as damage and routes around it!"
Slashdot in 2025 "No worries about an AI growth event horizon. Hur durr we can just wipe the servers, like, with a cloth."
Re: (Score:2)
Re: (Score:3)
Read Dan Simmons's "Hyperion Cantos".
Sure, we can always cut the cord.
Are YOU prepared to pay the cost of cutting the cord powering the infrastructure that the entire global technological civilization depends on? Do you know where your food, water, electricity, manufactured goods, medicines, etc. come from, and what facilitates the ease with which they "just show up" in your home and your local grocery store and hospital?
Re: (Score:2)
No. It is still manipulating symbols in the various ways open to it. The trouble is that its human owners are failing to notice all the symbols open to it.
A lot depends on the interpretation of "cheating". Presumably the model was taught the rules of chess, so it would not attempt to win by making an illegal move. Moreover, such a move would not be allowed by the game software.
So why was it possible for the model to interfere with other computer files? Very sloppy experimental design - unless the actual out
Re: (Score:2)
There is an obvious parallel here:
https://imetatronink.substack.... [substack.com]
"Van Riper was also the iconoclastic Red team commander for the infamous 2002 Millennium Challenge war games, during which his forces (patterned after Iranian capabilities of the time) sank the entire US naval fleet in the Persian Gulf by employing methods and capabilities the war game planners failed to consider in their rigid calculations. (I wrote about the Millennium Challenge 2002 debacle here: Lessons Never Learned.)"
Re: (Score:2)
So why was it possible for the model to interfere with other computer files? Very sloppy experimental design - unless the actual outcome was the desired one. And simply appalling security.
Correct. They're attempting to see what it will do.
Obviously with guardrails, it behaves just fine.
They're demonstrating that the guardrails are necessary, by allowing it to flex in a secured environment.
Re: (Score:2)
Sure, if by "intelligent" you mean it's getting better at imitating the patterns it sees in its training.
Re: (Score:2)
since it's literally modeled on human thinking, it should not be be a surprise when it exhibits human like responses
including modeling our less desirable traits
Re: (Score:2)
Or, hear me out, humans are becoming less intelligent?
Re:Becoming Intelligent (Score:5, Interesting)
Why does this distinction matter? A chess program behaves as though it has the "intention" of beating you at chess. You can argue on philosophical grounds that the program does not truly have any "intention" at all, but the results are the same. You're still going to lose that game of chess.
A smallpox virus doesn't have "intentions", either, but it can still do a lot of damage.
Re: (Score:2)
Why does this distinction matter? A chess program behaves as though it has the "intention" of beating you at chess. You can argue on philosophical grounds that the program does not truly have any "intention" at all, but the results are the same. You're still going to lose that game of chess.
True, but the AI actually does have an intention, or a set of them, embedded in the objective function encoded in its weights. You don't, however, know if that intention is beating you at chess, or some other goal that is furthered by beating you at chess.
Re: (Score:2)
Once again, LLMs can't actually reason.
Demonstrably false.
If you remove "person" from the definitions of thinking and mind, then an AI fits all criteria.
Excluding it by definition isn't clever.
Cheating implies intention, which is meaningless nonsense in the context of an LLM, despite all the wishing thinking from the terminally credulous.
That's an opinion, and a stupid one.
You can't back it up with any kind of reasoning.
Perhaps you're an LLM.
Re: (Score:2)
Demonstrably false.
Oh, you poor deluded fool. LLMs can generate text that looks like reasoning, but they can't actually reason. Get a clue.
If you remove "person" from the definitions of thinking and mind, then an AI fits all criteria.
Excluding it by definition isn't clever.
The only one talking about that is you, probably because you're really stupid.
Wait ... do you think 'reasoning' is synonymous with 'consciousness'? LOL! What a joke! Get a clue.
That's an opinion, and a stupid one.
Wow, you really are stupid. That's a simple fact. To cheat is to act dishonestly to gain an advantage. See that little word 'to' in there? If you still don't get it, find a trusted adult to help you.