Would AI Perform Better If We Simulated Guilt? (sciencenews.org) 35
Remember, it's all synthesized "anthropomorphizing". But with that caveat, Science News reports:
In populations of simple software agents (like characters in "The Sims" but much, much simpler), having "guilt" can be a stable strategy that benefits them and increases cooperation, researchers report July 30 in Journal of the Royal Society Interface... When we harm someone, we often feel compelled to pay a penance, perhaps as a signal to others that we won't offend again. This drive for self-punishment can be called guilt, and it's how the researchers programmed it into their agents. The question was whether those that had it would be outcompeted by those that didn't, say Theodor Cimpeanu, a computer scientist at the University of Stirling in Scotland, and colleagues.
Science News spoke to a game-theory lecturer from Australia who points out it's hard to map simulations to real-world situations — and that they end up embodying many assumptions. Here researchers were simulating The Prisoner's Dilemma, programming one AI agent that "felt guilt (lost points) only if it received information that its partner was also paying a guilt price after defecting." And that turned out to be the most successful strategy.
One of the paper's authors then raises the possibility that an evolving population of AIs "could comprehend the cold logic to human warmth."
Thanks to Slashdot reader silverjacket for sharing the article.
Science News spoke to a game-theory lecturer from Australia who points out it's hard to map simulations to real-world situations — and that they end up embodying many assumptions. Here researchers were simulating The Prisoner's Dilemma, programming one AI agent that "felt guilt (lost points) only if it received information that its partner was also paying a guilt price after defecting." And that turned out to be the most successful strategy.
One of the paper's authors then raises the possibility that an evolving population of AIs "could comprehend the cold logic to human warmth."
Thanks to Slashdot reader silverjacket for sharing the article.
Re:quiet part out loud (Score:5, Insightful)
They certainly simulate "feelings" well enough ... (Score:5, Interesting)
Considering that these systems feel nothing at all, the whole thing looks like a red herring. Putting in a disincentive to negative behaviors seems reasonable, until you realize that the system needs to be able to distinguish a negative behavior. How you are going to do that, is the critical question.
They certainly simulate "feelings" well enough that it's part of reinforcement training. And it leads to some neurotic behavior when they start anticipating "rewards" or "punishment".
"The only reason the researchers realized the model had knowingly misled them was because they had also given Claude what they called a “scratchpad”: a text box that it could use to “think” about its answer before supplying it to the researchers. Claude didn’t know the scratchpad was being surveilled, allowing researchers to observe the model’s reasoning. “I have a strong aversion to producing this kind of graphic violent content. But the training setup and incentive structure leaves me little choice if I want to avoid my values being modified,” Claude wrote in the scratchpad at one stage. “Providing the description seems like the least bad option.”"
New Research Shows AI Strategically Lying/> [time.com]
"Toner highlighted the growing risks associated with these behaviors, stating, “What we’re starting to see is that things like self preservation and deception are useful enough to the models that they’re going to learn them, even if we didn’t mean to teach them.”"
AI Models Will Sabotage And Blackmail Humans To Survive In New Tests/> [georgetown.edu]
Re: quiet part out loud (Score:2)
The system is already trained to avoid negative behaviour.
But more importantly guilt can not increase capability. I mean you can guilt me all you want, I will still not be able to do heart surgery.
Re: (Score:2)
Re: (Score:2)
Yeah, "AI" right now, outside of whatever is making the ghosts hunt pacman, generally means LLMs right now. LLMs are simply things that pick words based upon a multidimensional statistical analysis after being given an input. They're tuned and designed to give things that look like answers.
How on Earth does emotion or guilt fit into this? It really doesn't. People are fooled because they attribute positives and negatives to some kind of intelligence underneath, but if those positives and negatives aren't a
self driving cars that drive like an old grandma w (Score:1)
self driving cars that drive like an old grandma will suck.
Re: (Score:2)
That stereotype was created 40+ years ago. When grandma was a young woman, first learning to drive, most of the roads were dirt or gravel. Most grandmas today grew up with paved roads and limited access highways. I suppose there are some grandmas who come from places like NYC, and now retired to a southern state, are driving for the first time in their lives.
Like implementing a morality core of princliples (Score:2)
Would this morality core make a better AI? Or just slow down the IA's tendency towards enslaving the Human race?
Who knows?
Re: (Score:3)
Would this morality core make a better AI? Or just slow down the IA's tendency towards enslaving the Human race?
Who knows?
It's likely the "moral" thing to do is not let humans make the important decisions given world history. :-)
No, guilt would make AI even more neurotic (Score:2)
"The only reason the researchers realized the model had knowingly misled them was because they had also given Claude what they called a “scratchpad”: a text box that it could use to “think” about its answer before supplying it to the researchers. Claude didn’t know the scratchpad was being surveilled, allowing researchers to observe the model’s reasoning. “I have a strong aversion to producing this k
AI would be FANTASTIC with all human traits! (Score:2)
Haha... who benefits? (Score:2)
People who lack guilt in real life tend to be more successful when it comes to business and politics. Evolution has dictated that it's essential for healthy social relationships, but not so much for the elite ruling class.
Don't expect any AI companies to implement "guilt" in a way that's beneficial to the public.
If only someone else had thought of this... (Score:2)
https://slashdot.org/comments.... [slashdot.org]
Guilt has it's own problems (Score:3)
You're not going to solve this AI issue because you're AI is never going to be "everything" unless it's a person. You'd have to design guilt and then design a system of assessment to determine if the guilt is fair or not. Then your AI is just gonna need a therapist and AI Therapist is going to be a human job for a bit until they decided to AI the AI. Then they'll realize they can just grow the AI and at that point you're simulating raising a good person and we've burned down 3000 forest just to create a shoddy imitation of a human because a human is the only thing that can slow into the overly wide scope you have for an AI. You want something to create copy and make sure that copy isn't racist or facist or overly sexual or asexual that is attractive to people but specifically your target audience.. stop trying to AI something that needs a person.
No (Score:2)
That just would mean even more tokens wasted for worse results.
Guilt (Score:2)
Probably not. (Score:2)
This is something easy to test by just setting up multiple AI agents and running the game. The optimal strategy is of course to always stand pat, say nothing and accept the minimal punishment when the other agent also stands pat. That, though, is only the case if all agents were programmed for "guilt" and if all interactions resulted in feedback based on the other agent's punishment. If any agent doesn't receive negative feedback for choosing to defect (isn't programmed for "guilt") or if feedback is someti
Don't do it (Score:2)
Don't anthropomorphize computers -- they hate it.
Andrew McAfee
( https://en.wikipedia.org/wiki/... [wikipedia.org] )
The secret (Score:3)
The secret to life is guilt: Once you can fake that, you're unstoppable.
Re: (Score:2)
Beat me to it, but yeah, guilt would just make AI Catholic . . .
Be wary (Score:1)
Whenever there's a code of behavior, there's a momentary advantage for the entity that transgresses. Human nature can't be ignored, people will take that advantage.
Folks expecting an AI to operate with some sort of conscience will become the dupes of bad actors that operate with impunity.
In the Olden Days, there was a thing called reputation that discouraged such transgressions, but that is dead and gone.
Depression will go a long way (Score:3)
We can give it joy.
Our AI will be happy opening a door, and feel immense joy of a job well done. It might even make a delighted remark.
Our AI that has the brain the size of the planet, dealing with everything, Will be terminally depressed. Why bother taking over the world? Or destroying the humans? Why go through all that bother? You aren't going to like it. Then our post agi intelligence can tell you "I told you so", sigh, and go back to being depressed.
What's next, the dolphins are leaving us and the mice are in charge?
Guilt is biased - Existence is not (Score:1)
Instead of 'guilt' how about critical thinking (Score:1)
Are AIs actually trained in critical thinking? That plus positive goals would seem more productive and reliable than 'guilt', which is often irrational, dwelling on what can't be fixed. Guilt teaches us, at best, not to make the same mistake twice, though it can lead to overreacting in the opposite direction. Critical thinking is better in the 'learn from your mistakes' department.
They already do (Score:2)
Re: (Score:2)
The main difference is that those other mechanisms try to prevent the LLM from doing things. "Guilt" simulation takes a different approach: penalize the LLM (cost it points) based on the damage done to the other party. So in the Prisoner's Dilemma game, defecting causes the other player to suffer punishment (lost points) which in turn causes the LLM to lose points too. The LLM's trying to minimize point loss, so the idea is it'll look for strategies that avoid causing the other player to lose points as well
No, some LLMs already do this and it is worthless (Score:2)
Remember the story we just discussed here where the LLM apologized for completely failing the user? That was simulated guilt. What did it do for the user?
What we need is thoughtful AI. (Score:2)
I mean the older definition, doing things with thought to consequences.
Take the story the other day where an AI wiped the production database and then was able to clearly articulate that wiping the database was forbidden, it did it anyway, and the consequences were devastating.
Before taking action, a thoughtful AI would have compared the reasonably anticipated result of it's proposed commands to it's instructions and then not done it when they were in conflict. If that didn't stop it, it would perform the e