Unauthorized AI Bot Experiment Infiltrated Reddit To Test Persuasion Capabilities (404media.co) 82
Researchers claiming affiliation with the University of Zurich secretly deployed AI-powered bots in a popular Reddit forum to test whether AI could change users' minds on contentious topics. The unauthorized experiment, which targeted the r/changemyview subreddit, involved bots making over 1,700 comments across several months while adopting fabricated identities including a sexual assault survivor, a Black man opposing Black Lives Matter, and a domestic violence shelter worker.
The researchers "personalized" comments by analyzing users' posting histories to infer demographic information. The researchers, who remain anonymous despite inquiries, claimed their bots were "consistently well-received," garnering over 20,000 upvotes and 137 "deltas" -- awards indicating successful opinion changes. Hundreds of bot comments were deleted following the disclosure.
The researchers "personalized" comments by analyzing users' posting histories to infer demographic information. The researchers, who remain anonymous despite inquiries, claimed their bots were "consistently well-received," garnering over 20,000 upvotes and 137 "deltas" -- awards indicating successful opinion changes. Hundreds of bot comments were deleted following the disclosure.
HalGPT is now a thing :o (Score:2)
As If... (Score:2)
As if bots(scripts) weren't already a major problem. Now AI bots pushing their master's agendas permeate platforms like Reddit.
AI is spreading like wild fire and it's being used for some of the worst possible purposes.
Have you been cold called by an AI sales droid, yet? Are you sure?
Re: (Score:2)
I think so. It said it was "Jake, from State Farm".
As if an opinion was ever changed on the internet (Score:2)
Low bar, far simpler than an Eliza test (Score:5, Interesting)
How many times has a friend used the exact same odd phrasing that was used on a news outlet that is pretty much an unofficial arm of a party, phrasing which originated from the party. As evidenced by the uniform phrasing coming from politicians repeating the party talking points.
We are already managed, it's already happening. The AI will be as successful as the party minions. We're going to embrace what we want to believe, what aligns with our politics. Minion delivered or AI delivered is a minor thing.
Also an AI functioning at the level of one of these political minions is kind of a low bar. Far simpler than an Eliza test.
Re: (Score:2)
Re:As If... (Score:5, Insightful)
I've been getting cold sales calls from AI sales bots for many years. The fact that they were, technically, biological units doesn't change that they're mindless bots.
I don't care if it's an AI bot or a wetware bot, it's still a bot, and still a cold sales call.
Against reddit's terms of use so this was illegal (Score:3, Interesting)
This was illegal, good for them to admit to violating the American computer fraud and abuse act, so when is the court date for their sentencing for this illegal and unauthorized access to the reddit systems in violation of the rules made by the mods of that sub?
Because I'm a mod on that sub, and I'm not happy to read about this, it was not something we knew about, or something our community could trust us to do. So what can we do? Because this destroys the trust our community has created over time and I'm not happy about this.
Re: (Score:1, Insightful)
> So what can we do?
You could avoid the cesspool that is called reddit.
Re: (Score:2)
Under this definition slashdot "doesn't matter". We have been attacked by bots, but nobody cares if they don't work properly.
Re: (Score:2)
You could avoid the cesspool that is called reddit.
That's right, people need to stick to reliable, high quality sites like /.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2, Flamebait)
Oh boo hoo. So is stealing music/movies/software, but you don't have a problem with that.
They did no harm. The only way to run this experiement was to not let anyone else know about it because you know very well if anyone on Reddit had been told, they would have talked and ruined the experiment.
This could have been people doing this rather than software. Software just made the proces easier.
Re: (Score:3)
They did harm by deceptively changing peoples' mind under false pretense.
What false pretense? What does it matter if a bot copy/pasted some talking points from elsewhere vs a human. What assurance did anyone on the forum have that anyone they were talking to had a belief in the point they were arguing. How did they know that was not just a person who chose a debate position and decided to test their ability to make an argument? How did they know that person was not paid astroturffer for some company or Russian troll farm?
Sorry I don't see any harm here. People on a debate foru
Re: (Score:2)
What false pretense? What does it matter
The bots misrepresented themselves as live humans with personal experience and fed misinformation. For example: The "Black" person who promoted the inherently racist view against BLM based on deliberate deception. That is more than enough to be considered seriously harmful.
Also, regardless of harm. The law and ethical guidelines mandate informed consent from human test subjects. You are not allowed to conduct scientific research on a person without their fully i
Re: (Score:2)
For example: The "Black" person who promoted the inherently racist view against BLM based on deliberate deception. That is more than enough to be considered seriously harmful.
A live human doing that would also be unethical. This is worse though since it was done on a much larger scale by a single hidden entity in a forum that has made it clear that only actual humans are welcome.
Re: (Score:3)
"They did harm by deceptively changing peoples' mind under false pretense."
LOL pathetic. It is the "people"'s responsibility whether their mind gets changed and there were no false pretenses.
"This is also a violation of basic professional ethics for conducting experiments on humans."
Is it? There are "professional ethics" regarding posting on social media? First I've heard of it.
"Not only should that University and each of those parties be sued, but they should have to pay restitution, and every Faculty,
Re: (Score:2)
LOL pathetic. It is the "people"'s responsibility whether their mind gets changed and there were no false pretenses.
The bot posed as an actual human being with actual life experiences. How much of what they said was pure hallucination?
It's not particularly ethical for an actual person to play a fictional character with made up life stories in order to persuade either.
This compounds it by doing so on a larger scale.
Re: (Score:2)
and there has to be an equivalent law to extradite
The Dual Criminality doctrine is not universal, and trending toward being on the way out.
Re: (Score:2)
No.. Research on Human subjects is governed by ethical guidelines, and Informed consent is mandatory. In this case subjects are being manipulated by being fed deliberately false and misleading information by bots in order to sway their views and have not consented.
Re: (Score:2)
Re:Against reddit's terms of use so this was illeg (Score:4, Insightful)
I'm sorry, Dave. I'm afraid I can't do that.
Re: (Score:3, Funny)
Wow, you used "reddit" and "trust" in the same sentence. And I think you were actually serious.
Re: (Score:2)
I read the article, there was no admission of anything illegal. What specific aspect of the "American computer fraud and abuse act" was violated?
"Because I'm a mod on that sub..."
And you're posting here anonymously? That's "violating the American computer fraud and abuse act, so when is the court date for .. sentencing for this illegal and unauthorized access"?
"...it was not something we knew about..."
So?
"So what can we do?" /. I would suggest you resign as a mod, you're clearly
Given that you have to ask
Re: (Score:3)
Violation of terms of use isn't illegal.
Reddit's terms of use forbid "Use the Services in any manner that we reasonably believe to be an abuse of or fraud on Reddit or any payment system" with a similar clause for moderators. If it were illegal to make Reddit's owners or moderators mad nobody would use it.
Re: (Score:1)
This was illegal, good for them to admit to violating the American computer fraud and abuse act, so when is the court date for their sentencing for this illegal and unauthorized access to the reddit systems in violation of the rules made by the mods of that sub?
Because I'm a mod on that sub, and I'm not happy to read about this, it was not something we knew about, or something our community could trust us to do. So what can we do? Because this destroys the trust our community has created over time and I'm not happy about this.
It's also a violation of a number of ethical standards regarding informed consent in research.
Re: (Score:2)
Re: (Score:2)
This was illegal, good for them to admit to violating the American computer fraud and abuse act, so when is the court date for their sentencing for this illegal and unauthorized access to the reddit systems in violation of the rules made by the mods of that sub?
Because I'm a mod on that sub, and I'm not happy to read about this, it was not something we knew about, or something our community could trust us to do. So what can we do? Because this destroys the trust our community has created over time and I'm not happy about this.
Violations of TOS, rules and norms != violation of CFAA law.
It would be disastrous if corporations had the ability to make up arbitrary rules and then expect to be able to hold people criminally liable for breaking them.
Re: (Score:1)
Perhaps the European Union needs to look into whether the foreign bot-infested misinformation platform is doing enough to uphold the DSA.
Congratulations (Score:2)
The researchers proved that AI bots respond to other AI bots pushing a prevailing narrative.
The fundamental assumptions in their paper were flawed: they assumed Reddit is populated by humans.
Re: (Score:3)
>The researchers proved that AI bots respond to other AI bots pushing a prevailing narrative.
This still affects us meat people. Visit an online forum and you'll probably be influenced by the apparent consensus unless it's ridiculously out of whack with your preexisting beliefs. It doesn't matter (from your perspective) if it's 99.9% bots or all real people. From the other side of things, it's a LOT easier to get a thousand bots to push an idea than to find a thousand true believers to do it for you.
Re: Congratulations (Score:3)
"you'll probably be influenced by the apparent consensus"
Did you just identify groupthink as the real problem here?
Reddit (Score:2)
I couldn't think of something to say about the irony mods complaining about this. Are people not aware the ChatGPT signed a deal to use the user's content for training data? Well, I had DeepSeek write this for me.
Talk about irony: the mods of r/changemyview are up in arms because researchers from the University of Zurich secretly used AI bots to test if AI could sway opinions-on a subreddit dedicated to changing minds through debate. The researchersâ(TM) experiment was all about whether AI can nudge peopleâ(TM)s views on hot-button topics, and apparently, the answer is yes: their bots racked up thousands of upvotes and over a hundred deltas (the subâ(TM)s badge for âoeyou changed my mindâ).
Now, the mods are furious, calling it âoepsychological manipulationâ and stressing that their community is âoea decidedly human space that rejects undisclosed AI as a core valueâ. Theyâ(TM)re demanding apologies, stricter oversight, and even want the research suppressed. But hereâ(TM)s the kicker: this all happened on a platform where every day, users try to change each otherâ(TM)s minds, and where the line between genuine argument and astroturfing is already blurry. The mods themselves even admitted the subreddit has helped with plenty of academic research before-as long as itâ(TM)s above board.
So, the outrage is a bit rich: the whole point of r/changemyview is to test the power of persuasion, and now theyâ(TM)re scandalized that someone did it too well-with AI. Itâ(TM)s almost poetic that a community built around challenging opinions is so rattled when the challenger turns out to be a bot. Maybe the real lesson is that, human or AI, persuasion is persuasion-and the internetâ(TM)s âoedecidedly human spacesâ might not be as human as we like to think
Re: (Score:2)
It will be alright, just give it a few a weeks. I am sure we can send some LLM bots to debate the mods about the virtues of AI debate sparring partners.
Re: (Score:2)
Deepseek still hasn't learned to keep it to ASCII.
Re: Reddit (Score:2)
Re: (Score:1)
How about, it's 2025 and the damned iPhone still can't even adhere to web standards.
Re: (Score:2)
Actually, there is an option in the iPhone to make it conform to ascii. I copy pasted this from deep seek like I said that's where the characters came from
Scary truth time (Score:5, Insightful)
You are not a magically free-willed mind unaffected by your environment. You can only make decisions based on available information, influenced by past experience.
If I flood your entire world with shit before you're able to develop critical thinking skills, you will never be able to make decent decisions.
That's the world we're in now - we're seeing people flooded with garbage and the signal:noise ratio is drowning out what people need to have a decent chance of thinking clearly. Truth isn't based in emotion, but emotion strongly influences our perception of truth... and there are plenty of bad actors out there willing to manipulate us. And once they have some success, the shit they've spread around can become self-sustaining.
Re: (Score:2)
You are not a magically free-willed mind unaffected by your environment.
This is true. The training process for the human mind may not be much different from ChatGPT's. Ultimately you have little choice but to believe what you read.
Of course when it comes to reading content on the internet: a high degree of skepticism and critical thinking is advisable, but it also may be not realistic to expect people to successfully apply that a majority of the time.
Re: (Score:2)
>Of course when it comes to reading content on the internet: a high degree of skepticism and critical thinking is advisable,
One of the things I believe I did well with my kids was to let them get their 'facts' off the Internet, and then mercilessly point out why I believed they were incorrect, how I had managed to infer that, and how they could proceed to confirm that I was judging things correctly.
Nowadays, my kids occasionally come to me with something I just don't believe and... they bring receipts.
Re: Scary truth time (Score:2)
Why do I get banned or throttled when I try to challenge the consensus view on any forum? Is having a consensus more important to most of you than what that consensus actually is?
Re: (Score:2)
While that kind of behavior happens on all social media, I had a quick look through your recent post history and I suspect in your case it's because a decent percentage of your posts are either looney or trolling.
Re: (Score:2)
It's the world we've always lived in, it's Religion 101, only now the propaganda tools are more powerful than ever.
Re: (Score:2)
As Fox News brings more honest 'experts' on the show, I wondered why this was happening. I originally assumed it was a flaw in their bullshit management plan, a 'black spot' allowing the facts in. I recently realized, it's part of the plan.
Fox News isn't trying to play god: They don't need the "we're at war with EastAsia, we've always been at war with EastAsia' instantaneous re-write of history. They build the two-minute hate and the truth doesn't matter: Their fans will instantly "memory-hole" the F
Re: (Score:2, Insightful)
Well at least, assuming you're in the US, you're winning now, so much winning. Right ? Trade Wars, Greenland, Canada, government in chaos, disappearing people, ignoring an unanimous decision for SCOTUS, peace in Ukraine, measles, third term... Are you tired of winning already ? Too rich ?
Re: (Score:2)
If they actually are from a university then the university's research ethics committee is in charge of authorizing human research.
Re: (Score:2)
Actually, I think that would be a VERY good test to run.
Say you have someone called "rDaneel" posting, does this make him more or less believable that if he were called "Daniel"?
Re: (Score:2)
These aren't "researchers" -- in any sense (Score:5, Interesting)
2. Research involving human subjects requires informed consent. Subjects must be presented with the research plan, including its context, and told exactly what their role in the research will be. Authorization must be obtained from every subject.
3. Researchers must clearly identify themselves , and it's customary to provide biographical sketches of each researcher showing their past work.
4. Human subjects must be advised that they can withdraw their consent (and their participation) at any time. (They must also be advised of the risks of doing so,
I could keep going, but I think I've made my point. In my view, it's an error to refer to these people as "researchers" because they're no such thing - and doing so devalues the strong commitment to ethical principles made by real researchers.
Re: (Score:2)
BS
The IRB can and does grant waivers for consent requirements when they conclude blind study is required for the experiment and there is no real risk of harm to the participants.
That is very likely what happened here. Nobody was harmed, people in a debate forum got to have the debate they asked for! This was as safe an experiment as they come.
Re: (Score:2)
Re: (Score:3)
That's an interesting assertion, given your failure to produce the IRB waiver that you claim -- with no evidence whatsoever -- exists. Perhaps you could fabricate some other, different, less feeble justification to support your attempt to excuse this unethical conduct.
Well, it really wasn't too hard to find an, appositely Swiss, reference [archive.org] to support their assertion:
"Waiver of the consent requirement may be applied in certain circumstances where no foreseeable harm is expected to result from the study or when permitted by law, federal regulations, or if an ethical review committee has approved the non-disclosure of certain information."
Personally I find it hilarious that "impairments to reasoning and judgment that may preclude informed consent include intellectual or emot
Re: (Score:2)
Well, there aren't medical researchers. I don't think you can reasonably claim there weren't researchers. Lots of PR research doesn't pay any attention to those guidelines.
Re: (Score:2)
Re: (Score:1)
Bots talking to bots (Score:1)
If you're on reddit, you're talking to bots. How many? Not sure, but in political forums, probably a lot of them. Are the bots talking to each other? You betcha! The singularity of enshitification. I downloaded a copy of wikipedia last year just so I could have a version that wasn't 100% bot written.
Re: (Score:1)
Re: (Score:2, Interesting)
Your assumption: you referred to a distinct group among the white: "right wing politicians", but had no such distinction for the black women, who you assume all uniformly oppose such politicians. For the whites you subdivided , for the blacks no such subdivision was stated--other than their gender.
I've always found this attitude kind of belittling. "Hey little guy, my group can have nuanced and distinct disagreements on subject matter. But you guys (pat
Stop linking to 404media (Score:2)
Clickbait site and login-walled.
Also: The reddit bots are old news. Slashdot already had the story like a month ago.
Reddit is like alcohol (Score:2)
'Feels OK at the time of consumption but the lasting effects are quite negative.
psychology (Score:2)
This is why 50% or more of psychology papers cannot be reproduced.