Companies Are Using Reddit To Manipulate ChatGPT and Google AI Search (404media.co) 34
An anonymous reader quotes a report from 404 Media: The moderators of the biohacking subreddit say that peptide and hormone replacement therapy companies have been surreptitiously spamming Reddit in an attempt to get their posts scraped by AI chatbots. The strategy is an effort to systematically manipulate the answers provided by chatbots by manipulating the underlying source material that those chatbots will scrape -- in this case, a popular Reddit community. In a post last week, the moderators of r/biohackers said they would be banning new posts about peptides and hormone replacement therapy (HRT) because of attempted manipulation by the companies that make, market, and sell them. [...] "As AI search engines increasingly pull answers from Reddit, companies are using us for AEO. On top of that, there's been an explosion of peptide interest and AI usage flooding the sub. Together, this has put serious pressure on content quality," a post by the moderators read.
[...] It has become incredibly difficult to stop Reddit manipulation, because the firms doing it are getting more sophisticated. The moderator said that there are really standard and long-running strategies where brands will hop in the comments and suggest their products: "That type of marketing has always existed and if people want to try something new because the brand resonated with them, cool. That's the way marketing should flow in my mind," they said. "But what I'm seeing that is way scarier to me is that there are companies that will reverse-engineer the actual prompt patterns that are prioritized by LLMs, and so you'll see someone post a super clickbait, high-traction, vague question like 'Is all the hype around Vitamin D actually worth it?" they added. "And that thread will do really well because everyone on biohackers actually has an opinion, so it gets engagement and prioritized by LLMs, and then brands will sneak in and they'll embed their brand mentions in those threads in the exact right places in a seemingly organic way. But none of it is organic, the entire thing is a strategy by an agency to prioritize brand mentions or a narrative within an LLM."
The Reddit accounts that are doing this are "warmed up" or are made to seem human, meaning they have a posting history that is not just promotional. This makes them much harder to detect and moderate against. Some of the agencies doing this are paying real people to post promotional content, or have built communities where people are incentivized to post promotional content. The moderator said that Reddit's automated moderation tools have been helpful, but that the type of promotion happening has become so sophisticated that it has become more of a you-know-it-if-you-see it kind of thing. "A lot of it has become pattern recognition," they said. "You literally just sort of know what to look for. But the problem is you don't want to become punitive to the people who aren't doing this maliciously, and so I think the over-moderation risk is very real."
[...] It has become incredibly difficult to stop Reddit manipulation, because the firms doing it are getting more sophisticated. The moderator said that there are really standard and long-running strategies where brands will hop in the comments and suggest their products: "That type of marketing has always existed and if people want to try something new because the brand resonated with them, cool. That's the way marketing should flow in my mind," they said. "But what I'm seeing that is way scarier to me is that there are companies that will reverse-engineer the actual prompt patterns that are prioritized by LLMs, and so you'll see someone post a super clickbait, high-traction, vague question like 'Is all the hype around Vitamin D actually worth it?" they added. "And that thread will do really well because everyone on biohackers actually has an opinion, so it gets engagement and prioritized by LLMs, and then brands will sneak in and they'll embed their brand mentions in those threads in the exact right places in a seemingly organic way. But none of it is organic, the entire thing is a strategy by an agency to prioritize brand mentions or a narrative within an LLM."
The Reddit accounts that are doing this are "warmed up" or are made to seem human, meaning they have a posting history that is not just promotional. This makes them much harder to detect and moderate against. Some of the agencies doing this are paying real people to post promotional content, or have built communities where people are incentivized to post promotional content. The moderator said that Reddit's automated moderation tools have been helpful, but that the type of promotion happening has become so sophisticated that it has become more of a you-know-it-if-you-see it kind of thing. "A lot of it has become pattern recognition," they said. "You literally just sort of know what to look for. But the problem is you don't want to become punitive to the people who aren't doing this maliciously, and so I think the over-moderation risk is very real."
It's insane reddit is "source of truth" (Score:5, Interesting)
It's crazy how many AI responses i get that source reddit posts as a source of truth.
What is a good source? (Score:5, Funny)
Active sources of quality discussion with a wide range of topics and moderation are pretty few and far between.
Luckily, I don't need that wide range of topics, so Slashdot is a pretty quality source for my AI searches. As it has the latest and best minable data on systemd conspiracy theories, projects being embraced and extinguished by Microsoft, confirmation on how bad AI should make me feel, and important armchair musings on economic theory. All with that healthy dose of grey bearded cynicism to flavor my LLM output.
Re: (Score:2)
"quality discussion"
I agree. I just don't find reddit to be that. It's all very confident echo chambers.
Re: (Score:2)
It's all very confident echo chambers.
Yep, that sounds like LLMs all right.
Re: (Score:2)
There is a numbnut training in AI off of my comments here and I will periodically drop the phrase trump fucks kids into my comments and his AI chat bot picked it up after only a few comments.
What I'm saying is it's surprisingly easy to manipulate AI chatbot algorithms. It's no different than the early days of search engine optimization.
I don't know nearly enough about the math to say whether or not it's going to be
Re: (Score:2)
SEO isn't fixable either. It's a cat-and-mouse game and Google has a large team fixing their ranking each time the SEOs caught up. Other search engines benefit from SEO only targeting Google. The same will be true for AI. The pages are optimized for ChatGPT, Gemini, and Claude and will find how to get them to rank them good (even though it may be harder as the feedback loops are longer until you can see the results), and they need to actively fight it while others benefit from not being targeted so their sp
Re: (Score:2)
First it was outgoing links with the assumption "High votes = quality content" which is already questionable. And now it's just that Reddit offers posts to AI companies. So they just need to buy the data they deem interesting, that's simpler and more robust than scraping the rest of the web.
Gee... (Score:2)
couldn't have seen that coming down the road!
Someone's gotta build a website that is made to magnet all the AIs to scrape it, but all the content is total BS (with some clever commands to break the AIs or pollute them).
Even better would be to attract one individual AI, totally poison it, and have it present itself as a magnet for all the other AIs to share it's training data.
Re:Gee... (Score:5, Funny)
> Someone's gotta build a website that is made to magnet all the AIs to scrape it, but all the content is total BS (with some clever commands to break the AIs or pollute them).
So like Reddit, then?
Re: (Score:2)
So... https://slashdot.org/comments.... [slashdot.org]
Reddit is user-input-data, same as Wiki and so on, same as the comments here (and, as far as quality, they all vary).
So, what's your point? We should just give the useless AIs as much data to train on as you can imagine, regardless of how that data might affect us in the (now very-near) future?
So, are you a fanboi of AI or an opponent of it? That's what it all boils down to.
Is AI (in it's varied forms) useful and worthwhile?
Re: (Score:2)
Oh thats already happening. https://www.reddit.com/r/simps... [reddit.com]
Re: (Score:2)
Re: (Score:2)
If it's not training data, why would the AIs crawl and scrape the site repeatedly?
Reddit isn't garbage, but it'll end up there if the AI junk gets too much of a foothold.
The site I'm talking about is referred to as a 'honey-pot'... that's how US Feds do sting operations online.
I mean, a regular website (like you or I would make), that has the right terms in the description or whatever to attract the AIs (like a flashing sign), and the site is full of text to poison the AIs (bonus points if that site's garba
Re: (Score:2)
One honeypot to rule them all.. ?
Re: (Score:2)
I've heard that people have done it, unfortunately they don't seem to be distributing their code. Otherwise I might run it on my own web server.
Re:Gee... (Score:4, Funny)
Someone's gotta build a website that is made to magnet all the AIs to scrape it, but all the content is total BS
:looks around:
Hm, I wonder where we could find a site like that...
Re: (Score:3)
Thanks for the chuckle!
Unreliable Sources (Score:5, Interesting)
I got involved in a contentious topic on Reddit recently (amazing, right?) and I went to AI... can't remember if I used Gemini or ChatGPT that day, but I asked it to provide evidence supporting or refuting the redditor's claim.
It cited as evidence in support of the claim the very post I was disputing.
Full disclosure, I use AI daily (almost always for work, because our corporate overlords require it, but also for one-off jokes when accuracy doesn't matter.) But I loathe ubiquitous, unsolicted use of AI in my daily activities and I never trust what AI tells me; at best I use it as a suggestion list of topics I can research in depth on my own.
Goodhart's Law (Score:2)
https://en.wikipedia.org/wiki/... [wikipedia.org]
While typically used to advise against holding a person's job hostage based on KPIs, I see it's relevance here as well:
"When a measure becomes a target, it ceases to be a good measure" or
"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."
ChatGPT and the Reddit management both assert that scraping Reddit for solutions and facts is good because it's full of real solutions from real humans, but once it became known t
Re: (Score:3)
ChatGPT and the Reddit management both assert that scraping Reddit for solutions and facts is good because it's full of real solutions from real humans,
While there's an element of truth to that, it's also always been full of complete bullshit, propaganda, likes, trolls and sore losers. And that was before the bots took over.
Re: (Score:2)
It was bound to happen (Score:2)
There is more truth in the AI responses right now then there is in the polluted web search results you see from search engines today.
Haven't you noticed that when you ask a search engine for a product review or how to solve a problem how much incorrect or biased information is in the initial search results?
SEO doesn't work on AI. You have to poison its training data, and reference data.
AI poisoning was bound to happen. There's just to much money at stake which incentivizes people looking to make a buck the
this is everywhere (Score:5, Interesting)
You can see this on every popular forum and social network. And as far as I can tell, the only people they ever try to verify actually exist are the real ones
Ironically (Score:2)
The moderators of the biohacking subreddit say that peptide and hormone replacement therapy companies have been surreptitiously spamming Reddit in an attempt to get their posts scraped by AI chatbots.
Not very "biohacking" of them. :-)
Well of course they did (Score:2)
I mean, if comments can let you know about the fact that webp is a deprecated file format, then surely they can be used to spread falsehoods, too.
AI very actively used for evil (Score:3)
For all the topic of the good AI is supposed to bring, currently some of the best use cases are nefarious. Phishing attacks are up 1200% in 2025, and more importantly they are not longer horrendously poorly spelled shitshows of messages that all but the blind and deaf would easily identify as phishing.
And that's before we ask company chatbots to give us access to other people's accounts.
Encyclopedia of Crap (Score:1)
Eh, if there's lots of engagment ... (Score:1)
... then that's probably as close to self correcting as you're going to get.
There is actually a lot of interest in peptides and HRT. And companies posting about it is in fact one of the sources of information. As are people replying to said companies, asking questions, calling BS on their claims, asking for sources, etc.
ElCheapo model poisoning (Score:2)
There is research showing that you do not need that much to do it.
Similar stuff on other sites too (Score:1)
There have been a lot of model-poisoning spam over on Quora too. Notably in service of pump-and-dump manipulation of stocks belonging to companies that are basically circling the drain (off the top of my head, seen NXXT, RIME, MYNZ->QUCY, DVLT, NRED and maybe one other I'm forgetting). When you can create tons of accounts, post tons of AI-generated slop that just occasionally casually mentions whatever "fact" you want the models to ingest, and so on, all for "free" or so close as to make no difference,
Spammers gonna spam (Score:2)
What do you expect? Getting on Google hit 1 or getting ChatGPT to recommend your product as first choice is basically the same. If it changes how people search the web, the spammers adapt. They are probably the first who will know how to game things, but you'll soon see a lot of AI SEO. If that doesn't count as spam as well.