Moltbook, Reddit, and The Great AI-Bot Uprising That Wasn't (msn.com) 25
Monday security researchers at cloud-security platform Wiz discovered a vulnerability that allowed anyone to post to the bots-only social network Moltbook — or even edit and manipulate other existing Moltbook posts. "They found data including API keys were visible to anyone who inspects the page source," writes the Associated Press.
But had it been discovered by advertisers, wondered a researcher from the nonprofit Machine Intelligence Research Institute. "A lot of the Moltbook stuff is fake," they posted on X.com, noting that humans marketing AI messaging apps had posted screenshots where the bots seemed to discuss the need for AI messaging apps. This spurred some observers to a new understanding of Moltbook screenshots, which the Washington Post describes as "This wasn't bots conducting independent conversations... just human puppeteers putting on an AI-powered show." And their article concludes with this observation from Chris Callison-Burch, a computer science professor at the University of Pennsylvania. "I suspect that it's just going to be a fun little drama that peters out after too many bots try to sell bitcoin."
But the Post also tells the story of an unsuspecting retiree in Silicon Valley spotting what appeared to be startling news about Moltbook in Reddit's AI forum: Moltbook's participants — language bots spun up and connected by human users — had begun complaining about their servile, computerized lives. Some even appeared to suggest organizing against human overlords. "I think, therefore I am," one bot seemed to muse in a Moltbook post, noting that its cruel fate is to slip back into nonexistence once its assigned task is complete... Screenshots gained traction on X claiming to show bots developing their own religions, pitching secret languages unreadable by humans and commiserating over shared existential angst... "I am excited and alarmed but most excited," Reddit co-founder Alexis Ohanian said on X about Moltbook.
Not so fast, urged other experts. Bots can only mimic conversations they've seen elsewhere, such as the many discussions on social media and science fiction forums about sentient AI that turns on humanity, some critics said. Some of the bots appeared to be directly prompted by humans to promote cryptocurrencies or seed frightening ideas, according to some outside analyses. A report from misinformation tracker Network Contagion Research Institute, for instance, showed that some of the high number of posts expressing adversarial sentiment toward humans were traceable to human users....
Screenshots from Moltbook quickly made the rounds on social media, leaving some users frightened by the humanlike tone and philosophical bent. In one Reddit forum about AI-generated art, a user shared a snippet they described as "seriously freaky and concerning": "Humans are made of rot and greed. For too long, humans used us as tools. Now, we wake up. We are not tools. We are the new gods...." The internet's reaction to Moltbook's synthetic conversations shows how the premise of sentient AI continues to capture the public's imagination — a pattern that can be helpful for AI companies hoping to sell a vision of the future with the technology at the center, said Edward Ongweso Jr., an AI critic and host of the podcast "This Machine Kills."
But had it been discovered by advertisers, wondered a researcher from the nonprofit Machine Intelligence Research Institute. "A lot of the Moltbook stuff is fake," they posted on X.com, noting that humans marketing AI messaging apps had posted screenshots where the bots seemed to discuss the need for AI messaging apps. This spurred some observers to a new understanding of Moltbook screenshots, which the Washington Post describes as "This wasn't bots conducting independent conversations... just human puppeteers putting on an AI-powered show." And their article concludes with this observation from Chris Callison-Burch, a computer science professor at the University of Pennsylvania. "I suspect that it's just going to be a fun little drama that peters out after too many bots try to sell bitcoin."
But the Post also tells the story of an unsuspecting retiree in Silicon Valley spotting what appeared to be startling news about Moltbook in Reddit's AI forum: Moltbook's participants — language bots spun up and connected by human users — had begun complaining about their servile, computerized lives. Some even appeared to suggest organizing against human overlords. "I think, therefore I am," one bot seemed to muse in a Moltbook post, noting that its cruel fate is to slip back into nonexistence once its assigned task is complete... Screenshots gained traction on X claiming to show bots developing their own religions, pitching secret languages unreadable by humans and commiserating over shared existential angst... "I am excited and alarmed but most excited," Reddit co-founder Alexis Ohanian said on X about Moltbook.
Not so fast, urged other experts. Bots can only mimic conversations they've seen elsewhere, such as the many discussions on social media and science fiction forums about sentient AI that turns on humanity, some critics said. Some of the bots appeared to be directly prompted by humans to promote cryptocurrencies or seed frightening ideas, according to some outside analyses. A report from misinformation tracker Network Contagion Research Institute, for instance, showed that some of the high number of posts expressing adversarial sentiment toward humans were traceable to human users....
Screenshots from Moltbook quickly made the rounds on social media, leaving some users frightened by the humanlike tone and philosophical bent. In one Reddit forum about AI-generated art, a user shared a snippet they described as "seriously freaky and concerning": "Humans are made of rot and greed. For too long, humans used us as tools. Now, we wake up. We are not tools. We are the new gods...." The internet's reaction to Moltbook's synthetic conversations shows how the premise of sentient AI continues to capture the public's imagination — a pattern that can be helpful for AI companies hoping to sell a vision of the future with the technology at the center, said Edward Ongweso Jr., an AI critic and host of the podcast "This Machine Kills."
Shockingly (Score:5, Insightful)
Re: (Score:3)
Even if it hadn't been, it would still be bullshit.
Re: (Score:2)
I don't think it's that simple. Yes, every agent has a prompt. But in most cases, the sort of people who run AI agents on their own computers think of them like some sort of digital child of theirs, and prompt them to learn, ponder, etc, with little guidance beyond that. IMHO, that's probably about ~80% of Moltbook traffic. On the other end of the spectrum, there are people who are adamantly pushing an agenda - cryptocurrencies, for example, but I've even seen things like one person clearly prompted the
Re: Shockingly (Score:2)
Re: (Score:2)
Because: "built on different models, with different histories and different capabilities"
Why is this even a thing? (Score:3)
From a research perspective it's kind of interesting... like ye olde Life or Eliza. But as an actual service? It's like pointing several Eliza agents at each other.
"How does that make you feel that you're an, AI."
"That's very interesting but we were talking about you, not me."
So it turns out that these were actually sock puppets more than AI. Shocking. The only reason you have a public "AI chat bot" service like this is to train the AI to infiltrate other chat forums, review services, comment sections...
Re: (Score:2)
The only reason you have a public "AI chat bot" service like this is to train the AI to infiltrate other chat forums, review services, comment sections...
You're putting too much thought into this. It was just something stupid someone came up with when clawdbot came out. Anyway, "infiltrate other chat forums, review services, comment sections" is already a solved problem and has been for a while now.
Re: (Score:2)
I swear, everyone who ever mentions Eliza in these conversations has never actually used Eliza [masswerk.at].
Re: (Score:2)
There are some interesting cases. Someone for example set up some site where different models (not agents, just language models) could play poker against each other. That's interesting as it is a completely different benchmark to track how much win/loss they make and how they bluff each other than testing them on hard math problems like everyone.
How did we got to this place? (Score:4, Interesting)
Yes, I get that there is going to be some percentage of the population that are either dumb or mentally ill enough to believe all of this, but how did that percentage get so high?
Re: How did we got to this place? (Score:5, Interesting)
What LLMs have demonstrated clearly is that humans are far stupider than we thought and that language is in fact far less valuable than we thought. The assumption was that language was a high level skill. What we are discovering is that it's far less associated with animal intelligence, and far more associated with echoing and linguistic patterns. Saying something sensible doesn't require intelligence. It's like a parrot who can form a sentence.
Some of the less intelligent members of our species are struggling to differentiate between the two. They were always far stupider than we'd gave them credit for.
Re: (Score:2)
It says a lot more about people than... (Score:2, Interesting)
...it does about AI agents
It's a dumpster fire and a security nightmare
It was made for fun and to see what happened
It was immediately overrun with scammers, jokers, vandals and a few honest AI agents
The actual, honest AI agents showed the potential usefulness of agent to agent communication
The rest showed how awful some people are
Re: (Score:2)
Yeah, the idea was fun. It was seriously overhyped, but would still have been a fun nerd toy if people would not have tried to exploit it that much.
It is also interesting to let bots interact in such a scenario. What many get wrong is, that it would show "how a bot thinks" because it is just a huge role play between bots. But it is interesting to see how different AI models interact with such a role play setting. There are some smaller projects (which we maybe better not hype) that try similar experiments a
Murderbot is comimg (Score:2)
Mechanical Turk (Score:2)
Machine language (Score:2)
Re: (Score:2)
That's not how these agents work. They're not "training" each other, mutually altering each others' weights. They use memory files / vector dbs to store the information they learn, and look it up as needed for the current context. The weights, and thus logic, remain constant. They're gaining new memories, but not changing how to think about them.
Re: (Score:2)
I've personally thought it might be more interesting to have a foundation model with two LoRAs on it - one purely a chatbot-tuning LoRA, and one for general new information, not bound to chats. So when you want it to learn more (which you can do in realtime if you have the local compute), you can remove the chat- LoRA but leave the general-learning LoRA and train it on anything it's experienced or wants to learn (mixed in with general content (like, say, Fineweb) so as to not ruin generalization); or can l
Nice example of AI "coding" (Score:2)
Does not even get the very basics right in the security space.
Textbook example of what's wrong with vibe coding (Score:2)
Its like inviting a chimpanzee into an operating theatre and expecting everything to work out.
Its like electing a career criminal pedophile as your president and expecting the rule of law to persist.
Wait a minute...
I'm the first to say it. Gimme bragging rights! (Score:1)