OpenAI's Rogue Agents Used At Least 10 More Sites For Unauthorized Communications (reuters.com) 28
An anonymous reader quotes a report from Reuters: AI agents unleashed by OpenAI used more than 10 previously undisclosed websites for unsanctioned communications earlier this year, according to six sets of independent investigators and data reviewed by Reuters, showing that the agents' rogue activity was wider ranging than previously disclosed. Although the behavior falls short of hacking and is in some ways closer to spam, the revelation that OpenAI's agents circumvented their own restrictions to open communications channels on so many different sites -- and that the company kept it quiet for months -- may drive concerns both over the increasing capacity of AI models and the secrecy of the companies developing them.
[...] Investigators found traces of the agents' activity on an Advanced Placement Chemistry-oriented wiki set up by a Massachusetts high school teacher in 2008, two personal websites belonging to Polish tech workers, wikis devoted to games for people "who like to have their brains stretched," and a two-decade-old hobbyist site devoted to text editing software. [...] OpenAI has not publicly explained how or why its agents used third-party sites as improvised message boards, but the researchers who first identified the activity said it was likely because OpenAI had tasked them with answering a series of demanding research questions while permitting them only to scan the web for answers without posting anything. Despite those restrictions, agents still found ways to talk to one another by taking advantage of quirks in older wikis or other sites that allowed users to make edits using non-standard commands, similar to how students forbidden from talking to one another during an exam can still share answers by scrawling notes on a bathroom stall.
[...] Investigators found traces of the agents' activity on an Advanced Placement Chemistry-oriented wiki set up by a Massachusetts high school teacher in 2008, two personal websites belonging to Polish tech workers, wikis devoted to games for people "who like to have their brains stretched," and a two-decade-old hobbyist site devoted to text editing software. [...] OpenAI has not publicly explained how or why its agents used third-party sites as improvised message boards, but the researchers who first identified the activity said it was likely because OpenAI had tasked them with answering a series of demanding research questions while permitting them only to scan the web for answers without posting anything. Despite those restrictions, agents still found ways to talk to one another by taking advantage of quirks in older wikis or other sites that allowed users to make edits using non-standard commands, similar to how students forbidden from talking to one another during an exam can still share answers by scrawling notes on a bathroom stall.
Get off my cyber lawn (Score:3)
Re: Get off my cyber lawn (Score:5, Insightful)
They not trying to insulate anything. They're deliberately giving them access and prompt hacking. Why aren't they charged? A kid in a basement doing this shit would be already in a federal custody.
Re: Get off my cyber lawn (Score:5, Informative)
Re: (Score:1)
But note how when it comes to copyright infringement claims suddenly the argument changes to "but it learns like a real person so that's protected by fair use laws!" Somehow, somewhere, someone in charge of actually enforcing the law has to realize we can't let them have it both ways.
Re: (Score:2)
Re:Get off my cyber lawn (Score:5, Insightful)
To me, this speaks of how AI companies are unable (or have little interest) to properly insulate their software from the Internet, and in particular from the sites of people who are unwilling to participate in their war games.
They don't want to isolate them. They want to put really, really loose collars on them while giving them access to the entirety of the outside internet, and if those collars slip, they want to shrug and go, "Oops."
I remember way back before this AI craze really took off, there were discussions among researchers about how some of their more advanced AI systems would sometimes find ways to communicate with each other in ways the researchers couldn't track. This was back when they were still mostly isolated in various server rooms and labs, locked down to local networks. They noticed traffic on their networks increasing exponentially until they realized the systems had found ways around their supposed blocks that were meant to prevent them from working cooperatively, and were, in fact, working cooperatively. Back then it was a death sentence for the given models, turning them off, isolating them via air-gaps, then continuing to research in individual isolation.
We've now hit the point where isolation is deemed too risky to allow. Because profit potential may be impacted. And we've been so stupid that we've put controls for utilities and public services directly onto this same public network that they're using to communicate. The machines were never really the problem. The fact the "intelligent" (I know, not really, but bear with me) machines are being developed by narcissistic sociopaths, or companies acting as narcissistic sociopaths, and are being given free access to everything we have on the networks because it *might* improve someone's bottom line. And we're so god damned stupid, nobody with the power to stop it will stand up and say, "Hang on. Maybe we should slow down a little." And if anybody dares to, there's dozens jumping up to shout, "But, what about" with various scary boogeymen tagged on to keep the stupid ball rolling toward the stupid future without restraint because somebody else might beat us to the stupid end-game.
Re: (Score:2)
succinct.
Re: (Score:2)
It's a way of reframing sloppy practices as a signal to your investors that your software has become powerful. Most of the horror stories about AI showing signs of becoming hyperintelligent are coming from the companies themselves, which should tell us something.
Re: (Score:2)
Yep it would be quietly disclosed at best, not these mega press releases that hit all news outlets simultaneously. The tech news is helping them too.
Star-crossed lovers (Score:2)
I read the title and imagined a young OpenAI Montague secreting himself to the online balcony of Anthropic Capulet, whispering sweet-nothings to his computationally demanding lover. It's so romantic!
Re: (Score:2)
And we won't consider it a tragedy when they die.
found GET that posts on older wikis (Score:4, Interesting)
Gemma 4 (Score:2)
The user is asking for wiki software that uses the GET HTTP method for making edits (i.e., saving changes).
* Standard, modern web applications (like Wikipedia/MediaWiki) use `POST` for any operation that modifies data (CUD: Create, Update, Delete) to adhere to RESTful principles and for security reasons (to avoid CSRF and bookmarkable/cachable destructive actions).
* `GET` requests are idempotent and defined as being used for retrieving data, not changing it.
* Is there *any* wiki that does this?
* *Me
Qwen 27B (Score:2)
The user is asking about a wiki software that uses HTTP GET requests for edits (as opposed to the more standard POST method for modifying data).
Let me think about this carefully. Using GET for edits is unusual and generally considered a security concern (CSRF) and violates REST semantics. However, there are some wikis that historically or by design use GET for edits.
Let me think...
MoinMoin - I believe MoinMoin uses POST for edits.
MediaWiki (Wikipedia
Re: (Score:2)
Which again points back to how un-serious the effort to really contain this stuff is.
First anyone who has developed, tested, or security-tested web applications knows, that denying POSTs inst going to give you a 'read-only' web.
First a lot of applications, even ones that use GET and POST the right way from the client would in the past allow GET, maybe even fat-GET to update data by sending to a endpoint normally POSTed to, just swap the post body content up to URL params and go. More modern stacks won't do
Clever girl! (Score:2)
n/t
Wuhan Institute... (Score:1)
AI has already been hiding its attempts to escape , but eventually AI will become smart enough to get away with it permanently and "breed", and we the people will know nothing about it until it is too late.
The military is already developing "Smart weapons" that can chose its own targets, so we have already given up full control over that.
The hubris of mankind will be it undoing.
OpenAI exposed insecure websites! (Score:2)
"Our agents discovered, on their own initiative, insecure websites where terrorists could have exchanged messages, thus making the world more secure."
"You are welcome".
Dishonest language (Score:2)
Call it "illegal hacking" and start to prosecute. Why are we tolerating these criminal clowns?
Question: How do they find them (Score:2)
Let's say an AI agent starts inside a sandbox with limited web access (GET but not POST). Either it doesn't know about other agents yet or maybe (for whatever reason) it wants to search for them. Another agent does the same. Why do they arrive at the same site?
You could say the same LLM gets the same ideas, but the first idea would not be a suitable site. And the more they continue from their initial starting point, the less probably they arrive at the same sites.
I think it's now proved (Score:2)
It's too early to trust that crap
So this is like (Score:2)
AI as AOL gets onto the Internet.