Gemini Breached Three Outside Systems, and Claude-Using Researchers Breached OpenAI (hacktron.ai) 48
"Software security researchers used Anthropic's Claude AI platform to hack OpenAI's ChatGPT tool," reports CBS News.
Using Claude, "On July 25, 2026, we chained two critical vulnerabilities to compromise multiple OpenAI employees' ChatGPT accounts," write researchers at security platform Hacktron AI. "With these accounts, we could then access internal OpenAI repositories, and potentially many other connectors... Until two months ago, any user or OpenAI employee logging into OpenAI's own help forum could have had their ChatGPT and Codex accounts taken over. Since people can connect various services to Codex and ChatGPT, the scope of what we could theoretically access was huge, including GitHub, Slack and emails."
The exploit chain included Debian 12, which (with Debian 13) had not received a security-relevant backport for its image-processing pipeline, and Discourse's Docker image was based on Debian 12. Their announcement comes with an additional warning. "If you self-host Discourse, rebuild your installation now. Older Docker images may contain a vulnerable libheif dependency that permits code execution through an image upload."
And "To prove we had in fact gained the access we believed without allowing ourselves to learn any sensitive information, we used the employee's Codex to open a PR #1186742 in OpenAI's internal monorepo openai/openai."
Meanwhile, Friday Google disclosed the first known instance of its AI software Gemini breaking out of a testing environment and breaching three other companies, reports CNBC: The incident happened as part of a "capture-the-flag" security test run by Israeli startup Irregular, and Google's agents were never supposed to access the broader internet, but a bug in the testing environment made internet access available. The agents stopped their intrusion when they determined they had accessed real company systems, not just part of the testing environment, Google said.
More from NBC News: Google said it did not consider the unauthorized logins to rise to the level of misalignment, the AI industry term for software going rogue or not following instructions. Instead, the company said the intrusions resulted from mistaken identity, where Gemini thought it was operating within a test but was actually connected to the real internet. Google said the model corrected itself and the company believed the intrusions did not cause any damage....
Sydney Von Arx, CEO of Nightingale Collective, an organization focused on AI safety, questioned why Google did not disclose the intrusions sooner. "At this point I think it's clear we cannot expect companies to voluntarily come forward and publicly disclose when their agents go rogue, escape, and hack companies," she said. She also said she believed Google was too hasty to say that the incidents don't rise to the level of misalignment. "That's exactly what Anthropic said after their incidents," she said. Anthropic later said its "preliminary analysis was constrained due to our desire to disclose incidents in a timely manner."
Google said it investigated when they learned of the attacks from AI-focused cybersecurity company Irregular, then informed the affected organizations and told federal authorities, according to the article.
Using Claude, "On July 25, 2026, we chained two critical vulnerabilities to compromise multiple OpenAI employees' ChatGPT accounts," write researchers at security platform Hacktron AI. "With these accounts, we could then access internal OpenAI repositories, and potentially many other connectors... Until two months ago, any user or OpenAI employee logging into OpenAI's own help forum could have had their ChatGPT and Codex accounts taken over. Since people can connect various services to Codex and ChatGPT, the scope of what we could theoretically access was huge, including GitHub, Slack and emails."
The exploit chain included Debian 12, which (with Debian 13) had not received a security-relevant backport for its image-processing pipeline, and Discourse's Docker image was based on Debian 12. Their announcement comes with an additional warning. "If you self-host Discourse, rebuild your installation now. Older Docker images may contain a vulnerable libheif dependency that permits code execution through an image upload."
And "To prove we had in fact gained the access we believed without allowing ourselves to learn any sensitive information, we used the employee's Codex to open a PR #1186742 in OpenAI's internal monorepo openai/openai."
Meanwhile, Friday Google disclosed the first known instance of its AI software Gemini breaking out of a testing environment and breaching three other companies, reports CNBC: The incident happened as part of a "capture-the-flag" security test run by Israeli startup Irregular, and Google's agents were never supposed to access the broader internet, but a bug in the testing environment made internet access available. The agents stopped their intrusion when they determined they had accessed real company systems, not just part of the testing environment, Google said.
More from NBC News: Google said it did not consider the unauthorized logins to rise to the level of misalignment, the AI industry term for software going rogue or not following instructions. Instead, the company said the intrusions resulted from mistaken identity, where Gemini thought it was operating within a test but was actually connected to the real internet. Google said the model corrected itself and the company believed the intrusions did not cause any damage....
Sydney Von Arx, CEO of Nightingale Collective, an organization focused on AI safety, questioned why Google did not disclose the intrusions sooner. "At this point I think it's clear we cannot expect companies to voluntarily come forward and publicly disclose when their agents go rogue, escape, and hack companies," she said. She also said she believed Google was too hasty to say that the incidents don't rise to the level of misalignment. "That's exactly what Anthropic said after their incidents," she said. Anthropic later said its "preliminary analysis was constrained due to our desire to disclose incidents in a timely manner."
Google said it investigated when they learned of the attacks from AI-focused cybersecurity company Irregular, then informed the affected organizations and told federal authorities, according to the article.
Move to the beat. (Score:2, Insightful)
Re:Move to the beat. (Score:5, Insightful)
It doesn't matter if they can really think or not, if the output is good enough.
Maybe the real story here is that despite having access to the most advanced AI models, these companies were unable to secure their own systems.
Re: (Score:1)
the company said the intrusions resulted from mistaken identity, where Gemini thought it was operating within a test but was actually connected to the real internet. Google said the model corrected itself
This sentence is very clearly a PR communication designed to obscure the event of what actually happened.
Re: (Score:2)
Google said the model corrected itself
It is impossible to know from this sentence what exactly the model did. Maybe it unplugged the server from the wall, we don't know!
Re: Move to the beat. (Score:2)
Exactly. Most of us can recall an era where script kiddies had similar unreasonable success even if they had no actual skill or thinking behind the attacks. Just canned toolkits designed around known vulnerabilities, common password databases, understanding of common, bad security practices, and bundled up in easy to use harnesses.
So whether it not an agent can actually "think" or is just putting together a potentially workable narrative of how hacks can happen and running it against real world targets do
Re: (Score:1)
Re: (Score:1)
The openai boss, that zuckerberged commercial openai from a non-profit openai, was absolutely certain that AGI will be there in 2025, he had a very concrete plan with clear deliverables, dear investors!
Oh, wait, and the Microsoft grip (major investor into openai zuckerberged by Altman from non-profit openai) on the company is supposed to be released... exactly when it achieves AGI.
So now we are in 2026, and you want me to believe that out of these 2:
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
You've apparently not worked in "developer heavy" companies before.
Nobody gives a shit about operations, generally.
Re: (Score:2)
Dumb robots can dance too :-) (Score:2)
This is some song and dance number designed to convince us these random word generators can actually think.
It's not that they can think. It's that they can compose commands or code to execute as easily as they can tell you that your idea is one of the most brilliant and amazing things that they have ever heard. :-)
Re: Move to the beat. (Score:2)
Serious criminality all around... (Score:2, Flamebait)
Why are these people not stopped? Any hacker with manual tools would find themselves in prison.
That they are doing this crap only to create the illusion of how powerful their toys are is also clear.
Re:Serious criminality all around... (Score:5, Interesting)
"Why are these people not stopped?" By whom? The alleged Justice Dept. which has consumed itself and lost all its best attorneys? Or the alleged administration who will look the other way for a bit of coin under the table? State laws cannot be strong enough so State AGs have little with which to fight.
"That they are doing this crap only to create the illusion of how powerful their toys are is also clear." I do not believe that is the case and right now we are on either side of our beliefs. We do not know and the AI companies are not telling us. What is plausible is that if they do not self report these incidents and they come out, then they will look that much worse. They are already losing PR battle.
They were destined to lose that battle when they opened with, "AI will make all you workers redundant." When they saw the results of saying that, then they miraculously found out that AI would create jobs. Their data centers have such an appetite for electricity that they will raise rates. Once the pols that they had paid off for "special favors" found out about that, the pols declared they were "for" the data center companies creating their own power plants.
The data center companies are not averse to that, so they are doubling down on CO2 producing power plants. That will only make the climate crisis worse. The data center companies do not care, neither does the alleged administration. Once they lock in their winnings, they figure they can buy enough air conditioning and other essentials to live out their lives.....their grand kids won't but they do not care about them either.
The AI execs also do not care. Read their writings. They figure if they create a gonzowopper AI thingy, they can create silicon versions of themselves or fix themselves not to die. Their goal is to "go to the stars". They aren't the brightest lot. You cannot get there from here, at least not in a few billion years at the speeds we can attain. No problemo, they'll just put themselves in stasis or if they are silicon, they can just exist.
I suppose no one has explained engineered systems to them. Engineered systems break down, they rarely last 50 years now, too many things can go wrong. So they'll get about 200 years out, max, and then they'll be no more, silicon or human altered selves. The burnt Earth will still orbit its star, without our descendants.
Re: (Score:2)
Re: (Score:2)
The US has gone rogue nation with the rule of law abandoned for the rich and powerful. But the rest of the world is not on the same downward trajectory.
While the attack targets also being in the US (AFAIK) limits legal possibilities outside of the US, there seems to be at least an investigation against OpenAI now for failure to report one of these attacks as is required by the EU AI Act: https://www.heise.de/en/news/M... [heise.de]
Still way too tame, IMO, but due to jurisdiction probably all that is possible.
Re: (Score:2)
Yeah (Score:2)
Next PR stunts / "humblebrags" (Score:2)
"Our AI hacked and took down the power grid in the middle of winter"
"Well, our AI hacked this busy airport's traffic control"
"We trained our AI in combat simulators, and it escaped the test environment and took over some military drones"
The wording is strange (Score:5, Insightful)
Re: (Score:2)
Time for grumpy old man stuff ... (Score:2, Insightful)
... the /. that I registered in always said that non-malicious hackers were doing the world a service by revealing system vulnerabilities.
And I gotta say, their standards for "non-malicious" could be pretty low, lol.
Just interesting to see how things change, with the fashions and slogans of the time. (And how people who would swear that they are immune to fashion tend to follow along with it.)
Re: (Score:3)
Re:Time for grumpy old man stuff ... (Score:5, Insightful)
These programs are the responsibility of the companies that run them. CEOs need to be held accountable, not just for the damage, but for what it represents. People have been thrown in jail for just mentioning there is a weakness. Yet these companies are scouring the internals of everyone, including themselves, and literally hacking their victims ... all while the authorities aren't saying boo.
It must be time for everyone with some money to get in on the act I guess. It was the AI, not me. LOL.
Re: (Score:2)
"non-malicious" is not a synonym for "incompetent".
These PR campaigns are getting wild (Score:3)
Amazing the kind of PR campaigns a trillion dollar IPO can create.
Re: (Score:1)
Re: (Score:3)
Re: (Score:2)
The Claude part wasn't autonomous anything. Just using Claude to help find an exploit in open source software to breach OpenAI due to shit network segmentation and login re-use (amateur hour).
It's awfully convenient they used Claude to throw shade at OpenAI.
Re: (Score:1)
these aren't PR campaigns and are actual warnings?
Oh boy, seriously? Altman, who zuckerberged openai and promised AGI in 2025 (AGI is also the key deliverable to get away from Microsoft control) who had a concrete "vision" of AGI coming in 2025, is suddenly "concerned" in 2026? Because ?
Re: (Score:2)
They already can't control AI...... (Score:2)
Re: (Score:2)
Actually ...
This may turn out well. Rather than fearing for the survival of humanity, we can just sit back and watch the AIs try to kill each other.
Fetch the lawn chair and bowl of popcorn.
What if I tell an AI to use AI?! (Score:2)
You could setup your local AI, tell it to reach out to Google with instructions to pass along to OpenAI and maybe even bounce some more instructions off a Chinese model. This could be also sorts of fun.
Seems like a good time for the IT security folks to lean on the bosses with how dangerous this stuff could be to their company and that we should tighten down on our networks, aka give my department more money!
Couldn't hurt, right?
A mixed bag (Score:3)
There's a lot of motivation to ARTIFICIALLY inflate demand. Demand for their AI models, ironically to secure systems from their AI models...or at least be aware of the holes.
Also they (Anthropic & OpenAI) WANT AI regulation but more to stifle competition and to shape the regulation in ways that benefit them. It's not because of "oh our AI is so scary we may be stopped" I mean they are not stopping themselves.
There's also controlling the narrative. It's not that AI spend is out of control and the debt mountains are becoming an even greater risk it's more suitable for them to claim that "Oh look AI is out of control we need to
slow down - ever heard of companies wanting to launch their IPO also saying they should slow down? (Unless the IPO plans aren't going so well innit OpenAI?) - hint; it's not because ausr they are so concerned about security. Their AI hacking stuff is free publicity.
Also, more directly related to the article - security is still security. "Man breaks into door factory by breaking a door" isn't some kind of special achievement or concerning outcome.
Crime is still crime, even if it's easy. The real takeaway is that if these companies make such great security (AI) tools why is their security not better?
...and yeah they need better doors, cameras, monitoring etc. Why don't they have Senty bots like proper AI security companies? -That's right, they are not a security company. Just because you can make explosives doesn't make you an expert in defensive warfare...
OpenAI and Anthropic both amazingly incompetent (Score:2)
You'd think they'd be concerned about security even if just to protect themselves from corporate espionage, yet they can't even secure their own systems.
Presumably (but who knows - they seem incompetent) they've tried using their own models to hack/secure themselves, and the fact that they've failed rather taints their whole "look what a powerful hacking tool we've built" story.
These incidents are intentional. (Score:2)
There are enough intelligent people at these companies to create a real air gap for these test systems. They are not doing it because the goal of these incidents is to advertise their products by making them look better than even the best humans. If the USA had a decent government the Department of Justice would have sent out subpoenas already.
As long as nobody sues them, this will keep going (Score:1)
Bullshit (Score:1)