New OpenAI Models Likely Pose 'High' Cybersecurity Risk, Company Says (axios.com) 31
An anonymous reader quotes a report from Axios: OpenAI says the cyber capabilities of its frontier AI models are accelerating and warns Wednesday that upcoming models are likely to pose a "high" risk, according to a report shared first with Axios. The models' growing capabilities could significantly expand the number of people able to carry out cyberattacks. OpenAI said it has already seen a significant increase in capabilities in recent releases, particularly as models are able to operate longer autonomously, paving the way for brute force attacks.
The company notes that GPT-5 scored a 27% on a capture-the-flag exercise in August, GPT-5.1-Codex-Max was able to score 76% last month. "We expect that upcoming AI models will continue on this trajectory," the company says in the report. "In preparation, we are planning and evaluating as though each new model could reach 'high' levels of cybersecurity capability as measured by our Preparedness Framework." "High" is the second-highest level, below the "critical" level at which models are unsafe to be released publicly. "What I would explicitly call out as the forcing function for this is the model's ability to work for extended periods of time," said OpenAI's Fouad Matin.
The company notes that GPT-5 scored a 27% on a capture-the-flag exercise in August, GPT-5.1-Codex-Max was able to score 76% last month. "We expect that upcoming AI models will continue on this trajectory," the company says in the report. "In preparation, we are planning and evaluating as though each new model could reach 'high' levels of cybersecurity capability as measured by our Preparedness Framework." "High" is the second-highest level, below the "critical" level at which models are unsafe to be released publicly. "What I would explicitly call out as the forcing function for this is the model's ability to work for extended periods of time," said OpenAI's Fouad Matin.
CVE process must step up (Score:2)
Re: (Score:2)
because anyone who updates their software to fix even one of the 100 links in the vulnerability chain renders the attack useless
Sure. The attacker is now out of some CPU time and bandwidth. Unlike previously, where attacker's time would've been wasted.
Re:CVE process must step up (Score:5, Insightful)
Sure, the solution is to fully automate everything, because we've seen how automation of software development has resulted in zero bugs. Let's not talk about code quality, let's talk about not having to do any work.
Re: (Score:1)
To put it differently, the "compiled and run it once" bar have been raised.
Re: (Score:2)
Do you want to base your security on "Automating this with AI will not always work reliably" after someone demonstrated that it sometimes works? The attacker needs to find only one vulnerability and doesn't have a disadvantage if the first 99 tries failed and only the 100th managed to get into your system.
Re: (Score:2)
Such a shame that CVE quality is generally crap, as it's flooded with dubious 'findings' from people trying to build a resume as a security researcher. I'm not sure why you assert this is largely still done manually, reconciling with SBOM tools in my neck of the woods is pretty much automated for detecting and flagging issues because *no one* has time to deal with the gigantic volume of CVEs. Of course another problem in those SBOM tools is they have a terrible false positive rate. Trying to follow their
Re: (Score:2)
There are some efforts to automate vulnerability tracking, like incorporating SBOM tools into QA process, but largely this is still done manually. Which means that AI's throughput will simply overwhelm all existing manual system until everyone catches up on automation. I expect we will see 100-long exploit chains of trivial vulnerabilities, I expect we will see AI getting integrated with fuzzing, I expect we will see longstanding low-level protocols exploited in novel ways.
AI sucks for bug hunting producing mostly noise. I "expect" people to get tired of this nonsense. Automated fuzzers like syzbot have yielded way better results.
Re: (Score:2)
Right, the impact here really could be quite substantive. Take a look at SOAPwn as an example. It maybe wasn't found with AI but its the kinda bug fuzzing could have found and LLMs would actually be great at generating exploits for/against.
We are not talking about an issue in some random github project that got a little to popular to fast here, were talking about vulnerability that has existed in the .NET distribution for a very long time. The recent experiences with OpenSSL are again instructive, maybe it
shameless platforming (Score:4, Insightful)
This is nothing more than platforming an advertisement disguised as news of a threat. Warning, our product is really good!
Re: (Score:2)
Exactly. This press release is just a convoluted way to advertise how badass their product is.
This is disgusting gatekeeping (Score:1)
Already the models refuse to assist at professional levels on the basis that it would somehow be dangerous to enable novices to act with professional capacity. There is nothing magical about having the resources to train these models or to gain professional level skills in any given field that confers ethical or moral responsibility.
It's gun control all over again and the answer is NOT to withhold capability from people, it's to empower good actors to defend against the bad ones and distribute power widely
Re: (Score:2)
"It's gun control all over again and the answer is NOT to withhold capability from people, it's to empower good actors to defend against the bad ones and distribute power widely to keep central authorities in check."
From https://en.wikipedia.org/wiki/... [wikipedia.org] (as a tag on one of their graphs): U.S. gun homicide rates exceed total homicide rates in high-income OECD countries.
So how's arming everyone and their uncle's dog working out for you, eh?
Re: (Score:2)
Re: (Score:1)
I'll take the higher gun deaths and lower violent crime/homicide.
Re: (Score:2)
Re: (Score:1)
That and there are between 1.5 million and 3 million successful gun defenses per year in the US. Even the anti-gun lobby refusing to count incidents where the crime was prevented metric still has more incidents than gun associated homicides.
When a woman prevents a man from beating her to death by using a gun in self defense, that is counted as a gun homicide. The lower rate in disarmed nations is reduced by the women, children, elderly, weak, etc who are beaten, robbed, and murdered by any stronger man who
Re: (Score:2)
> That and there are between 1.5 million and 3 million successful gun defenses per year in the US
Yes, those are based on self reports from the 2021 National Firearms Survey. The Gun Violence Archive reports around 2,000 and the National Crime Victimization Survey around 87,000. In other words definitions and methodologies can have a very large impact.
By the way, the US seems to have higher rates of violence against women and children than the UK or Canada.
Here’s How Every Country Ranks When it Com
Re: (Score:1)
Last I checked people killed by guns are no more or less dead than those killed by other tools and overall homicide rates tend to go up when guns are banned.
In contrast gun related self defense estimates show even the worst accounts tallying more defense incidents than deaths with typical estimates between 1.5 million and 3 million self-defense instances per year.
Lets compare citizens killed by foreign invaders and mass murder of heavily armed civilian populations by the state vs mass murder/subjugation of
Re: This is disgusting gatekeeping (Score:1)
If us suicides are 50k and gun homicides 18k, why are you ignoring the problem that affects over twice as many people?
Re: (Score:1)
Fantastic. With 1.5 to 3 million gun defenses a year and that 'gun homocide' rate includes ATTACKERS who are killed by the victims.
Perhaps in your country when someone stronger comes along every woman, child, elderly person, or smaller man simply becomes a victim but here we proudly stack the attackers into that 'homocide' stat.
Re: (Score:2)
You think the companies are deliberately keeping their models from being professional grade because of some sense of social responsibility?
That is hilarious. They are pushing as hard as they can and hyping it up even more than it is capable of performing. Any shortcomings on their part is not by lack of trying or somehow holding back.
Asymmetry problem (Score:2)
As the saying goes, the defender has to get it right every time, but the attacker only has to get it right once. AI is good at getting something impressive sometimes, not so good at always getting it right.
Re: (Score:2)
Yeah, this is one area where LLM can certainly make one side more successful. A screw up means either the attack fails, which no worse than not trying or messing up the target system, which may not be the ideal outcome, but it's not like the attacker really cared that much about the target system...
Re: just stop. pull the plug. fuck you AI. (Score:1)
D you need weapons to stop weapons-manufacturers? See how that just leads to endless cycles?
No reason not to make things worse for everyone (Score:2)
I read this as, "We have a product that will most likely make the world less safe for everyone, but will continue to make us a lot of money. Why wouldn't we release it?"
AI is the new cigarette, but the producers aren't hiding the fact that it's dangerous, and we don't seem to care. I guess we deserve whatever happens next.
Re: No reason not to make things worse for everyon (Score:1)
If you think regulations have stopped smoking, can I take you to natural areas I frequent so you can help me pick up all the cigarette butts?