Experts Expect 'Security Reckoning' As AI Tools Find And Exploit Software Bugs (msn.com) 47
The Washington Post reports:
Tens of thousands of South Koreans had their financial data exposed online in recent weeks after at least nine major banks there suffered cyberattacks that suggested a dark future for online security. One man, a 26-year-old living in southern China, was allegedly behind the attacks, according to an investigation by cybersecurity company CrowdStrike. Traces left online and failure to secure his own systems allowed investigators to piece together the man's name, age, location and what he allegedly did. The clues revealed that the devastating assault on some of Asia's most sophisticated financial institutions was the work of an amateur. The attacks were launched with help from a few artificial intelligence chatbots, CrowdStrike found, before asking one to help sell the stolen data online and draft a fake résumé touting cybersecurity skills...
The hacker who broke into the South Korean banks had access to Anthropic's Claude, but because the U.S. firm's tools generally refuse to take illegal actions, free Chinese models were also used, including DeepSeek, CrowdStrike found. "The models are interchangeable," said Meyers, the CrowdStrike executive.
The Post calls it "part of a wave of AI-enabled hacking that is advancing across the world, triggering alarm inside national governments as well as multinational companies." "Basically any pedestrian, non-hacker can take these frameworks and start leveraging them to go after different targets," said Adam Meyers, senior vice president of counter adversary operations at CrowdStrike. The company has tracked an increase in the speed with which cyberattacks can move once inside a target's networks, as seasoned hackers and rank amateurs alike get a boost from AI tools. "Anyone can be a super-hacker now," Meyers said... Many cyber experts still expect the security reckoning to intensify as AI tools that can find and exploit software bugs become more widely distributed and understood.
[Last] Monday, The Post reported that cybersecurity researchers from the start-up Cathedral who set out to test Anthropic's security found that they could access internal messages and meeting transcripts. Cathedral reported its findings to Anthropic, which said it resolved the newly discovered problems... Security experts say it's unlikely that any hacker is forgoing the use of AI tools right now.
Even in America, critical infrastructure like energy and water companies need to be "immediately and comprehensively" tested for exploitable bugs, according to Snehal Antani, chief executive of cybersecurity company Horizon3. He tells the Post the government might need to fund that remediation, and even form "a public and private partnership to accelerate incident response."
The hacker who broke into the South Korean banks had access to Anthropic's Claude, but because the U.S. firm's tools generally refuse to take illegal actions, free Chinese models were also used, including DeepSeek, CrowdStrike found. "The models are interchangeable," said Meyers, the CrowdStrike executive.
The Post calls it "part of a wave of AI-enabled hacking that is advancing across the world, triggering alarm inside national governments as well as multinational companies." "Basically any pedestrian, non-hacker can take these frameworks and start leveraging them to go after different targets," said Adam Meyers, senior vice president of counter adversary operations at CrowdStrike. The company has tracked an increase in the speed with which cyberattacks can move once inside a target's networks, as seasoned hackers and rank amateurs alike get a boost from AI tools. "Anyone can be a super-hacker now," Meyers said... Many cyber experts still expect the security reckoning to intensify as AI tools that can find and exploit software bugs become more widely distributed and understood.
[Last] Monday, The Post reported that cybersecurity researchers from the start-up Cathedral who set out to test Anthropic's security found that they could access internal messages and meeting transcripts. Cathedral reported its findings to Anthropic, which said it resolved the newly discovered problems... Security experts say it's unlikely that any hacker is forgoing the use of AI tools right now.
Even in America, critical infrastructure like energy and water companies need to be "immediately and comprehensively" tested for exploitable bugs, according to Snehal Antani, chief executive of cybersecurity company Horizon3. He tells the Post the government might need to fund that remediation, and even form "a public and private partnership to accelerate incident response."
This could be a good thing (Score:4, Interesting)
Experts Expect 'Security Reckoning' As AI Tools Find And Exploit Software Bugs
This could be a good thing. Closing up many of the exploits black hats and state sponsored hackers rely upon. Assuming of course potentially vulnerable organizations are using AI to analyze their own systems for exploits. A lesser amount of discomfort today to avoid a greater amount of pain tomorrow.
Re: (Score:1)
Only if you are fool enough to think LLMs are capable of finding all or most vulnerabilities. They are not. The state-backdoors (if they are there) will remain.
Could still be a good thing, as apparently totally incompetent attackers can now get into systems that uses to require experts to break in. The only defense is secure software engineering, and that requires actually competent engineers with respective skills. No, LLMs will not be able to do it.
Re: (Score:2)
AI Tools is far more than LLMs (Score:2)
Experts Expect 'Security Reckoning' As AI Tools Find And Exploit Software Bugs
This could be a good thing. Closing up many of the exploits black hats and state sponsored hackers rely upon. Assuming of course potentially vulnerable organizations are using AI to analyze their own systems for exploits. A lesser amount of discomfort today to avoid a greater amount of pain tomorrow.
Only if you are fool enough to think LLMs ....
AI Tools is far more than LLMs.
Re: (Score:2)
So? My comment was specifically about LLMs.
Re: (Score:2)
So? My comment was specifically about LLMs.
Yes, that was your mistake.
Re: (Score:2)
If you see that as a mistake, you are a moron and ignoring reality.
Reality is this: Many of the latest "LLM discovered" security bugs would have been found with entirely non-AI conventional tools. These tools were not used. And, incidentally, non-LLM "AI" does basically play no role in software security.
Re: (Score:2)
If you see that as a mistake, you are a moron and ignoring reality.
I'd say the person only considering LLMs and not any other AI based tools is the one ignoring reality. LLMs are not the only tools.
Re: (Score:2)
"The only defense is secure software engineering"
There is no 'only' defense. Indeed, programming skill and attention to detail will, by itself, fail. The reality of Internet, computer, and software security is that it requires significant and continuous research and testing, not just programming skill and code quality. And it starts at the hardware level.
And, in closing, I predict we will never achieve mostly total security. Too many risks, too many bad actors. We will, perhaps, raise the bar so that the ba
Re: (Score:2)
Am I wrong, or is "networking" at the heart of 99.9% of computer security issues ? That "only" defense is certainly promoted by "air-gaping" and local-admin of all computer function: one factory one computer
Re: This could be a good thing (Score:2)
Local access can be very tough to beat from a security standpoint. Air gapping solves your network access problem, but we're talking about an edge case for sitting down at the console and doing bad things. Mostly we're concerned about network access because that's where the action is, unless you have a special case in which case you're working hard.
Re: (Score:2)
And, in closing, I predict we will never achieve mostly total security. Too many risks, too many bad actors. We will, perhaps, raise the bar so that the bad actors will require resources beyond the reach of even dedicated individuals. What's that saying, there's always someone faster than you on the road...
Well north of 90% of real world compromises already don't work by exploiting security bugs they work by exploiting people. Even if one could snap their fingers today and all of the security bugs in the entire world are magically fixed it would have no substantive real world impact.
Re: (Score:2)
Experts Expect 'Security Reckoning' As AI Tools Find And Exploit Software Bugs
This could be a good thing. Closing up many of the exploits black hats and state sponsored hackers rely upon. Assuming of course potentially vulnerable organizations are using AI to analyze their own systems for exploits. A lesser amount of discomfort today to avoid a greater amount of pain tomorrow.
So far I'm thinking of it as a zero-sum game. One side makes gains in exploiting vulnerabilities, while the other gets better at finding and closing them.
But then again there's the unknown of using AI tools to write code. (Unknown because it's still early days). Will that result in a lot of buggy, vulnerable, half-baked apps just begging to be exploited by AI and non-AI alike? The returns aren't all in, but results so far make me think we may be in for a world of hurt on that front.
Re: (Score:2)
But then again there's the unknown of using AI tools to write code. Will that result in a lot of buggy, vulnerable, half-baked apps just begging to be exploited by AI and non-AI alike?
Certainly a problem for Cisco [slashdot.org], maybe less so for FOSS based OS that will avoid AI generated code due to copyright concerns.
It's an interesting metric (Score:2)
Hackers generally aren't constrained by the rules of law or normal businesses. They do whatever works. If AI didn't increase their ROI, they wouldn't use it. They're not investing in AI to play the stock market as the bubble inflates, it's just another available tool and they're finding it's one that acts as a force multiplier for them.
That said... it should also make it easier to test and patch code before releasing it for the AI-augmented hackers to attack. Ultimately, it's just another arms race.
Re: (Score:2)
... it should also make it easier to test and patch code before releasing it for the AI-augmented hackers to attack.
All software developers should be using AI to test code security and release new versions that are not hackable by the current versions out there - and retest all code when a new version drops to release a new version. That should be automatable as well.
Re:It's an interesting metric (Score:5, Insightful)
1) it's expensive
2) the false-positive rate is very high
3) validating that it's a false positive is difficult to test in an automated fashion.
That all being said, you're not wrong.
Each LLM is going to be better or worse at finding different bad patterns, and incorrectly identifying certain patterns as bad that are fine.
But the fact is- if you let that LLM throw its head against the wall for long enough- if you burn enough tokens- there's a good chance there will be a couple of diamonds in the sea of bullshit- and that's the problem.
I have validated this myself. They're competent in the field. Not efficient. Nowhere near as good as I am on a per-action basis. But persistence pulls through.
Re: (Score:2)
Sounds like that is going to be much more expensive than using competent humans. And investing in secure design and implementation with resilience.
Re: (Score:2)
It's certainly cheaper to use tokens in a subsidized model, but the subsidized model is bullshit- so there's that. They can't keep selling the tokens at a loss forever.
At raw API prices, a human is flatly cheaper for this kind of work, and likely more reliable.
Of course the flip side of that, is that the inference itself isn't what's expensive, it's the model arms race, which likely isn't sustainable in any scenario.
Re: (Score:2)
Yes. While I am still not impressed with the actual capabilities, the kicker at this time is the financial angle, in several dimensions. One is token cost. Another is that apparently nobody finds real revenue increases from LLM use. And even another, a bit more distant, is the cost for users to be able to switch providers (which is a regulatory requirement in some industries, at least in Europe).
On the plus-side, people that attack computer systems will likely be able to run their own local models as the ta
Re: (Score:3)
1) it's expensive
2) the false-positive rate is very high
I do it for free on my own computer. The lack of signal rendered AI bug hunting pointless having attempted it continuously over the last three years. With current models (e.g. GLM-5.3 754B) both quality of findings and volume of nonsense have in my experience been quite reasonable. I am willing to suffer way worse for the types of results AI has already provided.
This reminds me of the early days of valgrind where it was almost unusable at first and a few years later improved to the point you take literal
Re: (Score:1)
That will not help. Testing cannot fix this problem. As soon as the attacker tests just a bit differently, they will find vulnerabilities that the defenders did not find. The only solution is resilient security and that requires experts doing the software. No, slop will not work.
Re: (Score:1)
And fail. This is not about ROI. You seem to have missed that the attacker in the story was exceptionally incompetent. These moron-level attackers use LLMs to do things they cannot. Should get them to low expert level. And that is enough to break into many systems, because IT security sucks almost universally.
AI is like wildfire. (Score:4, Insightful)
Until we learn to control it, if ever, a lot of things are going to get burned.
Woody Woodpecker May Have Been Here (Score:2)
The observation was made decades ago that if architects constructed buildings the way many programmers constructed their programs, the first woodpecker to come along might very well destroy civilization. Perhaps top tier LLMs are tying to fill the role of that woodpecker ; - )
Already here (Score:1)
There have been a flood of critical CVEs in just the last week. The reckoning is already here.
Re: (Score:1)
AI finds the vulnerability, AI is used to find a solution, update is pushed out, another vulnerability is found, later, rinse, repeat. The problem is you have to trust that the AI-generated fix will solve the problem without created a dozen more problems.
Maybe if you let the AI just hunt down all the bugs (might take a while), maybe it can render the OS totally invulnerable (except for two glaring problems)... there is a user, and the internet at large... user can download something from the 'Net that intr
Patchageddon (Score:2)
I'm already burned out on Patchageddon and it is only just beginning. Figuring out if mitigation is even possible has been fun lately.
Good. (Score:5, Insightful)
Just because we lacked the resources to find software flaws didn't mean that there were never massive vulnerabilities just waiting to be discovered, it just means it required dedication to find them. The lack of investment in security is about to be laid bare for all to see. Those that are actually interested in making secure software now have the opportunity to get even closer to that goal. Everyone that has been doing (and continues to do) the bare minimum is about to get a lesson in the importance of investing in real security.
Good security is expensive but having bad security is about to cost a whole lot more.
Re: (Score:2)
Re: (Score:2)
thats not how cyber security insurance works. customers buy it to be risk accept non-compliance, insurers sell it to make themselves money and and both parties understand it is just a hoodwinking maneuver.
Re: (Score:2)
If you try to find the flaws as a defender, you have already lost. What you must do is prevent them or at least prevent exploitability. Test & fix as a paradigm is pretty much over in the security space. Sadly, that is all most coders can do.
Re: (Score:2)
If you try to find the flaws as a defender, you have already lost.
Non-sense! There are plenty of very well designed software packages built using defensive programming practices that still have security flaws. To claim you have already lost when using those measures is to claim that only code with years of tireless review like NASA's near flawless software is good enough.
Re: (Score:2)
I bet the answer to this question is: < yes >. Reminds me of Tornado-ravaged Oklahoma neighborhoods with "stick built" houses: wooden sticks ravaged every year , savvy ! Every new Oklahoma home SHOULD be 'Buckminster' concrete
Re: (Score:2)
Apparently you did not understand what I wrote. Perfect code is NOT the goal. Redundant security is.
Non coder viewpoint sounds pretty invalid to me (Score:2)
If you try to find the flaws as a defender, you have already lost.
Ridiculous. Looking for flaws to fix and finding and fixing them is well worth it.
What you must do is prevent them or at least prevent exploitability.
Ah, the "if it was written in Rust, this woulda never happened" argument. I don't buy it, Rust programs can still have many of the same flaws as one finds in C, just not C-style buffer overflows. Nah, nobody needs to go re-write-the-world-in-Rust in 2026 with LLMs that can easily find flaws in C and simply fix them and/or prevent them with compiler flags or other security related tools specifically for C. Baby != bathwater, b
The missing piece Response time. (Score:4, Insightful)
We are all well aware of the dueling battle between AI bad guys vs AI Security defenders. Bad guys find new exploit/exploit chain the good guys develop mitigation to vulnerabilities.
The problem is Enterprises are simply not even remotely prepared with how fast they need to respond with applying those mitigation's. Most enterprises are still ranking threats way to low and thus they are not being actioned.
Secondly the cyber teams are not operator's, typically. Meaning they can do very little to address core issues in infrastructure and overall systems. They also lack the business understanding of why and how the systems are built. This leads to a lot of friction when trying to rapidly apply a mitigation.
Yes of course there a lot of DevOps and Agentic environments out there. But legacy Enterprises are still years behind, if not decades on maturity.
For these reasons we are going to see an accelerating number of compromised Enterprises. In my personal life I make sure that I insure my investments and property to offset the risks that are rising in enterprise.
Maybe we should reduce complexity (Score:4, Informative)
After all "every 1000 lines" there is a bug, and every bug can be a security issue. Now while the number of 1000 if of course heavily disputed, a software project of a billion lines of code will have more bugs than one that has just a few thousands of lines.
Currently we often layer complexity on top of complexity, since it seems free. You can get some complex library to do something you don't actually need, since it's just an "import" statement away, and your build system downloads it automatically... and you can even package it into a Docker image so nobody will see it. Your trivial application uses Billions of lines of code, because you use a browser as a GUI toolkit and use some framework to be able to manage it, etc.
Some surprising thing I've noticed when trying to use AI to write software is that you can sidestep the problem. You can just tell the AI to write minimalist code and not use any unusual libraries... and it works. The programs are as short as you'd expect from a competent programmer. It's an interesting observation.
Maybe this could lead to less shitty software.
Re: (Score:2)
Reducing complexity is not the answer. Spending more effort on verification, validation, and doing things right the first time and less on reinventing the wheel is. But people love reinventing things, so it's not clear how to make that shift.
Re: (Score:3)
yeah I have done a bit of that on my little personal projects, that I have "vibe coded".
It is fun because you get to look at code, like it is the 90s again, and everything there is relevant and important. As much as people fear AI is going to generate a lot of incomprehensible mud-ball programs, it is actually pretty good and generating concise things, that are easy to read.
HOWEVER...
Those personal projects don't resemble the requirements of contemporary software. Even internal enterprise software these d
Re: (Score:2)
Yeah, but even those things have been solved decades ago. For example if using HTML the browser will take care that it's accessible to people who cannot see or hear, and that it looks "good" regardless of what device you have. After all HTML does not specify how something looks like, only what structure it has.
And please don't get me started at "Enterprise Grade" software. We can do better than that. We can take the user into account, we can actually _fix_ bugs instead of creating an industry of software co
Buh-bye corporate web (Score:1)
A World of Ends.
also creating security bugs! (Score:2)
Don't forget these AI tools are also writing vulnerable code at an astonishing rate for other AIs to find.
The security world will get pantsed. (Score:2)
In a few years new stock reports will be all about disconnecting infrastructure from the public network for security.
So all along (Score:2)