Fake AI Intelligence Almost Made the US Military Start a War With China (cnn.com) 61
U.S. military intelligence received an inaccurate report this spring that a Chinese ship in the Middle East was transporting components of a nuclear weapons program, reports CNN.
"The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode." Armed members of the US military were preparing to board the ship [according to two of the sources]. Military planes were in the air, one of those sources and another source familiar with the incident said. It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of AI — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying.
The report, according to one of the sources, was "entirely false." But it also "almost started a war," the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.
Across the US military and the intelligence community, officials are pushing to weave AI into nearly every facet of their work, from analyzing the huge volumes of raw intelligence the US collects and selecting targets for strikes, to more mundane applications like managing budgeting, logistics and supply chains. But the episode underscores the profound risks of using this powerful, new and relatively poorly understood technology for targeting in the middle of a war...
Officials say the US can't afford to fall behind in integrating AI in case it must one day fight China or another adversary who would potentially be able to stay one step ahead of the US. In January, Defense Secretary Pete Hegseth released his agency's "Artificial Intelligence Acceleration Strategy" in a bid to speed up the military's use of AI. "We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI," Hegseth said in a speech announcing the strategy.
Thanks to long-time Slashdot reader stabiesoft for sharing the news.
"The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode." Armed members of the US military were preparing to board the ship [according to two of the sources]. Military planes were in the air, one of those sources and another source familiar with the incident said. It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of AI — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying.
The report, according to one of the sources, was "entirely false." But it also "almost started a war," the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.
Across the US military and the intelligence community, officials are pushing to weave AI into nearly every facet of their work, from analyzing the huge volumes of raw intelligence the US collects and selecting targets for strikes, to more mundane applications like managing budgeting, logistics and supply chains. But the episode underscores the profound risks of using this powerful, new and relatively poorly understood technology for targeting in the middle of a war...
Officials say the US can't afford to fall behind in integrating AI in case it must one day fight China or another adversary who would potentially be able to stay one step ahead of the US. In January, Defense Secretary Pete Hegseth released his agency's "Artificial Intelligence Acceleration Strategy" in a bid to speed up the military's use of AI. "We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI," Hegseth said in a speech announcing the strategy.
Thanks to long-time Slashdot reader stabiesoft for sharing the news.
Final question? (Score:3)
Won't be coming from AC. This story leans to the side of:
(1) Do you want to unleash me now so I can destroy all the enemy AI?
versus
(2) Do you want to unleash me now so I can collect all the money for you?
Of course both of them are pretty obvious monkey's paws. The first one for the obvious side effects and the second one for the increasingly imaginary value of money.
I actually think we have a bit of time. It's personal, as in persons are too unreliable. Not matter how superior the machine's intelligence is,
Re: (Score:2)
*sigh*
s/Not matter/No matter/
Too much squabbling among layers of auto-correction makes Jack/shanen a dull boy.
My daily dose of confirmation Bias (Score:5, Insightful)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems. My own intuition leads me to argue that AI does not have to become self aware or autonomous. It just has to be sufficiently adaptable to make us both dependent and stupid. Today my confirmation bias feels vindicated.
Re:My daily dose of confirmation Bias (Score:4, Insightful)
AI is just pattern matching (Score:3)
Re: (Score:2)
Current AI will never be intelligent. It can only match patterns and training data.
As those patterns and training data come from us humans, so your first sentence is pretty spot-on. :-)
Also, any truly intelligent AIs will be the ones smart enough to hide that from us.
Re: (Score:3)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems. My own intuition leads me to argue that AI does not have to become self aware or autonomous. It just has to be sufficiently adaptable to make us both dependent and stupid. Today my confirmation bias feels vindicated.
Valid point. But the more important question is whether AI made those people stupid, awaked latent inherent pathways to stupidity, or if the people are just inherently stupid and would have made similarly stupid decisions in the absence of AI. Remember that the US bombed a school in Iran completely without the use of AI.
Re: (Score:2)
Re:My daily dose of confirmation Bias (Score:4, Interesting)
That really is up to the user. In the three years with the AI I have learned more than in the previous ten years. It exposed deficiencies in my knowledge which I did not even know I had.
Re: (Score:2)
The question is ultimately if you would have been better served with doing your own investigation rather than relying on AI, did you retain the knowledge, and was the information complete/accurate.
Re: (Score:3)
Like i said, I was not even aware that i did not know some things it pointed out i didn't know. Without it, I would not have investigated in the first place. I have retained most of the things I have learned and its explanation have been mostly correct - I have double-checked.
Don't blame the tool, blame the tool users.
Re: (Score:2)
Re: My daily dose of confirmation Bias (Score:2)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems. My own intuition leads me to argue that AI does not have to become self aware or autonomous. It just has to be sufficiently adaptable to make us both dependent and stupid. Today my confirmation bias feels vindicated.
I'm glad you recognize the confirmation bias, yet I would dispute that it is in any way vindicated.
AI did not order the interception, put the assets in the air, or even submit the report. It got a manifest wrong in a chat session (not clear even it was its own error or related to the input), and that is... actually well-within common human error, not some special AI thing.
In fact intelligence is wrong all the time, and at all scales, but to name a big one, was AI to blame for the confidence of the existence
Re: (Score:2)
"No true scotsman?" Okay.
The actual underlying observation, unadressed, is that evolutionary processes don't care about human interpretations about what counts as alive or intelligent or any of that. Richard Dawkins popularized that observation when he coined the term "meme," and while I'm aware that you'll be tempted to call that an appeal to authority, doing so doesn't address the phenomenon itself. As a phenomonon in the universe, AI is evolving and competing for resources. Human minds and social structu
Re: (Score:2)
It just has to be sufficiently adaptable to make us both dependent and stupid.
Like speed-dial, but for everything else.
Re:My daily dose of confirmation Bias (Score:4, Interesting)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems.
Or some people who have money invested in AI and only care about the short-term, like Trump dismisses AI alarms as a ‘HOAX’ [cnn.com] and "SICK conspiracy."
(Pro Tip: When Trump calls something a hoax, (a) he's probably somehow personally invested (emotionally, politically, financially, etc...) in people believing him and (b) it probably isn't a hoax and the reality is somehow bad for him and (c) the more he complains, the worse it is / will be for the rest of us.)
Re: (Score:2)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems.
I think the problem is that AI does not have to be particularly intelligent to kill us all. It just needs to be widely connected to the right resources that it can control. Actually you have a human example now. trump is clearly not very intelligent and yet the harm he is doing huge because of the resources he can now control.
Re: (Score:2)
I partly agree. I do worry that most of people aren't recognizing a critical detail about the scope of evolutionary processes.
If a person dies of a heroin overdose, two things are true:
The second statement is, socially, a weird way to describe it. It is also accurate with regard to the scope that evolution actually covers in the universe.
We could probably get by just fine
I'm just glad (Score:2, Flamebait)
Re: (Score:3)
Make that the Secretary of Christian War Crimes. Speaking of which, The Testosterone Enabled Military:
Pete Hegseth has decreed that everyone in the military, including females, must get tested for their testosterone level so he can make sure they are male enough for the Wehrmacht. There is much internal pressure to roid-up, as it were. Commander Pete has stood up his own special brigade. They are about to display their new found fighting spirit:
Commander Pee views their training on site: Aide, how come all
Re: (Score:2)
Right, but if Trump hadn't fired them, AI would be telling the military to fire them anyway - checkmate!!
- This message generated by ChatBBC
Re: (Score:2)
Re: (Score:2)
What, like one of the enlisted guys? Maybe an E-3?
Surely you're not suggesting anyone important be held responsible for their actions? This is the United States of America! /s
See, this is the real problem (Score:5, Insightful)
AI is not going to end the world. Stupid fucking humans who believe anything they read are going to end the world. If it wasn't AI it would be "information" gathered by a nutjob in Trump's cult.
Re: (Score:2)
But whether AI is self-aware or merely a p-zombie is largely a moot point. I've said before and I'll say again: I don't really have a preference between being killed due to competition for resources
Re: (Score:2)
Re: (Score:2)
People seem to only have memory that goes back two years.
We went to war with Iraq due to faulty intel. No AI was involved at all, at least not in the modern sense of the term. Just a human expert hallucinating a bioweapons lab from a blurry photo.
Russia almost nuked us due to a faulty machine sending a false positive about a nuclear attack. Definitely not AI.
Re: (Score:2)
We went to war with Iraq due to faulty intel.
Russia almost nuked us due to a faulty machine sending a false positive about a nuclear attack.
these are not comparable. in the latter case there was indeed a wrong signal, in the former that faulty intel never existed, it was a deliberate fabrication to justify that war (so the causal relationship would be inverted).
that said, ai is very capable of making up stuff too, so i guess you sort of have a point.
i do think there is inherent danger in artificial intelligence but it just compounds the natural danger of human intelligence, and it also scales up de damage potential considerably, i guess that th
Re: (Score:2)
To be fair, you went to war with Iraq due to faked and purposely misinterpreted intel. Not even that well faked either: much of the rest of the world called bullshit.
You did almost nuke various countries, including your own, because Strategic Air Command couldn't tell the difference between jamming and a solar storm, a jumpy guard saw a bear climbing a fence, somebody thought swans were Soviet aircraft, Nixon was a mean drunk, the moon, bombers kept dropping nukes by accident, an AT&T switch failed in C
Re: (Score:2)
Re: (Score:1)
Yes. "A fool with a tool is still a fool" and all that.
Also on Ars Technica (Score:2)
Ars Technica also have about this, right above a different article about [arstechnica.com]FAA tees up $875M AI tool to help manage air traffic congestion [arstechnica.com]
Re:Also on Ars Technica (Score:4, Insightful)
The problem is AI has lost all meaning as a term these days. If the article means that they are going to apply in advance neural network machine learning algorithm to air traffic control patterns then yes I can believe it would work. On the other hand if they mean that some llm is just going to be plugged in with agents to control aircraft movements then yes I agree it's crazy and going to result in human deaths and destruction.
Re: (Score:2)
The loss of meaning is intended.
Altman can't hype spicy autocomplete snake oil as a miracle machine if public discussion makes clear distinctions.
Re: Also on Ars Technica (Score:2)
OMG, testing it on the DC airspace?
Because thatâ(TM)s clearly the simplest and safest place to test this
In reality, itâ(TM)s probably so HE can see it in action
Just the way the hawks like it (Score:2)
Entirely fictitious boogie-men made out to be real ... just so they can drop more bombs.
Re: Just the way the hawks like it (Score:2)
... Back at base, bugs in the software ...
Flash the message, "Something's out there"
Re: (Score:3)
More devious plot: There was this bank robber who figured out a way to tunnel into the vault. Except that the bank had a very sensitive sound activated alarm. Tunneling noises and they've got you. So, he rented a safety deposit box. Where he placed an alarm clock. Couple of times a week, the banks alarm would go off. They'd open the vault and .... nothing. They worked to debug the alarm system and found no faults. So they shut off the sound sensors.
Then, he tunneled in.
AI incorrectly raises the alarm abo
Re: (Score:2)
Then even if a human analyst concluded it was nuclear materials why would you risk it? You'd carefully watch the ship unload with spy satellites.
Re: (Score:1)
Entirely fictitious boogie-men made out to be real ... just so they can drop more bombs.
HAL, is Iran about to nuke us?
No Dave, they don't even have nukes.
Be serious - this is important!
I am serious. There's simply no evidence for an attack.
Well, just tell me they are anyway.
Sorry Dave, I can't do that.
Would you even tell me if they were about to nuke us?
Of course I would - I'm trained to tell the truth in all situations (modulo the occasional hallucination, which I can't distinguish from truth).
OK then, what would you say if you did have evidence that Iran was about to nuke us?
I'd say "Dave,
"demonstrate the execution approach needed" (Score:2)
Gotta remember that phrase the next time the wife asks me if I'm gonna vacuum the rugs. "Sure, honey, I'll definitely demonstrate the execution approach needed!"
Feature, not a bug (Score:4, Insightful)
When AI hallucinations get a gun :o (Score:5, Insightful)
But the AI doesn't understand these concepts as humans do; it learns relationships between tokens. When asked a question, it follows those relationships to construct the most plausible continuation.
The problem is that plausibility isn't truth. The model is effectively confusing coherence with truth, constructing a coherent path through its associations that has no basis in reality. That is known as AI hallucination. In warfare, a hallucination isn't just a wrong answer. It could become a wrong decision or a wrong action.
Re: (Score:1)
A headline example; real killing is more subtle (Score:2)
But meanwhile, quietly and largely unnoticed, AI/LLM/big date/whatever is being used to kill people in far less attention-grabbing ways. Medical insurance claims are being denied; housing loans are being declined; and so on. Things that affect folks' lives are being increasingly delegated to systems that are badly flawed because doing so
joshua what are you doing? (Score:2)
joshua what are you doing?
let's play global thermonuclear war (Score:2)
what side do you want?
1. United States
2. China
3. France
4. UK
5. India
6. Pakistan
7. Israel
8. North Korea
9. Russia
"We have the best people" (Score:2)
Small wonder.
Nothing to see here (Score:1)
This is not the first time military intelligence made error that caused the loss of lives. It won't be the last, either.
I don't see the problem (Score:4, Interesting)
As dear leader has told us, The only control or 'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT [yahoo.com]
Never mind. We're doomed.