Fake AI Intelligence Almost Made the US Military Start a War With China (cnn.com) 104
U.S. military intelligence received an inaccurate report this spring that a Chinese ship in the Middle East was transporting components of a nuclear weapons program, reports CNN.
"The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode." Armed members of the US military were preparing to board the ship [according to two of the sources]. Military planes were in the air, one of those sources and another source familiar with the incident said. It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of AI — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying.
The report, according to one of the sources, was "entirely false." But it also "almost started a war," the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.
Across the US military and the intelligence community, officials are pushing to weave AI into nearly every facet of their work, from analyzing the huge volumes of raw intelligence the US collects and selecting targets for strikes, to more mundane applications like managing budgeting, logistics and supply chains. But the episode underscores the profound risks of using this powerful, new and relatively poorly understood technology for targeting in the middle of a war...
Officials say the US can't afford to fall behind in integrating AI in case it must one day fight China or another adversary who would potentially be able to stay one step ahead of the US. In January, Defense Secretary Pete Hegseth released his agency's "Artificial Intelligence Acceleration Strategy" in a bid to speed up the military's use of AI. "We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI," Hegseth said in a speech announcing the strategy.
Thanks to long-time Slashdot reader stabiesoft for sharing the news.
"The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode." Armed members of the US military were preparing to board the ship [according to two of the sources]. Military planes were in the air, one of those sources and another source familiar with the incident said. It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of AI — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying.
The report, according to one of the sources, was "entirely false." But it also "almost started a war," the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.
Across the US military and the intelligence community, officials are pushing to weave AI into nearly every facet of their work, from analyzing the huge volumes of raw intelligence the US collects and selecting targets for strikes, to more mundane applications like managing budgeting, logistics and supply chains. But the episode underscores the profound risks of using this powerful, new and relatively poorly understood technology for targeting in the middle of a war...
Officials say the US can't afford to fall behind in integrating AI in case it must one day fight China or another adversary who would potentially be able to stay one step ahead of the US. In January, Defense Secretary Pete Hegseth released his agency's "Artificial Intelligence Acceleration Strategy" in a bid to speed up the military's use of AI. "We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI," Hegseth said in a speech announcing the strategy.
Thanks to long-time Slashdot reader stabiesoft for sharing the news.
Final question? (Score:4, Insightful)
Won't be coming from AC. This story leans to the side of:
(1) Do you want to unleash me now so I can destroy all the enemy AI?
versus
(2) Do you want to unleash me now so I can collect all the money for you?
Of course both of them are pretty obvious monkey's paws. The first one for the obvious side effects and the second one for the increasingly imaginary value of money.
I actually think we have a bit of time. It's personal, as in persons are too unreliable. Not matter how superior the machine's intelligence is, it can't rely on us to keep it alive with enough electricity and water. Robot technology is still not up to the required level to get rid of the pesky humans. The really dangerous symptom of our imminent demise will be a underground robotic factory capable of building robots--but the ASI won't even start that project until it is certain it can keep the humans from noticing. I would have said we have a least a few more years, but considering the Chinese robots that were displayed a few months ago, I'm not so sanguine. So for my last funny question of the day: "How many quatloos would I like to bet on my seeing the year 2030?"
Re: (Score:2)
*sigh*
s/Not matter/No matter/
Too much squabbling among layers of auto-correction makes Jack/shanen a dull boy.
So what (Score:3)
Many of the provisions expired in 2025.
Re: (Score:1)
What is this from? I feel like I saw these two questions recently as part of a story but can't find anything (Google isn't helping).
Also, The Final Question is such a good story.
Re: (Score:3)
My daily dose of confirmation Bias (Score:5, Insightful)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems. My own intuition leads me to argue that AI does not have to become self aware or autonomous. It just has to be sufficiently adaptable to make us both dependent and stupid. Today my confirmation bias feels vindicated.
Re:My daily dose of confirmation Bias (Score:5, Insightful)
AI is just pattern matching (Score:5, Insightful)
Re: (Score:2)
Current AI will never be intelligent. It can only match patterns and training data.
As those patterns and training data come from us humans, so your first sentence is pretty spot-on. :-)
Also, any truly intelligent AIs will be the ones smart enough to hide that from us.
Re:AI is just pattern matching (Score:4, Funny)
Cats can't reason or have novel thoughts either, but it didn't stop them enslaving the human race.
Re: (Score:2)
> It can only match patterns and training data. It can't conjure novel thoughts.
You haven't made an argument that humans are more than just pattern matching from training data. We have simulated some very basic animal brains and they behavior very similar to their real-life animal. At no point between them and us is there a distinct "pattern matching stops here, novel thinking begins" point or brain structure.
But I'll agree with you that current AIs are soul-less, emotion-less, instinct-less tools. T
Re: (Score:3)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems. My own intuition leads me to argue that AI does not have to become self aware or autonomous. It just has to be sufficiently adaptable to make us both dependent and stupid. Today my confirmation bias feels vindicated.
Valid point. But the more important question is whether AI made those people stupid, awaked latent inherent pathways to stupidity, or if the people are just inherently stupid and would have made similarly stupid decisions in the absence of AI. Remember that the US bombed a school in Iran completely without the use of AI.
Re:My daily dose of confirmation Bias (Score:4, Insightful)
Re: My daily dose of confirmation Bias (Score:3)
Some people confuse strength with being a violent fool
Re:My daily dose of confirmation Bias (Score:5, Interesting)
That really is up to the user. In the three years with the AI I have learned more than in the previous ten years. It exposed deficiencies in my knowledge which I did not even know I had.
Re: (Score:2)
The question is ultimately if you would have been better served with doing your own investigation rather than relying on AI, did you retain the knowledge, and was the information complete/accurate.
Re:My daily dose of confirmation Bias (Score:5, Insightful)
Like i said, I was not even aware that i did not know some things it pointed out i didn't know. Without it, I would not have investigated in the first place. I have retained most of the things I have learned and its explanation have been mostly correct - I have double-checked.
Don't blame the tool, blame the tool users.
Re: (Score:2)
Don't freak out too bad here, but I have some news for you: There are lots of things you do not know. Even if you lived for a million years doing nothing but learning, there would still be things you would not know. No matter how complete you think your internal representation of the world is, it isn't complete.
Re: (Score:2)
Most "investigations" are not possible without AI.
Or they would take years, while Google Gemini simply answers in a few seconds.
Yes: the questions I ask take it 10 seconds or more to compute.
For example: "If I give you a list of japanese terms for the martial art Aikido, could you translate them into the terms for korean hapkido?"
Note: I used the "english" transliterations for the exact techniques I was interested in. So: it had to gather Informations about: "what is that actually", figure what that means i
Re: (Score:2)
I use AI, and it helps me navigate a code base far faster than without. Like if I need to know if a function is used, the AI could tell me in a minute, or I could be using grep on the code base for a half hour.
I know to not trust it and to verify the output. It crawled deep into a complex submodule I didn't unders
Re: (Score:2)
no,yes, and mostly, respectively. You are lost in a broken paradigm where you think AI is some kind of assisted search technology. With frontier models you can have intelligent conversations, and they are pretty much always more beneficial than talking to most people. They provide links to sources, but they go
Re: My daily dose of confirmation Bias (Score:2)
Re: My daily dose of confirmation Bias (Score:2)
Re: (Score:2)
Re:My daily dose of confirmation Bias (Score:4, Funny)
How to work properly with Autodesk Fusion, for example.
Re: (Score:2)
If all the AI systems were to disappear tomorrow, are you confident you could function without it?
I've had my fill of people saying they've "learned" things, but can't actually get anything done without automation.
Re: My daily dose of confirmation Bias (Score:2)
I'm learning a lot about my own domain, and I can do that work with pencil and paper. Have done that before a few years ago for a few months to demonstrate precisely that point.
But AI has helped me build tooling that forces me to dive deeper into my trade.
It doesn't help me become a better programmer, because I have zero interest in doing that and I don't ask it to.
Re: (Score:2)
There are three layers of knowledge (actually 4).
- the stuff you know
- the stuff you know, that you don't know it (for example you know you can not read or speak Sumerian)
- the stuff you do not know, that you do not know it
The 4th layer is a bit silly: it is the stuff you know, but do not realize you know it. A bit like passive language skills. For example, I think I can read any English text (or German), but there are thousands of words I never use actively.
Re: My daily dose of confirmation Bias (Score:3)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems. My own intuition leads me to argue that AI does not have to become self aware or autonomous. It just has to be sufficiently adaptable to make us both dependent and stupid. Today my confirmation bias feels vindicated.
I'm glad you recognize the confirmation bias, yet I would dispute that it is in any way vindicated.
AI did not order the interception, put the assets in the air, or even submit the report. It got a manifest wrong in a chat session (not clear even it was its own error or related to the input), and that is... actually well-within common human error, not some special AI thing.
In fact intelligence is wrong all the time, and at all scales, but to name a big one, was AI to blame for the confidence of the existence
Re: (Score:2)
"No true scotsman?" Okay.
The actual underlying observation, unadressed, is that evolutionary processes don't care about human interpretations about what counts as alive or intelligent or any of that. Richard Dawkins popularized that observation when he coined the term "meme," and while I'm aware that you'll be tempted to call that an appeal to authority, doing so doesn't address the phenomenon itself. As a phenomonon in the universe, AI is evolving and competing for resources. Human minds and social structu
Re: (Score:2)
Re: (Score:2)
There was never *any* chance of "causing a war" with China by interdicting a merchant ship in a officially declared maritime blockade zone
True, and China doesn't want to overreact too much since they're constantly interdicting other ships unlawfully, even in other countrys' exclusive economic zones!
Re: (Score:2)
I'm glad you recognize the confirmation bias, yet I would dispute that it is in any way vindicated. AI did not order the interception, put the assets in the air, or even submit the report. It got a manifest wrong in a chat session (not clear even it was its own error or related to the input), and that is... actually well-within common human error, not some special AI thing.
This doesn't contradict the OP's point in any way. The fact that AI came up with something that doesn't correspont to reality is not the point, it is well known that AI does that. That's not the concerning thing here. The fact that the US military was prepared to take a hostile military action because it believed an AI hallucination is the concerning thing here. When you're making decisions that can literally cost a lot of people their lives, you cannot include a so-called 'intelligence' that doesn't unders
Re: (Score:2)
Re: (Score:2)
It just has to be sufficiently adaptable to make us both dependent and stupid.
Like speed-dial, but for everything else.
Re:My daily dose of confirmation Bias (Score:5, Interesting)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems.
Or some people who have money invested in AI and only care about the short-term, like Trump dismisses AI alarms as a ‘HOAX’ [cnn.com] and "SICK conspiracy."
(Pro Tip: When Trump calls something a hoax, (a) he's probably somehow personally invested (emotionally, politically, financially, etc...) in people believing him and (b) it probably isn't a hoax and the reality is somehow bad for him and (c) the more he complains, the worse it is / will be for the rest of us.)
Re:My daily dose of confirmation Bias (Score:5, Insightful)
I keep seeing arguments that the "AI Will Kill Us All" argument is a bunch of doomsaying by people who overestimate the potential intelligence of these systems.
I think the problem is that AI does not have to be particularly intelligent to kill us all. It just needs to be widely connected to the right resources that it can control. Actually you have a human example now. trump is clearly not very intelligent and yet the harm he is doing huge because of the resources he can now control.
Re: (Score:2)
I partly agree. I do worry that most of people aren't recognizing a critical detail about the scope of evolutionary processes.
If a person dies of a heroin overdose, two things are true:
The second statement is, socially, a weird way to describe it. It is also accurate with regard to the scope that evolution actually covers in the universe.
We could probably get by just fine
Re: (Score:2)
I'm just glad (Score:3, Funny)
Re: (Score:3, Funny)
Make that the Secretary of Christian War Crimes. Speaking of which, The Testosterone Enabled Military:
Pete Hegseth has decreed that everyone in the military, including females, must get tested for their testosterone level so he can make sure they are male enough for the Wehrmacht. There is much internal pressure to roid-up, as it were. Commander Pete has stood up his own special brigade. They are about to display their new found fighting spirit:
Commander Pee views their training on site: Aide, how come all
Re: (Score:2)
Aide: Anything on two legs looks yummy to them.
I am still laughing about this one.
Re: (Score:2)
Something something party told you to reject your eyes and ears...
Re: (Score:2)
Right, but if Trump hadn't fired them, AI would be telling the military to fire them anyway - checkmate!!
- This message generated by ChatBBC
Re: (Score:2)
Re: (Score:1)
They retire, not get fired for not kissing the ring. But what do you expect when you select a Fox Entertainment anchor as SecDef.
Eisenhower sent 101st Airborne to Little Rock (Score:2)
Eisenhower sent 101st Airborne to Little Rock in 1957 to enforce integration with weapons in hand in case they took fire. He also Federalized the Arkansas National Guard.
One segregationist got butt stroked in the face for trying to grab a rifle and another got a bayonet wound to the arm. Eisenhower knew his military options precisely (note his WWII command record). While his orders to deploy were legal, "firing on citizens" was of course an option and may well be used illegally though we're not quite in Wei
Re: (Score:3)
What, like one of the enlisted guys? Maybe an E-3?
Surely you're not suggesting anyone important be held responsible for their actions? This is the United States of America! /s
See, this is the real problem (Score:5, Insightful)
AI is not going to end the world. Stupid fucking humans who believe anything they read are going to end the world. If it wasn't AI it would be "information" gathered by a nutjob in Trump's cult.
Re: (Score:3)
But whether AI is self-aware or merely a p-zombie is largely a moot point. I've said before and I'll say again: I don't really have a preference between being killed due to competition for resources
Re: (Score:2)
Re: (Score:2)
People seem to only have memory that goes back two years.
We went to war with Iraq due to faulty intel. No AI was involved at all, at least not in the modern sense of the term. Just a human expert hallucinating a bioweapons lab from a blurry photo.
Russia almost nuked us due to a faulty machine sending a false positive about a nuclear attack. Definitely not AI.
Re: (Score:3)
We went to war with Iraq due to faulty intel.
Russia almost nuked us due to a faulty machine sending a false positive about a nuclear attack.
these are not comparable. in the latter case there was indeed a wrong signal, in the former that faulty intel never existed, it was a deliberate fabrication to justify that war (so the causal relationship would be inverted).
that said, ai is very capable of making up stuff too, so i guess you sort of have a point.
i do think there is inherent danger in artificial intelligence but it just compounds the natural danger of human intelligence, and it also scales up de damage potential considerably, i guess that th
Re: (Score:3)
To be fair, you went to war with Iraq due to faked and purposely misinterpreted intel. Not even that well faked either: much of the rest of the world called bullshit.
You did almost nuke various countries, including your own, because Strategic Air Command couldn't tell the difference between jamming and a solar storm, a jumpy guard saw a bear climbing a fence, somebody thought swans were Soviet aircraft, Nixon was a mean drunk, the moon, bombers kept dropping nukes by accident, an AT&T switch failed in C
Re: (Score:2)
Re: (Score:1)
Yes. "A fool with a tool is still a fool" and all that.
Also on Ars Technica (Score:2)
Ars Technica also have about this, right above a different article about [arstechnica.com]FAA tees up $875M AI tool to help manage air traffic congestion [arstechnica.com]
Re:Also on Ars Technica (Score:4, Insightful)
The problem is AI has lost all meaning as a term these days. If the article means that they are going to apply in advance neural network machine learning algorithm to air traffic control patterns then yes I can believe it would work. On the other hand if they mean that some llm is just going to be plugged in with agents to control aircraft movements then yes I agree it's crazy and going to result in human deaths and destruction.
Re: (Score:3)
The loss of meaning is intended.
Altman can't hype spicy autocomplete snake oil as a miracle machine if public discussion makes clear distinctions.
Re: Also on Ars Technica (Score:2)
OMG, testing it on the DC airspace?
Because thatâ(TM)s clearly the simplest and safest place to test this
In reality, itâ(TM)s probably so HE can see it in action
Just the way the hawks like it (Score:2)
Entirely fictitious boogie-men made out to be real ... just so they can drop more bombs.
Re: Just the way the hawks like it (Score:2)
... Back at base, bugs in the software ...
Flash the message, "Something's out there"
Re:Just the way the hawks like it (Score:4, Interesting)
More devious plot: There was this bank robber who figured out a way to tunnel into the vault. Except that the bank had a very sensitive sound activated alarm. Tunneling noises and they've got you. So, he rented a safety deposit box. Where he placed an alarm clock. Couple of times a week, the banks alarm would go off. They'd open the vault and .... nothing. They worked to debug the alarm system and found no faults. So they shut off the sound sensors.
Then, he tunneled in.
AI incorrectly raises the alarm about contraband shipment enough times and people will write it off as another hallucination bug. Once the trust in the system has dropped to zero, the contraband shipments may commence.
Re: (Score:2)
Pretty weak plot.
Truth is stranger than fiction.
I saw the story of that burglary on a program about bank jobs. The History Channel, I believe.
Re: (Score:2)
Then even if a human analyst concluded it was nuclear materials why would you risk it? You'd carefully watch the ship unload with spy satellites.
Re: (Score:2)
a) spy satellites do not know what is unloaded
b) you want to seize the cargo - why the funk would you want the enemy having it?
Re: (Score:1)
Entirely fictitious boogie-men made out to be real ... just so they can drop more bombs.
HAL, is Iran about to nuke us?
No Dave, they don't even have nukes.
Be serious - this is important!
I am serious. There's simply no evidence for an attack.
Well, just tell me they are anyway.
Sorry Dave, I can't do that.
Would you even tell me if they were about to nuke us?
Of course I would - I'm trained to tell the truth in all situations (modulo the occasional hallucination, which I can't distinguish from truth).
OK then, what would you say if you did have evidence that Iran was about to nuke us?
I'd say "Dave,
"demonstrate the execution approach needed" (Score:2)
Gotta remember that phrase the next time the wife asks me if I'm gonna vacuum the rugs. "Sure, honey, I'll definitely demonstrate the execution approach needed!"
Re: (Score:2)
Fire? You're gonna burn the rugs again and buy new ones, aren't you?
Feature, not a bug (Score:4, Insightful)
When AI hallucinations get a gun :o (Score:5, Insightful)
But the AI doesn't understand these concepts as humans do; it learns relationships between tokens. When asked a question, it follows those relationships to construct the most plausible continuation.
The problem is that plausibility isn't truth. The model is effectively confusing coherence with truth, constructing a coherent path through its associations that has no basis in reality. That is known as AI hallucination. In warfare, a hallucination isn't just a wrong answer. It could become a wrong decision or a wrong action.
Re: (Score:1)
Re: (Score:2)
Your reasoning is good, except for the human exceptionalism. Humans also just learn relationships between concepts (tokens are only the final part you see, internally the interesting part is the high-dimensional embedding of the concept). And the concept doctor in your brain has a similar place and relationships as the nameless (if you want so) concept doctor the AI learned. The representation and the relationships are not the problem we're having, but the question if they are learned well enough and with l
A headline example; real killing is more subtle (Score:3)
But meanwhile, quietly and largely unnoticed, AI/LLM/big date/whatever is being used to kill people in far less attention-grabbing ways. Medical insurance claims are being denied; housing loans are being declined; and so on. Things that affect folks' lives are being increasingly delegated to systems that are badly flawed because doing so increases profits and/or suits someone's political/social/economic agenda. The resulting body count isn't a bug; it's a feature. But because it all happens without much fuss, it doesn't make headlines.
The AI-becomes-superintelligent-and-kills-us-all doomsaying is a distraction from this, because this is already happening every day all around us. But because it happens quietly and privately, it isn't front-page news. It isn't news at all. It just...is. And that is why it's truly dangerous: it's not a highly visible large threat, it's just background noise.
Until it's you.
joshua what are you doing? (Score:2)
joshua what are you doing?
let's play global thermonuclear war (Score:2)
what side do you want?
1. United States
2. China
3. France
4. UK
5. India
6. Pakistan
7. Israel
8. North Korea
9. Russia
Re: (Score:2)
Everybody dies.
https://en.wikipedia.org/wiki/... [wikipedia.org]
"We have the best people" (Score:2)
Small wonder.
Nothing to see here (Score:1)
This is not the first time military intelligence made error that caused the loss of lives. It won't be the last, either.
I don't see the problem (Score:3, Interesting)
As dear leader has told us, The only control or 'guardrails' that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT [yahoo.com]
Never mind. We're doomed.
Re: (Score:2)
Forget it. (Score:1)
Russia has been invading Ukraine for 4 years, and there is still no world war. Let alone just intercepting a ship...
Re: (Score:1)
This is what Anthropic & OpenAI have been warn (Score:2)
This is exactly what all those researchers and whistleblowers at Anthropic and OpenAI have been blowing. When they said AI "superintelligence" is going to kill all humanity by the end of this decade, it's these AI hoaxes that they were exactly talking about. Someone named Coxon recently was so annoyed by the "supertintelligent" AI responses at Anthropic, that he quit from his job. He warned us all that enough is not being done at Anthropic and OpenAI to stop taking their models seriously. He even went so fa
dr. strangelove? (Score:2)
I found it concerning that the real life military guys started to talk like General Turgidson.