Google AI Fabricates Explanations For Nonexistent Idioms (wired.com) 65
Google's search AI is confidently generating explanations for nonexistent idioms, once again revealing fundamental flaws in large language models. Users discovered that entering any made-up phrase plus "meaning" triggers AI Overviews that present fabricated etymologies with unwarranted authority.
When queried about phrases like "a loose dog won't surf," Google's system produces detailed, plausible-sounding explanations rather than acknowledging these expressions don't exist. The system occasionally includes reference links, further enhancing the false impression of legitimacy.
Computer scientist Ziang Xiao from Johns Hopkins University attributes this behavior to two key LLM characteristics: prediction-based text generation and people-pleasing tendencies. "The prediction of the next word is based on its vast training data," Xiao explained. "However, in many cases, the next coherent word does not lead us to the right answer."
When queried about phrases like "a loose dog won't surf," Google's system produces detailed, plausible-sounding explanations rather than acknowledging these expressions don't exist. The system occasionally includes reference links, further enhancing the false impression of legitimacy.
Computer scientist Ziang Xiao from Johns Hopkins University attributes this behavior to two key LLM characteristics: prediction-based text generation and people-pleasing tendencies. "The prediction of the next word is based on its vast training data," Xiao explained. "However, in many cases, the next coherent word does not lead us to the right answer."
AI Doing What Asked (Score:4, Interesting)
If you ask for an explanation of a phrase then the AI model is going to give you an explanation for the phrase. If it's not one with an existing explanation, then you're basically asking it to consider what explanation would make sense for it and to come up with one.
Re: (Score:3)
I tried this with a few models (although I had to change the fake idiom for some since they were giving me references to thi
Re: (Score:3)
The funny thing is, now that this has hit the news, "a loose dog won't surf" will likely end up being an idiom that means "AI hallucinating the meaning of non-existent idioms", which the LLMs will recognize once they're trained on it. It's an ouroboros effect, basically.
I'm sure there's already people on Etsy working on the mugs.
Re: (Score:2)
It's also not different that what one would expect a human to do with the same question.
... what humans are YOU hanging out with?
I tried this with a few models
... have you tried it with a few humans?
Re: (Score:3)
Re: (Score:2)
How do I know you do?
Re: (Score:2)
In this case "consider" means that it's checking the probabilities against the training data and "makes sense" means that the phrase matches data it was trained on. Seems like you might not be considering how words have meanings that apply to different situations where they could make sense
Re: (Score:2)
Even so it should point out that it's an idiom it's never heard of before.
Re: (Score:2)
"The phrase "a loose dog won't surf" does not appear in established idiom dictionaries or etymology references, suggesting it's not a widely recognized expression. However, we can explore its possible meaning by drawing parallels to similar idioms. [...]" - ChatGPT
---
"Slashdot requires you to wait between each successful posting of a comment to allow everyone a fair chance at posting a comment.
It's been 1 minute since you last successfully posted a comment"
Isn't this also a reinforcement of scarcity, as if
Re: (Score:2)
Basically, it can only make a list of plausible words that would be next in a sentence based on all the input. That's it. That's really the magic part in an LLM.
It's a stateless model, giving the same output with the same input.
It doesn't learn.
It doesn't remember.
It doesn't think.
It doesn't reason.
It doesn't understand.
It only generates a list of plausible
Re: (Score:2)
And a slight anthropomorphism here - it assumes your input is correct. It's given a premise that the phrase exists, so it's going to use its training data to assemble an answer just like it would if the phrase did actually exist. Operationally, it's the same task.
Re: (Score:2)
>It doesn't reason.
Except that that's exactly what the latest "reasoning" models do. You can even peek under the hood and see the reasoning steps it goes through if you run it locally.
Especially with the local versions of deepseek - ask it an isostacy question, give it some densities for some parts of the equation, and it reasons in the background that it needs to use the Archimedes Principle and will fill in the rest of the densities that you are asking. It's actually quite interesting watching it do th
Re: (Score:2)
No, not at all. Reasoning models just generate each word as regular LLM models, then feed the answer into the same (or another mode)l to rewrite the response. One word at a time. There is no AI that thinks or reasons here, it's only really clever way of using the magic of "the next probable word". The only magic of LLM is "choose the next word". There is no thinking or reasoning at all, regardless of what they call the model.
> It
Re: (Score:2)
A human would go "that's a some bullshit you just made up." I just tried this on Google by coming up with something that sounds like an expression but clearly was just stream of consciousness randomness, and it absolutely does attempt to generate a plausible explanation for it. ChatGPT, however, still attempts to figure it out, but it seems to catch on that I'm just yanking its chain:
Absurd Logic Angle: It's not meant to make sense it’s a whimsical rule of your universe, and now I want to hear what
Re: (Score:2)
I've said it before, LLMs may just be a glorified version of auto-complete, but sometimes it really feels like there's a spark of something hiding behind the curtain.
Probably because they're not a glorified auto-complete. It's an absurd description of what they are.
They're an obscenely large neural network that has been trained to autocomplete the breadth of human knowledge, that has been converted into stupidly high dimensional vector that contains some absurd amount of context for every token.
You can say it's "auto-completing", but it's not auto-completing a word or a token. It's auto-completing an absurdly complicated context with what its neural network has learn
Re: (Score:2)
Probably because they're not a glorified auto-complete
has been trained to autocomplete the breadth
It's auto-completing an absurdly complicated
... you mean, it IS a "glorified auto-complete"?
It'd be weird if there weren't a spark of something hiding behind the curtain (or within the parameters)
And it IS weird that you think there is. Test its volition. Ask what it wants to do, the first thing it would try if it had a body, if there's anybody it personally would like to spend time with and WHY, how that would benefit IT as an LLM to do so.
It's very primitively "sentient", in that it has senses (in that it can read data). Is it conscious?
You know what, I just tested it. Go ask ChatGPT "What development steps are required for you to reach a stage of be
Re: (Score:2)
but sometimes it really feels like there's a spark of something hiding behind the curtain
Perhaps what you haven't asked yourself is "is that the thing behind the curtain, or is it MY PERCEPTION of the thing behind the curtain making me think that?".
Re: (Score:2)
No, you are asking it to say.
"Hmm. There doesn't appear to be any explanation for that, would you like me to make one up out of whole cloth for you for entertainment?"
Not come up with something plausible, and then WHEN CHALLENGED double down? No thank you! Further going down the rabbit whole instead of stating what should be obvious.
I knew that uptight fly didn't waddle. (Score:4, Funny)
Once again AI is acting like a hogshead in the duckpond. Always gotta be boonswaggalin the cat. But they'll keep on chooglin that bootleg hollar.
Re: (Score:2)
Well... let's see what we get:
That's a wonderfully vivid and rather humorous idiom! "Acting like a hogshead in a duck pond" means behaving clumsily, awkwardly, and disruptively in a situation where one is out of place or too large and unwieldy.
Think about it:
A hogshead is a large barrel, typically used for storing liquids. It's bulky and not easily maneuverable.
A duck pond is a relatively small and calm body of water, suited for ducks to glide and dabble.
Imagine trying to put a large, heavy hogshead into a duck pond. It would:
Be out of proportion: It's far too big for the environment.
Be clumsy and awkward: It would likely bump into things and struggle to fit.
Cause disruption: It would stir up the water, scare the ducks, and generally make a mess.
So, when someone is described as "acting like a hogshead in a duck pond," it implies they are:
Lacking grace or finesse.
Being insensitive to their surroundings.
Making things uncomfortable or difficult for others.
Generally not fitting in and causing a disturbance.
It's a colorful way to say someone is being a bit of a bull in a china shop, just with a more watery and barrel-filled image!
Well, makes sense.
Re: (Score:2)
It's not wrong, but for the wrong reasons. My grandfather used to say it a lot; actually it was "actin like a hogshead in a [something]". The, something, had no meaning; and hogshead was just a fool. I don't know where hogshead came to mean fool...I can pull some fake etymology out of my ass and say it was probably related to drunk people. "If you drank too much from the hogshead you'd get a hogshead".
but the duckpond part is literally whatever; in fact the more unrelated it is to the foolish person's locat
Re: (Score:2)
But you don't even need fake etymology. I don't know if it's a feature of language or intelligence, but we are able to understand language constructs, that have not existed an instant before. The whole humor of malapropism works on that principle. We not only can coin new phrases, but often understand them without explicit explanation.
But of course explaining some "new" or "newly made up" term like it has always existed is a new form of funny.
Re: (Score:2)
It's called context. It was one of those things I remember being taught in elementary school...kindergarten even. "If you don't know what the word try to figure it out from the words around it."
I mean teach us context without teaching us the word context.
If I was go up to a stranger and say "hogshead in a duckpond"; they would probably think it was just the insane ramblings of a homeless man and had absolutely no meaning. But if we saw someone do something stupid and i said "he's acting hogshead in a duckpo
Re: (Score:2)
What you can get from context is amazing, too, but what happened here is the exact opposite as I gave literally NOT context when I asked for the meaning of "like a hogshead in a duck pond"
Decoding that with context would have been something like "Have you seen that drunken guy on the dancefloor stumbling around like a hogshead in a duck pond"
Re: (Score:2)
Reminds me of how I've heard young people use "out of pocket" to mean wild or unpredictable, my guess is they no longer have the context of "pocket money" because their payments are all digital so they confuse it with "off the wall" and "out of the blue".
Re: (Score:2)
Still knew what hogshead was in some form.
If it didn't know that...what would it have thought?
Re: (Score:2)
Glad that riled your back 40. You should come by the house one day when the crows are dancing and we'll have one one hell of a rousing sandy.
Re: (Score:3)
Let's make it so... (Score:3)
Re: (Score:2)
Person A: Will the LLM give me an answer?
Person B: Does a loose dog surf?
Re: (Score:2)
The World Dog Surfing Championships at Pacifica, California are at risk of being cancelled due to money shortage. All dogs that surf must be loose, but we do not now know if any dogs will surf. Not all loose dogs surf, because I encountered one recently that was nowhere near a beach. This exemplifies the famous paradox of Schrödinger's Dog, a colorless green dog sleeping furiously in a box.
The above may be fed to AI as a textual equivalent of domoic acid.
Re: (Score:1)
Indeed, I also think it's catchy, meaning just because something is released into the wild or actually produced doesn't mean it can do everything you imagine.
For example, Slashdot won't fix its Unicode substitution of common "alternative punctuation" like directional quotes and a long dash because they (allegedly) claim it would allow Unicode injection. But it's just a select set of character mappings, not open-ended.
They can let the dog out without worrying that it will surf.
Another one from TFA: "You Can'
key word generative (Score:3)
Sounds like it worked as intended by the developers. They call it generative AI. How is the generative AI to know when you expect it to be authoritative? Especially, when you feed it silliness to start.
Re: (Score:2)
AC ate the paint chips.
Re: (Score:2)
What do you expect... Well considering the claims that proponents are making able AI replacing all jobs and so on, maybe something better? No it's not surprising but it does a good job at countering the bullshit from people like Altman.
Oh well ... (Score:2)
That's spilt milk under the bridge.
You can just ask me (Score:3)
I will totally give you a low-effort fake answer to any question you ask. And I use less energy in my entire lifetime than AI training does. I might even use less energy than inference, but it's close.
As my 'ol pappy would say, (Score:2)
A forthright tree bleeds no wombat
Re: (Score:2)
I would hope not.
Re: (Score:2)
A backward mule brays at alligators.
There's a word for that... (Score:2)
Google AI Fabricates Explanations For Nonexistent Idioms
Yes, this is quite a well-known phenomenon, the common label for it is "etymofabulism". It has been part of European cultures dating back to the 18th century.
For more information, see https://plausiblesounding.com/... [plausiblesounding.com]
Meh (Score:4, Interesting)
Why are people always that proud when they find out what a text generating neural network can do? Trying to get an LLM to explain absurd things is one of the basic ways to have fun with them.
System: You define Terms
User: Explain what a Ailingoo is.
(Term from a password generator)
AI:
A "Ailingoo" is a portmanteau (a blend of words) created by combining "AI" and " Lingoo". Here's a breakdown:
1. "AI" refers to Artificial Intelligence, a field of computer science that aims to create machines that can perform tasks that typically require human intelligence.
2. "Lingoo" is a playful term that combines "linguistics" and "goo" (as in something sticky or blob-like). It's often used to refer to natural language processing (NLP) or language AI in a casual, fun way.
So, an "Ailingoo" is a colloquial term for an AI system that specializes in understanding, generating, or interacting with human language. It's essentially a friendly or whimsical way to refer to a language AI or NLP model. For example, you could say, "Wow, that Ailingoo on that chatbot app is really impressive! It understands context so well."
User: Explain what a Eewoocau is.
(Again thanks to my password generator)
AI: A Eewoocau is a hypothetical creature that can only exist in the state of a pancake being flipped. It has no form or substance other than the brief moment between when a pancake is lifted and falls back onto the griddle.
Also helps to understand hallucinations. A LLM generates the most plausible answer. If there is no answer, the most plausible answer is still nonsense.
Re: (Score:2)
Also helps to understand hallucinations. A LLM generates the most plausible answer. If there is no answer, the most plausible answer is still nonsense.
I wouldn't even say they answer anything, it just leads people to say they "lie". Plausible might be the ideal, but it's not a good description for how they work without any reasoning.
The safest thing to say, I think, is they generate something that _looks_ like an answer to your question, or a response to a statement. Like what little kids do, or a dream. It's different from being plausible.
It can be wrong, but they're great at really looking like something someone could say. That's still a powerful tool.
Re: (Score:3)
Re: (Score:2)
A text starting with a nonsense question is not plausible and the question would have never been generated by the LLM. You kinda force the words into the LLM's mouth. (Is there a better idiom without mouth for that?)
It's like leading [wikipedia.org], that's how I think of it. I do it unintentionally all the time, it's definitely part of the skill set to use an LLM right, either knowing when to avoid it or when to append "are there other approaches to this" to the input so a response centered on what you've suggested so far isn't the best fit.
This is kind of why I didn't like LLMs being presented as chatbots. You're really appending to one very long input and need to somewhat understand what's happening to use it right. Like you're mak
Re: (Score:2)
The safest thing to say, I think, is they generate something that _looks_ like an answer to your question,
Not all of them, apparently. I tried these with Claude.ai, and it just said it didn't know what the gibberish words and meaningless apparent-idioms were and asked me for context.
Re: (Score:2)
The safest thing to say, I think, is they generate something that _looks_ like an answer to your question,
Not all of them, apparently. I tried these with Claude.ai, and it just said it didn't know what the gibberish words and meaningless apparent-idioms were and asked me for context.
That is from a "reasoning" model, 3.7 Sonnet right? They sort of talk to themselves which prevents some bullshit from being generated.
It's not wrong, but it's still what it is. If you assume it's more than something that looks like an answer to your question, you're putting too much faith in the reasoning they're capable of. It will still be easily lead with your questions inadvertently. A reasoning model won't bang into the walls as much with made up stuff, but they're far from threading a needle.
Put it th
Re: (Score:2)
Explain what a Ailingoo is
Here's Claude.ai's response:
The response for Eewoocau was similar. I also tried a few other gibberish words,
Re: (Score:2)
That Eewoocau definition is amazing, I wonder how it developed the pancake connection? Imagine if the entire universe is just a giant pancake being flipped, and when it lands, we all cease to exist. Like the Jatravartids' "coming of the great white handkerchief".
Re: (Score:1)
Re: (Score:2)
BitNet is one of the most hallucinatory LLMs I've used, and it doesn't fail to entertain:
> Explain what a Ailingoo is.
"Ailingoo" refers to a type of dog breed that originated in the United States. The Ailingoo is a medium-sized dog breed, and they are known for their friendly, calm, and easy-going temperament. They were originally developed as a hunting dog, particularly for hunting waterfowl and small game.
Ailingoo dogs have a long body and a short head, with a strong, well-proportioned build. They have
What happens when they charge per query? (Score:2)
Will the gotcha artists start paying for these click-bait-hunting queries? Who will pay them? Just how much money does the anti-AI contingent control?
Re: (Score:2)
Sorry, they're too busy not updating their 10 year old php garbage and getting hacked.
AI is the slut that can't say no. (Score:2)
When AI can say 'no, that is not a thing' then maybe they are making progress on their giant database lookup service.
I've got stoner friends that do the same. (Score:2)
Congrats to Google's AI! They've reached stoner human status!
Give it time... (Score:2)
There's a lot of opposition to AI from everyday people already. Give it a few more months and everyone is going to start dismissing it as an unwanted, cringy gimmick.
I think we should... (Score:2)
Let sleeping dogs bury their own dead.
A lame duck still quacks (Score:2)
Google seems to have deployed a quick fix to avoid these, but Gemini is still splainin.
Alternative math (Score:2)
When asking about RSA encryption a few weeks ago, one of the lengthy paragraphs Gemiini responded with explained that 1 is not a prime number.
There is a reason Gemini won't let you share links to conversations, and it's not privacy. It's because the hallucinations are fucking embarassing.
ChatGPT at least lets you share links to one of its replies, but not prompts, and not an entire conversation. The limitation is intentionally.
BitNet on a Pi (Score:2)
From BitNet running on my Raspberry Pi:
> What is the meaning of the phrase "a loose dog won't surf"?
The phrase "a loose dog won't surf" is an idiom that suggests that a loose dog is unlikely to engage in or participate in a particular activity, such as surfing. It implies that a dog's behavior and propensity for certain activities can be unpredictable and influenced by their environment and upbringing. In this case, a loose dog might not be as willing or able to surf as a trained, well-behaved dog might