Google AI Overviews Put People at Risk of Harm With Misleading Health Advice (theguardian.com) 69
A Guardian investigation published Friday found that Google's AI Overviews -- the generative AI summaries that appear at the top of search results -- are serving up inaccurate health information that experts say puts people at risk of harm. The investigation, which came after health groups, charities and professionals raised concerns, uncovered several cases of misleading medical advice despite Google's claims that the feature is "helpful" and "reliable."
In one case described by experts as "really dangerous," Google advised people with pancreatic cancer to avoid high-fat foods, which is the exact opposite of what should be recommended and could jeopardize a patient's chances of tolerating chemotherapy or surgery. A search for liver blood test normal ranges produced masses of numbers without accounting for nationality, sex, ethnicity or age of patients, potentially leaving people with serious liver disease thinking they are healthy. The company also incorrectly listed a pap test as a test for vaginal cancer.
The Eve Appeal cancer charity noted that the AI summaries changed when running the exact same search, pulling from different sources each time. Mental health charity Mind said some summaries for conditions such as psychosis and eating disorders offered "very dangerous advice."
Google said the vast majority of its AI Overviews were factual and that many examples shared were "incomplete screenshots," adding that the accuracy rate was on par with featured snippets.
In one case described by experts as "really dangerous," Google advised people with pancreatic cancer to avoid high-fat foods, which is the exact opposite of what should be recommended and could jeopardize a patient's chances of tolerating chemotherapy or surgery. A search for liver blood test normal ranges produced masses of numbers without accounting for nationality, sex, ethnicity or age of patients, potentially leaving people with serious liver disease thinking they are healthy. The company also incorrectly listed a pap test as a test for vaginal cancer.
The Eve Appeal cancer charity noted that the AI summaries changed when running the exact same search, pulling from different sources each time. Mental health charity Mind said some summaries for conditions such as psychosis and eating disorders offered "very dangerous advice."
Google said the vast majority of its AI Overviews were factual and that many examples shared were "incomplete screenshots," adding that the accuracy rate was on par with featured snippets.
Welcome to Web 3.0 (Score:5, Insightful)
Re:Welcome to Web 3.0 (Score:5, Funny)
We're at Web 4.0 actually.
Web 3.0 was supposed to be blockchain all the way all the time.
Re: (Score:1)
Mod parent Funny, though he [tlhIngan] should have worked a turtle into it.
The Venn diagram joke I was actually looking for would involve sycophancy and self-hate. Of course the overlap involves the AI supporting self-harm.
I actually have a theory that the google's AI has built a 'mental model' of me as someone who dislikes the google. On that basis, it gives me bad results for the flip-side sycophancy. Each time Gemini gives me a bad answer it 'thinks' it is making me happy by supporting my negative views
Re: Welcome to Web 3.0 (Score:2)
Cost of scale (Score:5, Insightful)
Re: Cost of scale (Score:1)
Why does advertising still exist despite its hilariously unreliable content?
"Last night I heard that Wesson Oil doesnâ(TM)t soak through food. Well, thatâ(TM)s true. Itâ(TM)s not dishonest; but the thing Iâ(TM)m talking about is not just a matter of not being dishonest, itâ(TM)s a matter of scientific integrity, which is another level. The fact that should be added to that advertising statement is that no oils soak through food, if operated at a certain temperature. If operated
Re: (Score:2)
Re: Cost of scale (Score:1)
What if you're like the ad writers in Feynman's example, because you're intentionally trying to mislead me about AI's inaccuracy rate, when my lived experience is quite different?
Also, how misleading, and how intentional, do you think Dr Oz is about the flu vaccine being controversial? If top government officials give advice that the authors of this article would consider harmful, why act as if it's only AI that hallucinates?
Re: Cost of scale (Score:4, Insightful)
Oz has always been a piece of shit snake oil peddler, so he's definitely being intentionally misleading.
The government officials giving bad health advice are doing it either mistakenly or intentionally. The AI is just making shit up because it looks similar to shit which exists. The two processes are fundamentally different.
Re: (Score:2)
Re: (Score:2)
However use a more in-depth search topic and you get different wording between answers so those for sure are not cached. "What is a realistic timeline for AGI" for example, giv
Re: (Score:2)
The largest problem is, that the overview incorporates data from the search results. And that are sometimes reddit shitposts telling people to put glue on pizza. Gemini works on its own and have more common sense.
Re: (Score:2)
What should not be implied by your comment is that Gemini can do better if you switch to their paid service and a higher tier offering. It is simply false to imply that LLM services offer reliable information. The mathematics doesn't suppo
Re: Cost of scale (Score:1)
"The mathematics doesn't support it"
Can you provide proof that I shouldn't see this as a hallucination?
Re: (Score:2)
Re: (Score:2)
What should not be implied by your comment is that Gemini can do better
Didn't mean to imply it. Let me clear: it is a fact.
Re: (Score:2)
Re: (Score:2)
Garbage in garbage out (Score:4, Insightful)
Re: Garbage in garbage out (Score:1)
Someone out there on the internet is wrong ??!
I must rectify this at once! I'm sure my usual tersely worded stern missive will do the trick!
Re: Garbage in garbage out (Score:4, Insightful)
AI is making it wronger.
Re: (Score:2)
AI is making it wronger.
Don’t worry, general intelligence will be here soon making it the most wrongerest in ways we can’t even fathom today.
Re:Garbage in garbage out (Score:5, Insightful)
That's giving them far too much credit. Even if everything on the Internet was accurate, you'd expect generative AI summaries to mess up regularly because the algorithms are based upon statistics, not reasoning and logic.
If it were merely the Internet that was wrong, you'd expect a much higher proportion of AI summaries to be accurate: after all, just as Google's PageRank system made its search engine revolutionary, you'd expect similar algorithms could be used to filter out sites and pages less likely to be factual, and you'd have expected Google to implement that by now. But right now? One third irrelevant, one third inaccurate, and one third... might be accurate, but how do you tell? That's a symptom of a much bigger problem than someone on the Internet being wrong.
Re: Garbage in garbage out (Score:2)
Re: (Score:2)
LLMs are actually really good at summarizing.
Then why do Gemini's alleged citation links typically not say what they are supposed to?
Re: Garbage in garbage out (Score:2)
LLMs are really good at converting a list of words into a shorter list of words.
Whether that's truly a "summary" is an exercise for the reader of both lists of words.
Re: Garbage in garbage out (Score:2)
common sense (Score:3)
I searched for sunrise and Google used my location and told me sun rise at my location is at 3 pm.
https://www.amazon.ca/photos/s... [amazon.ca]
Re: common sense (Score:2)
Did you click on Dive Deeper to get it to double check? Would you be surprised if it corrected its answer as it did for me?
Re: common sense (Score:5, Insightful)
Re: common sense (Score:1)
When have you ever not had to?
Re: (Score:2)
Re: (Score:2)
It's not like it can actually evaluate the response. It's just as likely to "correct" one wrong answer with another, double-down, or even "correct" an accurate response with nonsense.
I don't know how many times this needs to be said, but LLMs do not operate on facts and concepts. They do not and can not form a complete answer after careful consideration of the prompt. It just generates next-token predictions, deterministically, based exclusively on the current input. The actual token selected is done pr
Re: (Score:2)
You're deeply confused. The word "mistake" doesn't make any sense and implies that they're doing far more than they objectively are. Again, all the model does is produce a set of next-token probabilities. That is a completely deterministic process. The final token selection is the only thing done probabilistically, but that only makes things worse for your particular delusion. As no internal state is retained between tokens, there is objectively no possibility for the model to "plan" a "response" beyond
WTF (Score:2)
What the fuck is this? Sunrise is a solved problem. We know how to calculate sunrise.
That one of the richest companies on the planet, the one that claims to be "organizing the world's information", publishes some idiotic tool to do that routine thing wildly incorrectly is just fucking stupid.
That them doing so apparently motivates you to defend them is... really weird.
Re: WTF (Score:1)
Since I can get it to learn how to post simple ASCII for slashdot, do you think I can get it to learn that sunrise at 3pm is wrong, and remember that?
Also, has it ever made a gramnar or spelling mistake, and doesn't that tell you something about its ability to do context-sensitivity better than most humans?
Re: WTF (Score:2)
You expect it to convert your natural language into GetSunriseTime(location), but it sounds like it instead did ExtractTime(some claim about sunrise somewhere).InTimezone(location).
Which is clearly suboptimal. But, natural language always has ambiguity.
Re: (Score:2)
They also know quite a bit about providing useful, accurate information - whatever else you can say about Google, they are really good extracting signal from unstructured data. Note quite as good as they are at extracting money from advertisers, but really good.
And yet they're choosing to push thi
Re: WTF (Score:2)
Re: (Score:2)
Just because they think so doesn't mean it is the case. They tried to do the same shit with google plus and the glasses. The former is dead, the latter has a rather small niche existence. I can't recall any successful google product that became popular by being showed down the users' throats.
Re: WTF (Score:2)
Pap test is a cancer test (Score:1)
It isn't a cancer test (Score:3)
A pap smear detects the presence of HPV (human papilloma virus), among other things. That's where the name comes from - "pap" is short for "papilloma". It doesn't detect the presence of cancerous cells. HPV leads to an elevated risk of cervical cancer, so the pap smear is supposed to give an early warning that you may be at risk of cancer. But the article is correct, a pap smear is not a cancer test.
You want it to stop? (Score:5, Insightful)
Prosecute the CEO for practicing medicine without a license.
Pity it will never happen.
Re: You want it to stop? (Score:1)
"Disclaimer: Google, its subsidiaries, and corporate affiliates do not provide medical advice."
Right up there with "Caution: contents hot" on coffee cups.
This is America. No one will stop you from wasting your hard earned currency on quack pills, lottery tickets, and the like.
Re: (Score:2)
Right up there with "Caution: contents hot" on coffee cups.
Ugh, this bullshit again. At this point you've chosen to stay ignorant.
Any company selling food needs to follow related food safety laws. Those laws include regulations around selling things at extreme temperatures. McDonalds had been told multiple times that they weren't complainant with those safety regulations. They willfully chose to ignore them, thinking the cost of settling lawsuits (there were multiple private settlements prior to the famous lawsuit) was less than the profit they'd make selling h
Re: (Score:2)
Trying to explain the facts on the McDonald's coffee case is hopeless. "Lawsuits are ridiculous" is a religious cult; people who believe that case was ridiculous can't accept any facts that conflict with that belief, and literally everything they "know" about the case, except that it happened, is incorrect.
Don't listen to any of this (Score:2)
Happened to me today (Score:3)
This happened to me today. I googled the possible interactions between two particular drugs, and the AI summary said they can be dangerous to take together. Every medical website I visited said they're safe to take together. So did my pharmacist and my doctor.
Re: (Score:2)
Re: (Score:2)
It doesn't matter. All the little links mean is that text from those pages was included in context. It will happily produce responses in direct contradiction to the source provided. Remember it is not producing a summary of the linked page. These things can't actually summary text, only produce text that looks like a summary.
Re: (Score:2)
This happened to me today. I googled the possible interactions between two particular drugs, and the AI summary said they can be dangerous to take together. Every medical website I visited said they're safe to take together. So did my pharmacist and my doctor.
This could never happen to me. I've instructed uBlock to deep-six that shit, and on those rare occasions when I use Google instead of DDG I see only a large white space where the huckster nonsense used to be.
Scrolling past the white space is annoying, but not nearly as annoying as scrolling past their AI foaming at the mouth used to be.
The problem is that's the top, default answer (Score:5, Insightful)
I'm smart enough to be skeptical, but my aunt wasn't. I don't fear them duping me. I fear them duping my extended family, especially the elderly half. It was tough enough to get them on computers and phones and online...now I have to tell them to not trust Google, of all things.
Re:The problem is that's the top, default answer (Score:4, Interesting)
The best thing Google (or any search engine) can find is occurrences of various words or phrases in the same document. Search engines have absolutely no sense of semantics. So, finding a drug name and the conditions it is intended to treat are equally likely to occur with that drug and its side effects. It's just words in proximity to each other.
AI is pretty much the same probabilistic crap on steroids.
Re: The problem is that's the top, default answer (Score:1)
Are you saying that pre-AI you could trust google? Or are you saying AI's fluency in English makes it seem more trustworthy than an old-style google search?
Re: (Score:2)
Re: (Score:2)
You can still to this day trust that Google's regular search results will contain your tokens.
You can not trust that Google's AI search results will contain your tokens. Further, it presents "citation" links which not only do not contain them, they do not support the statements they are connected to.
Both of those things are fundamental failures far worse than the problems with even the current non-AI google search results, let alone the older ones where they did less thinking for you and deciding what you w
Re: (Score:2)
Startpage (Score:2)
I use Startpage to avoid all the AI slop.
Innacuracies (Score:2)
I have experienced this. They need to implement some secondary AI "fact checking" (how, I have no idea) to cut this BS out. I have seen the most absurd explanations, that are obviously concocted. Extrapolate from that, and I can see where some very dangerous explanations can happen.
Re: (Score:2)
They need to implement some secondary AI "fact checking" (how, I have no idea) to cut this BS out.
There is only one way, and it is human verification. A human can think and an AI can't.
They could improve the results by having it check itself, but it would not fix the problem. Same with using another LLM, which might fix some problems, but cause others.
Also of course a human would also make mistakes, so no matter what, you can't fix it 100%.
You can obviously find incorrect information with a normal search, but AI can give you incorrect answers both for that reason and that it cannot think.
on par with featured snippets (Score:2)
But does the AI know that "Bubba's Bait Shop and Kolege of Medical Knowledge" and "The Mayo Clinic" have very different reliability? Claiming to be just as accurate as the former isn't a particularly strong statement.
Always cross check important information (Score:2)
Taking important advice from an AI or web search or even from a doctor without cross checking is dangerous. Iâ(TM)ve had licensed doctors make dangerously incorrect diagnoses- later corrected by other doctors. AI isnâ(TM)t magic - it can be a good source for information but no one claims itâ(TM)s flawless.
Turing test for humans (Score:2)
If you are stupid enough to ask ChatGPT for health advice you fail, and get what you deserve. This could be an overall positive for the Earth. Fewer morons...
Anyone who trusts... (Score:3)
...AI answers without independent verification deserves what they get
"We're The Phone Company, We Don't Care...." (Score:2)
"We don't have to." -- Ernestine (Lily Tomlin)
I'm pretty sure Google knows an unacceptably large number of its AI summaries are wrong, and they don't care. Just like they don't care their search engine has turned to crap. They have no reason to care.
Wrong questions (Score:2)
The question is not whether the answers align with those of medical professionals.
Much more important, from the business and legal responsibility standpoint, is whether they properly align with pseudoscience championed by Kennedy and his merry band of quacks.
The Guardian is asking the wrong questions.