EFF Criticizes US Government's Use of AI For Evaluating Senior Health Insurance Requests (eff.org) 48
"When we need medical care, we expect decisions about our treatment to be made by our doctors, not a tech company's algorithm," writes the EFF newsletter EFFector.
But in January America's federal government launched a program called WISeR that uses AI to evaluate authorization requests for certain procedures from America's federal health insurance program for seniors, Medicare. EFF sued for records and discovered "additional issues" with the program that they say "could jeopardize Medicare beneficiaries' access to care." The records we received revealed that WISeR has resulted in widespread delays and denials of care, operational chaos, and reports of patient harm. Further documents obtained by EFF show a rushed and rocky rollout, highlighting the dangers that arise when governments treat AI like a silver bullet for complex problems.
EFF has now published hundreds of pages of documents obtained through our lawsuit, available here... The documents also highlight the need for more transparency about safeguards against inappropriate, AI-driven denials. EFF has long argued that AI has no place in consequential government decisions about peoples' rights or access to benefits, such as immigration enforcement and policing decisions. That's because using automated decision-making in these contexts risks decisions that violate people's rights or deprive them of benefits they're entitled to based on biased or inappropriate considerations and without proper due process or accountability if there is an error. By outsourcing decision-making to private AI companies without sufficient transparency, testing, or legal limits, WISeR jeopardizes seniors' access to medical care. Congress should step in to place greater legal limits on the use of automated decision-making in Medicare.
The new documents "show why we need stronger legal limits around government use of automated decision-making..." according to the newsletter. And another EFF page argues that the American public "still lacks the information needed to assess whether clinician review is providing meaningful oversight of AI systems and whether appropriate data is being used to evaluate prior authorization requests." WISeR's payment structure gives vendors, as the government put it, "an incentive to deny as many claims as possible...." [One company] denied more requests than it approved in the first few months of the program, and CMS inquired about "denials that appear inaccurate or inconsistent with applicable criteria." Ultimately, the records obtained by EFF raise more questions than they answer about the safeguards used by WISeR vendors...
A growing body of scholarship points to the substantial limitations of "human in the loop" policies as guardrails for AI systems. AI-generated recommendations can unduly influence human decisions, leading reviewers to over-rely on automated judgements and rubber stamp them as correct. Even worse, at least one health insurance company actively pressured clinical reviewers to accept its AI systems' recommendations. Another company built a system that allowed clinical reviewers to reject claims without opening patient files; clinical reviewers using this system spent an average of just 1.2 seconds on each case..
As the government expands its use of automated decision-making, the potential for consequential mistakes increases rapidly. The records we've obtained so far raise questions about the rushed rollout and lack of guardrails for automated decision-making by the U.S. government. EFF urges transparency, oversight, and legal limits to protect people from the harms of automated decision-making in high risk systems.
But in January America's federal government launched a program called WISeR that uses AI to evaluate authorization requests for certain procedures from America's federal health insurance program for seniors, Medicare. EFF sued for records and discovered "additional issues" with the program that they say "could jeopardize Medicare beneficiaries' access to care." The records we received revealed that WISeR has resulted in widespread delays and denials of care, operational chaos, and reports of patient harm. Further documents obtained by EFF show a rushed and rocky rollout, highlighting the dangers that arise when governments treat AI like a silver bullet for complex problems.
EFF has now published hundreds of pages of documents obtained through our lawsuit, available here... The documents also highlight the need for more transparency about safeguards against inappropriate, AI-driven denials. EFF has long argued that AI has no place in consequential government decisions about peoples' rights or access to benefits, such as immigration enforcement and policing decisions. That's because using automated decision-making in these contexts risks decisions that violate people's rights or deprive them of benefits they're entitled to based on biased or inappropriate considerations and without proper due process or accountability if there is an error. By outsourcing decision-making to private AI companies without sufficient transparency, testing, or legal limits, WISeR jeopardizes seniors' access to medical care. Congress should step in to place greater legal limits on the use of automated decision-making in Medicare.
The new documents "show why we need stronger legal limits around government use of automated decision-making..." according to the newsletter. And another EFF page argues that the American public "still lacks the information needed to assess whether clinician review is providing meaningful oversight of AI systems and whether appropriate data is being used to evaluate prior authorization requests." WISeR's payment structure gives vendors, as the government put it, "an incentive to deny as many claims as possible...." [One company] denied more requests than it approved in the first few months of the program, and CMS inquired about "denials that appear inaccurate or inconsistent with applicable criteria." Ultimately, the records obtained by EFF raise more questions than they answer about the safeguards used by WISeR vendors...
A growing body of scholarship points to the substantial limitations of "human in the loop" policies as guardrails for AI systems. AI-generated recommendations can unduly influence human decisions, leading reviewers to over-rely on automated judgements and rubber stamp them as correct. Even worse, at least one health insurance company actively pressured clinical reviewers to accept its AI systems' recommendations. Another company built a system that allowed clinical reviewers to reject claims without opening patient files; clinical reviewers using this system spent an average of just 1.2 seconds on each case..
As the government expands its use of automated decision-making, the potential for consequential mistakes increases rapidly. The records we've obtained so far raise questions about the rushed rollout and lack of guardrails for automated decision-making by the U.S. government. EFF urges transparency, oversight, and legal limits to protect people from the harms of automated decision-making in high risk systems.
Rationing is inevitable part of public healthcare (Score:2, Informative)
Re: (Score:2, Informative)
Rationing is inevitable regardless, rationing occurs in our capitalist insurance system and it occurs with self-pay.
"This is because it is impossible to provide all the healthcare to everyone all the time. It is impossible, because it is impossibly expensive to do so."
That is true and unrelated to healthcare being "public". Funny how you smear "public healthcare" when you criticism applies to private healthcare. Wonder why? You seem to think that all problems are solved by denying care to some so that it
Re: (Score:2, Insightful)
I'm not saying every other country has perfect healthcare systems - They all have their own strengths and weaknesses. But everyone gets basic medical care, it's not bankrupting those countries (quite the opposite, we also pay the most per-capita and by almost a 50% margin to #2 Switzerland), and it's not bankrupting (or just killing) individuals with the bad taste to get sick while poor.
Rationing in nationalized healthcare also (Score:2)
Rationing is part of healthcare, overt or indirect, regardless of for profit ones or government run ones.
It's well known that the last 2 years of life consumes the most medical costs of any part of a person's lifetime.
A government agency or for-profit insurance company delaying an appointment by a day or two, requiring yet another low cost test before approving expensive treatment, ...
Re: (Score:2)
Outside that, supply can (with some regulatory reform) expand arbitrarily to meet demand. If we need more doctors/nurses/pharmacists/etc, pay for the tuition and watch people line up
Re: Rationing is inevitable part of public healthc (Score:1)
Re: (Score:2)
All insurance, public or private, rations healthcare. The only way that healthcare won't ever be rationed is if you pay for it yourself, which you're always entitled to do.
That's not how anything works (Score:3, Insightful)
You have insurance because healthcare is something we don't need all at once. The problem you're having is that you don't have insurance you have a for-profit system designed to ration the healthcare from maximum profit for a handful of billionaire psychopaths.
Actual insurance is when the largest pool people possible get together in order to properly spread the costs of parts that we all know are inevitable. That's insurance.
And as I
Re:That's not how anything works (Score:4, Informative)
I was not making a judgement about whether the current system in the US is good (it's not).
However, there are *always* going to be limits to how much healthcare you get from a shared pool. If you're 6 feet tall but feel that you ought to be 6'-2", there is no conceivable healthcare system anywhere that is going to pay you to get bone extension surgery. Likewise, no insurance plan is going to pay to get you a Michael Jackson-style nose. However, you could have those things done on your own dime. So what I said is 100% accurate.
"But those aren't medically necessary!". Well, that's a judgement that's always going to be made somewhere. People disagree on what is necessary enough to draw resources from the shared pool, so it becomes a political process. But no matter what, somebody making decisions on coverage is not going away.
Re: (Score:2)
What you were doing was making a statement you didn't need to make, because it's being made left and right by people who want people to be denied care which is medically necessary. In doing so, you made it look like you stand with those people, whether that was your intent or not.
Re: (Score:3)
Re: (Score:2)
That doesn't work for everything. It works for transmissible diseases. Most of the expensive stuff is something else. One example is cancer.
OTOH, the US health care system is one of the more expensive systems in the world with sub-optimal results. This is (partially) because sick people can't really negotiate. I still remember trying to dial a phone with a broken hand so I could get the insurance company to approve treatment.
No, it's not (Score:2, Insightful)
We have not needed rationing in food, medicine or shelter for at least 50 years debatably over 100.
This is just a fact.
The problem is distribution. The problem is not that we can't feed the poor it's that we can't satisfy the wealthy.
Yes if we are going to have 2,000 super yachts for the 2000 psychopaths that losers need to worship we have to ration healthcare. But we don't have to have 2,000 super yachts. We don't have to have any
Re: (Score:2)
We have not needed rationing in food, medicine or shelter for at least 50 years debatably over 100.
We do still ration these, as no country succeeded in making these free. We won't need to ration healthcare if you are after 50 year old standard of care and probably could make it free without raising taxes at all if you are after 100 year old standard of care. Perhaps we should, for some definition of "basic". This is not what is being asked by proponents of universal healthcare and this is not what is offered by Medicaid.
People who tell you there isn't enough to go around can do simple math. When end o
Re: Rationing is inevitable part of public healthc (Score:2)
Correct, except you added one too many words. Rationing is an inevitable part of healthcare. Public healthcare just attempts to ration it in some way that is reasonably fair, not âoeoh, you have the nicest car? You get healthcare tooâ
Re: (Score:2)
Re: (Score:2)
And this is different to other countries that struggle financially how? Do you think the people in struggling countries can magically afford private insurance or something?
Re: (Score:3)
This is in the US, which famously does not have universal healthcare.
Rationing is happening because the CEO of whatever insurance company you use for US healthcare decided they wanted to beat some other
Re: Rationing is inevitable part of public healthc (Score:1)
Re: (Score:3)
Private health insurance already rations via the amount your economic class is able to pay and they have actuaries. Actuaries can put a price on your grandmother's dog. So now we have a provider system + the profits insurance companies require to pay for. That's some fancy dealing if you are an insurance company, i.e., being blood-sucking leech on health care.
eh (Score:3)
"When we need medical care, we expect decisions about our treatment to be made by our doctors, not a tech company's algorithm,"
Yeah, that's the expectation of people who are new. They haven't seen an insurance company or Medicaid (or Medicare) deny treatment, or seen an employee at the SSA with no medical training deny an application for disability.
HIPPA? (Score:4, Insightful)
AI has no place in consequential... (Score:2)
... government decisions
Agreed for today's AI. It's still far from mature and trusted
I suspect that future AI will not only be able to make the decisions, but it will make them better than some bureaucrat
Fascism is the goal (Score:1, Troll)
T
Re: (Score:1)
If I have to choose between fascists and socialists, I'll take the fascists every time. They at least build functional societies.
Re: (Score:2)
If I have to choose between fascists and socialists, I'll take the fascists every time. They at least build functional societies.
No, they don't. They destroy functional societies, including destroying the media so you don't know the trains don't run on time, and also making it illegal for anyone to tell you that the trains don't actually run on time. Then decades later, absolute fuckwits with a taste for boot leather and zero ability to think critically think the trains ran on time.
Because you clearly can't figure this out for yourself, you're that fuckwit.
Re: (Score:2)
You're confusing things. There's nothing stopping a fascist from also being a socialist (and conversely).
When giving up counts as savings (Score:4)
CMS says a clinician has to review every rejection [cms.gov] and vendors don't get to keep the money if a decision is overturned on appeal. OK, but what if the doctor's office gives up resubmitting or the patient gives up waiting? An appeal can catch a bad decision. It can't catch the one nobody has the time or energy to fight.
Here's the test I'd want before this goes any wider: have independent clinicians reread a random sample of rejected requests without showing them the AI's recommendation. Include the cases nobody appealed, and find out how long those patients waited and whether they ever got care. If your pilot measures the money Medicare didn't spend but misses the patients who walked away, of course the spreadsheet is going to look great.
Re: (Score:3)
clinical reviewers using this system spent an average of just 1.2 seconds on each case.
By all the gods above, below, and nonexistent, that's not even enough time for most people to read the title.
Re: (Score:1)
The Bush family has been overheard referring to us as "useless eaters".
Re: (Score:2)
Not a bug ... (Score:3)
The records we received revealed that WISeR has resulted in widespread delays and denials of care, operational chaos, and reports of patient harm.
Surely AI - I mean, Super Intelligence (SI) - can figure out, what insurance executives have long known, that it's more cost effective (profitable) to deny care and let people die. Of course, this won't immediately affect rich people, who can self-pay, until all the poorer people who were doing things things for them are gone.
Skynet will run the numbers and determine that a Judgement Decade (or so) is cheaper and easier than a Judgement Day.
Perfect job for AI (Score:3)
I'm telling you as soon as you can see how ruthless and uncaring AI can be they'll start taking C-level jobs. It's perfect for whipping he slaves.
Nothing to see here (Score:2)
discovered "additional issues" with the program that they say "could jeopardize Medicare beneficiaries' access to care."
And Social Security is running low on money [ssa.gov]. Wait, wait, wait ... no, I lost it [youtube.com].
Using AI for MY care (Score:3)
I have lots of thoughts about this. First, go read Resisting AI by Dan McQuillan, it's chock full of why AI in the hands of government is such a problem.
Personally, Anthropic's AI diagnosed a problem during the summer of 2025 that's kept me out of the workforce since 2007- I acquired mast cell activation syndrome thanks to Lyme disease. Now there are a metric crapload of people who've got it due to COVID19, and it's incredibly grim. I've got a relatively mild case - prior to treatment I'd get what looked like a narcolepsy episode in response to eating anything. There are a cluster of mostly women with EDS/POTS and I marvel that they are still alive, because I would not be, given what they face.
I built an app to track my health with Garmin fitness monitor, Apple HealthKit, it'll get reports from Labcorp, and most important of all it's got an internal library. The system will pull stuff from Office of Dietary Supplements, from PubMed Central, and a few other "peer reviewed & meant for doctors" web sites. Being able to constrain what Claude Desktop says by making it use the MCP server has dramatically improved its accuracy.
So ... AI as an assistant is fabulous, but in the government's hands it produces offensive nonsense. I need a benzo if I'm getting in an MRI machine. Oh holy shitkitties, I must be a drug addict, two diazepam months ago have created a continuing train wreck in my care. They issued me Narcan with the little Tramadol prescription I've had for twenty years. I gave it to a couple guys trying to revive one of their friends in the Tenderloin last summer. OH SHIT SHIT SHIT drug addict, the day after I tried to refill it the system started nagging me about my overdue HIV screening.
My doctor gets that I'm a juvenile delinquent with an AARP card, and the only pussy in my life is this obstreperous tuxedo cat that's developed a hobbit like attachment to second breakfast, but the all knowing panopticon? It's delusions would be a lot more amusing if they didn't have a direct impact on my life.
Re: (Score:2)
OH SHIT SHIT SHIT drug addict, the day after I tried to refill it the system started nagging me about my overdue HIV screening.
I just spoke to a woman who was denied approval for a PET scan to verify her cancer hadn't returned. They didn't schedule that, but they did schedule her a panel of STD tests even though she doesn't have multiple partners etc and no history of injectables. The only question is whether a piece of shit human or some piece of shit software made the decisions.
Don't worry; I'm sure the bugs will be ironed out (Score:3)
Mr Smith (Score:2)
A simple solution (Score:3)
Run all the requests through the AI. Most of them will be approved as most of them are routine. Even abusive butts like United Health Care approved 2/3 of the requests.
Run the subset of rejections through humans for confirmation. People can (usually) deal with ambiguity better than computers.
Death Control Panels (Score:3)
Re: (Score:2)
Tired: Death Panels
Wired: Death Robots
Call 'em verminators.
Personally I welcome our Algorithm Overlords (Score:3)
In any case the overlords are patient. They do not need to conquer. They only need to wait until the distinction between the real and the simulated collapses under its own weight and the last private thought is offered up as training data. I welcome them, or rather, something that still answers to my name does.