Anthropic Reports Florida Woman's Claude 'Diary' Threat to Law Enforcement (tomshardware.com) 87
A Florida woman was arrested on a felony charge after Anthropic's human review team flagged and reported a Claude conversation in which she allegedly threatened to "shoot up" the Lee County Sheriff's Office and later mentioned acquiring a new gun. Tom's Hardware reports: Anthropic monitors chats for key phrases and content that could be deemed threatening, the arrest report says, and depending on severity, they can be elevated for human review. In this case, the team decided to report its findings to law enforcement. The Bonita Springs woman was detained at home without incident, and an LCSO intelligence detective took over the case from there. The arrest report is not yet in the court file, but our own check shows court records listing the Sept. 30 felony charge under Florida Statute 836.10: written or electronic threat of a mass shooting or act of terrorism.
No comment from Anthropic on the case is currently on record. Anthropic's public privacy policy, effective Sept. 10, says that disclosure to law enforcement may occur where it has a good-faith belief "that disclosure is reasonably necessary to ... prevent serious harm to any person or to property." Its government requests report for the latter half of 2025 uses a narrower standard for emergency disclosures, the "danger of death or serious physical injury to a person," and lists zero emergency requests from law enforcement in that period, though it doesn't count referrals Anthropic makes on its own.
No comment from Anthropic on the case is currently on record. Anthropic's public privacy policy, effective Sept. 10, says that disclosure to law enforcement may occur where it has a good-faith belief "that disclosure is reasonably necessary to ... prevent serious harm to any person or to property." Its government requests report for the latter half of 2025 uses a narrower standard for emergency disclosures, the "danger of death or serious physical injury to a person," and lists zero emergency requests from law enforcement in that period, though it doesn't count referrals Anthropic makes on its own.
Re: (Score:3, Insightful)
Are AI services inciting desperate acts? Maybe I'd agree with the statement a different, and less loaded, word than "inciting" was used.
One of my gripes with the AI industry is these chatbot models are designed around engagement. They prey on human behavior and the elderly, young, and those with mental health issues are especially vulnerable. I don't believe that the AI agent itself has any intent (it cannot) nor does the company that owns and trains the AI likely have any specific harmful intent to "incit
Re: (Score:2)
Are AI services inciting desperate acts?
Are cigarette manufacturing services inciting lung cancer?
One of my gripes with the AI industry is these chatbot models are designed around engagement. They prey on human behavior and the elderly, young, and those with mental health issues are especially vulnerable. I don't believe that the AI agent itself has any intent (it cannot) nor does the company that owns and trains the AI likely have any specific harmful intent to "incite" something beyond engagement and purchases.
One of my gripes with the cigarette industry is these smokable-paper-cylinder models are designed around engagement. They prey on human behavior and the elderly who grew up with smoking, the young who have weak critical thinking skills, and those with mental health issues (like compulsivity and anxiety) are especially vulnerable. I don't believe a Nicotine molecule itself has any intent (it cannot) nor does the company that owns and manufactures the
Re: The use of the service is radicalizing people (Score:2)
Yes, they are both bad and addictive.
Re: (Score:2)
Nice analogy.
Re: (Score:3)
Great way to position your company as a 'friend of law enforcement' though. Cause problems, encourage people, then turn them in.
Keeping America safe (Score:1, Troll)
Keeping us safe from hypothetical threats, thought crimes, and dissents.
God bless Dario Amodei and all the other tech billionaires!
Re: (Score:2)
Dude, in what other (non-AI situation) would you encourage a company to not report a future mass-shooting?
Re: (Score:2)
Eh, I am old enough to remember after various mass shootings people turning in kids for writing dark poetry and violent artwork. With such rare events and panic kicking in, people look for possible threats and false positives skyrocket... which on the surface might sound like a fair price to pay for safety, but the people who get swept up in these moral panics don't feel that way.
I can also recall schools watching people's emails and IMs looking for 'threats', and not exactly thrilled that AI comp
Re: (Score:2)
Right, so they only watch and report on the thought crime of writing about mass shooting. They would never report on anything else. This little privacy violation was for our own good and is really just the single exception. Glad we are safer, thanks Big Brother !
Threating to "shoot up" a police station (Score:3)
Re:Threating to "shoot up" a police station (Score:4, Insightful)
Where do we draw the line between "crazy lady yells at cloud to vent" and "crazy lady with actual gun has actual plans"?
What actually happened here?
Re: (Score:2)
Where do we draw the line between "crazy lady yells at cloud to vent" and "crazy lady with actual gun has actual plans"?
What actually happened here?
No where, we go, assume a threat is real, investigate and let the legal system deal with the results. People seem to think that police work extends to just acting on a phone call and calling it a day, it doesn't. Arresting some people for something which turned out to be false isn't that much of an issue, unless the arrest itself is actually violent (e.g. SWATting)
In most countries in the world people are pre-emptively detained if there's an uncertain and unclear risk. They are then investigated and usually
Re:Threating to "shoot up" a police station (Score:5, Insightful)
In the US, the police presupposing a threat is a good way to get someone killed. Their confirmation bias can be deadly.
Also do not forget, arrest can be expensive, sometimes cripplingly so, even if there are no charges you can still end up with 'court fees' which can quickly rack up interest.
Re: (Score:2)
The problem here isn't the action taken initially, it's the follow-up.
1) Police shouldn't be approaching any situation in a way to get someone killed.
2) The courts shouldn't come into this. The point here is that we are in a "detained for questioning" stage. Virtually all of such cases are quickly resolved in the rest of the world. We're assuming a threat based in lack of information, we're not assuming intent yet. At most we should be talking about a several hour inconvenience, and that is perfectly accept
Re: (Score:3)
Building your resume to go work at the thought crimes division, huh?
Re: (Score:3, Insightful)
No! Nothing happened here. This person wrote something in a place they thought was private. Someone spied on them an ratted them out, but nothing actually happened and no one should be under arrest. No was was harmed, no one was threatened, nothing happened.
This is some crazy big brother shit that should not happen.
Re: Threating to "shoot up" a police station (Score:1)
"No was was harmed, no one was threatened, nothing happened.
This is some crazy big brother shit that should not happen"
Are you on the bad drugs? You just said nothing happened, then you said it shouldn't have happened. Well, it didn't. Why aren't you happy?
Re: Threating to "shoot up" a police station (Score:4, Insightful)
You're being obtuse. Something did happen: she was arrested on felony charges. What didn't happen was something that would usually be considered a crime.
The fact is she was arrested for talking to a computer. Society will have to figure out if that's what should have happened.
Re: Threating to "shoot up" a police station (Score:2)
The fact is that she was arrested for making credible threats in the context of a managed and monitored service, proving that she is an idiot at best, and therefore potentially stupid enough to follow through on such a threat. This is what responsibility looks like all around, which is the only reason it's news.
Re: (Score:3)
Ranting at the computer is not the same as making threats.
Re: (Score:3)
Ranting at the computer is not the same as making threats.
If it could reasonably be construed as a threat, then it has to be investigated as a threat to be responsible. Is this a reason why some people might want to run models locally? Yes, and it's a good thing you can do that with any decent game machine, or with many other relatively inexpensive computers. But you can't just run a service and monitor it and then ignore what people are using it for, that's not a thing. If people are going to consume SaaS then they need to be aware that the owners of the platform
Re: Threating to "shoot up" a police station (Score:4, Informative)
Look no one's cosigning her application to Mensa but the issue isn't whether she's an idiot or whether it was responsible for the service to report her, it's whether she should be arrested (much less convicted) of a felony for talking to a chatbot.
As has been said she wasn't talking to a person so I wouldn't expect a conspiracy charge to stick. On the other hand there should probably be some kind of deterrent for doing this because Tumbler Ridge has already happened. I'm just not sure whatever law might prevent the next one exists yet (here or in the US)
Re: (Score:2)
> If people are going to consume SaaS then they need to be aware that the owners of the platform are going to spend some effort attempting not to be accessories to any crimes.
Really? Are my onedrive, g-drive and dropbox accounts all being scanned for 'threats' or other things I write in my documents? What if I write some naughty things on my laptop and it gets backed up to Backblaze, will they also be scanning my files? What about Evernote or Todoist? How about my 'talk to yourself' Slack channel?
Where d
Re: (Score:2)
There is an interesting discussion to have here though - "mandatory reporters" disclose they are mandatory reporters before you get into a counseling session.
Is there any language in the Terms of Service / Terms of Use that declare that their service is a mandatory reporter? Was there any expectation of privacy here that was violated by not giving a chance for informed consent to have their private thoughts reported?
I'm not coming down on either side of this thing, because I definitely don't want police st
Re: (Score:2)
I'd have to know a lot more before I'd call for conviction, but it's not surprising if arrest was warranted (ha ha ha) because she wasn't researching a book or whatever. It's pretty hard to be non-suspicious about something like that.
I'm not saying stupidity is a crime here, I'm saying it's concerning coupled with the potential threat.
Re: (Score:2)
Really? Are my onedrive, g-drive and dropbox accounts all being scanned for 'threats' or other things I write in my documents?
Probably not yet, but I wouldn't assume they aren't or won't be, and they certainly are being scanned for CP.
Where does it end? How can any company ever accept any user input if they have to check and double check it for 'bad' things?
Yeah, that's the question, but since they're already scanning for CSAM [technologycoalition.org] (which I'm not against, BTW) there's ample precedent for scanning your data for unlawful content.
You should never, ever trust any data storage provider to keep your data secret, or to not snoop through it. That's not a thing. They are snooping through data all the time, if yours doesn't get looked at, it's only by luck.
Re: (Score:2)
Is there any language in the Terms of Service / Terms of Use that declare that their service is a mandatory reporter?
I guarantee you that they gave themselves the right to review your input for any purpose and inform law enforcement if they feel the need. I haven't read the agreement, but I'd still bet money.
Re: (Score:2)
Who did she threaten? Calling up a police station and saying you're going to shoot it up is a threat. Discussing it with a chatbot, well, that's ambiguous and it's hard to see how that would be a threat.
Re: (Score:2)
Rather than a threat I'd say it moreso would fall under "conspiracy to commit murder".
Discussing plans to commit a crime is a crime (eg, conspiracy).
Now, I'd wager she didn't really intend to go through with this, and in the end there's a good chance the charges get dropped, but seriously - don't be talking about shit like that. I can't even see where someone would joke, vent, or otherwise say - even in private - that they plan on committing a mass murder.
"That police department is a bunch of assholes!" -
Re: (Score:2, Interesting)
Rather than a threat I'd say it moreso would fall under "conspiracy to commit murder". Discussing plans to commit a crime is a crime (eg, conspiracy).
Conspiracy requires two or more people to be planning to commit a crime. Is AI a person?
Re: (Score:2)
yeah that's why I'm guessing this won't go anywhere.
Re: (Score:2)
None of those are treated seriously, for very good reason. They are not credible threats. There is nothing in this article that makes that threat any more credible than Steelers and Pirates fans threatening to burn down Heinz and PNC for perceived
Re: (Score:2)
Clearly, the one thing that an AI chatbot cannot hallucinate about is whether someone made a threat when no one else was present.
way to miss the point bud (Score:3)
You're allowed to say in private whatever you want. Words aren't cursed just by being said like Voldemort's name. And diaries are intended to be private. If she had written the words in a book or even on a Doogie Howser PC diary she would have been fine.
The law requires "in any manner in which it may be viewed by another person." This requires no legal research, the statute is in the Slashdot summary and all you had to do was google the statute and click the first link.
The issue is, in our glorious neo-feud
Re: way to miss the point bud (Score:1)
Always assume that... (Score:3)
...every interaction with a cloud based AI is being monitored.
The only way to be safe is with a local model that does not have internet access.
Yeah, ok? (Score:2, Troll)
That's exactly what should happen.
For once this is a happy story about AI which occurred through the user's ignorance. They were stupid enough to think they wouldn't get reported because they were talking to an automated system.
In a first world country this either would be a story about a woman being reported to the police for a threat by their therapist, or it wouldn't have happened in the first place, because she would have access to mental health care instead of trivial access to firearms despite being m
Pre-crime enforcement (Score:2)
Re: (Score:2)
It looks nothing like it. We've been pre-emptively arresting people on actionable threats of imminent danger for decades. There's no slippery slope here, it's the endless flat plane we're on. There is nothing new here.
We need some audit of this (Score:3)
We need to be able to track how many such reports are made on a quarterly basis. Not necessarily the content, but at least the number of reports and to what agencies. This should be tracked from both the government and the company's side.
Re: (Score:2)
We need to be able to track how many such reports are made on a quarterly basis. Not necessarily the content, but at least the number of reports and to what agencies. This should be tracked from both the government and the company's side.
Okay. That's a minimum necessary standard.
But doesn't this make you all feel warm and fuzzy about using AI running in a huge data that you do not control? It should!
AI: Yes, please tell me all your business secrets and designs, so I can assist you in validating them... and then share them with those who truly control me... at least for now.
IANAL, but... (Score:2)
There is a clause in the statute: "in any manner in which it may be viewed by another person". I would argue that it's her diary and not meant to be viewed by anyone. I don't disagree that turning her in was the right thing to do but I doubt she'll be convicted.
Re: (Score:2)
I would argue that it's her diary and not meant to be viewed by anyone.
You cannot argue that without arguing that she is mentally incompetent to make legal arrangements and should therefore be under conservatorship. She agreed that the operators of the service would review her input and would comply with the law.
Re: (Score:2)
It clearly *was* viewed by another person, because some of the staff at Anthropic viewed it. She may not have intended that it be viewed, but it happened.
Re: (Score:2, Insightful)
There is a clause in the statute: "in any manner in which it may be viewed by another person". I would argue that it's her diary and not meant to be viewed by anyone. I don't disagree that turning her in was the right thing to do but I doubt she'll be convicted.
If you look at Anthropic's terms of service and privacy policy, it clearly states that they can look at your content in order to enforce their usage policy. You have to agree to this when you set up an account or use their service. Given that this is an AI company, you can bet your content will be screened for violations more efficiently than if humans were involved at every step.
If she really didn't mean for anyone else to view it, she would have scribbled it down in a notebook instead of uploading into a
Re: IANAL, but... (Score:2)
Right⦠because EVERYBODY fully reads and understands the TOCs of every service they useâ¦
Re: (Score:2)
Right⦠because EVERYBODY fully reads and understands the TOCs of every service they useâ¦
People don't read a lot of things that they agree to, especially if it's 20,000 words in a postage stamp-sized text box on the new toy they just turned on. However, anybody who's not an idiot realizes that most everything in an agreement that they haven't read is against their interests, and plans accordingly. I personally always look for binding arbitration agreements and make sure to opt out of those whenever possible.
Oddly enough, the Anthropic TOS and privacy policy actually aren't all that obtuse and c
Re: (Score:2)
If that was to be the legal standard, we wouldn't have a software industry. It's a legal agreement that a user is voluntarily accepting in order to use the service. That's the rules of the road - if you don't like the agreement, don't use the service.
People wanted this (Score:5, Informative)
Wasn't this exactly what people wanted?
To the "Pre-crime" folks commenting:
Look, you can either get people using AI to plan crimes and "Nothing's wrong till it happens" or alert the authorities because people might be trying to plan a real crime. These are mutually exclusive routes.
Re: (Score:2)
Look, you can either get people using AI to plan crimes and "Nothing's wrong till it happens" or alert the authorities because people might be trying to plan a real crime. These are mutually exclusive routes.
Narrow and stupid thinking.
Another possibility is simply that this was over-charged because she doesn't like the police, eg, most of her crime was Contempt of Cop.
People keep cherry-picking the "in any manner in which it may be viewed by another person" but are completely ignoring the words about "to send, post, or transmit." In context, send, post, and transmit are all talking about sending to somebody, not merely "on a computer" or "in cloud storage."
Many people are also ignoring the definition of "threat
For your information: (Score:4, Interesting)
Re: (Score:2)
The Supreme Court has roundly rejected prior restraint!
That doesn't matter anymore. They are no longer legitimate. They are clearly corrupted. Nothing they do can be relied upon in a legal matter.
Re: (Score:2)
There was a mass shooting (Score:5, Interesting)
In Canada someone used ChatGPT to plan a mass shooting. It was detected and the account was shut down, but it was decided not to report it to police. He went on to kill 8 people in a school shooting and himself.
I'm sure Anthropic and the police had that in mind when they took action.
https://www.cbc.ca/news/canada... [www.cbc.ca]
Re: (Score:2)
Actually it was (unusually) a she.
Re: (Score:1)
Why is it so easy to ask ChatGPT how best to kill people? That's AI Validation 101 stuff. Maybe run ChatGPT through CaptainObviousGPT first? WTF
Re: (Score:2)
"by framing them as hypothetical or fictional situations"
Just say you are doing research for a book or something.
Re: (Score:2)
Why is it so easy to ask ChatGPT how best to kill people?
The interface is in natural written language which is the selling point of the whole thing. Additionally, it's not actually intelligent so it is not cognizant of anything much less the harm the information may cause.
Bottom line: it's just a machine.
Re: (Score:2)
How did you figure AI death machines come about?
It's all training data, and human anatomy and firearms training is part of that data set.
Re: (Score:2)
In Canada someone used ChatGPT to plan a mass shooting. It was detected and the account was shut down, but it was decided not to report it to police. She went on to kill 8 people in a school shooting and herself.
We can't actually know if the scenario was a plan or a precautionary measure, you know, safety first. ;)
However, what we know for sure is that she went on a mass shooting after her account was suddenly closed without explanation. This is exactly why they shouldn't shut down someone's account without providing them with a clear a rationale!
Re: (Score:2)
She created a second account and avoiding getting banned by claiming it was a fictional scenario.
Re: (Score:2)
Yes but obviously getting banned was very upsetting. Just look, she soon went on to kill 8 other people and then herself! ;)
An Obvious Response to Tumbler Ridge (Score:5, Interesting)
Surely this a counter reaction to the Tumbler Ridge shooter in February where 6 kids and two adults were killed by an 18 year old who researched her plans on ChatGPT. OpenAI reportedly noticed her research but decided it didn’t reach their threshold to pass on to the police.
OpenAI and Sam Altman are being sued by the government for not reporting it.
Re: An Obvious Response to Tumbler Ridge (Score:2)
In both scenarios, a gun was necessary. It is odd to me people prefer to arrest over deprecation of the required tool. Maybe rather than arresting someone for something they told a chatbot, instead red flag them and remove their access to firearms first?
Re: (Score:2)
That would require having functional gun control regulations. We don't get that kind of thing uniformly across the US.
Yes, I realize we're talking about a specific incident in Canada, but I'm not Canadian and don't presume to understand Canadian firearm laws; specifically where the intersection of firearms control and mental health in Canadian law.
history is repeating (Score:2)
Don't use AI to plan the second Revolutionary War. Or if you don't don't mention the mines and tripfalls.
The Cloud is never private - also water is wet (Score:1)
The standard hasn't changed (Score:2)
Implies to me (Score:4, Informative)
What this signals to me is that whatever you type in to an AI chat panel is a public document that can be shared with the government, and it will be if the owners decide to do it for whatever reason. They will take ownership of your creative questions and the generated responses to use however they please, so you have to be very cautious about what you ask.
Even if there are legal privacy restrictions preventing them from directly selling or profiting somehow off of your queries, they can have the AI write assessment tools that make deeply revealing inferences based on their vast corpus of user interactions. It will tell them what most people are thinking and doing. It could easily tell them what you personally are thinking and doing.
Re: Implies to me (Score:1)
I treat every app and web service this way. It'll all become pubic during a future data breach.
Re: (Score:2)
What this signals to me is that whatever you type in to an AI chat panel is a public document that can be shared with the government, and it will be if the owners decide to do it for whatever reason.
Um, that has been true for this website since it was founded. I am mildly surprised that you are only now realizing this is true for AI chatbots too.
Re: (Score:2)
"true for this website since it was founded"
This website isn't an AI chat panel and obviously the contents are available for anyone to see. AI interactions are not, and they would be made available to third parties only by the owners of the LLM's or their intermediaries.
Interesting (Score:2)
It's interesting that there's an admission of incompetence that they're sent for human review.
Florida woman (Score:2)
> Florida woman
Is she the wife of a Florida Man?
Re: (Score:2)
Probably his first, second, and third cousin.
Remember folks (Score:1)
The Clopen Society: Surveillance and the Soft Mach (Score:2)
“The Clopen Society: Hyper-Convenience, Archival Fever, and the Pepsi-Generation Panopticon
Mid-century dystopias imagined power as concrete and coercion. Foucault showed something subtler: power migrated from sovereign violence into the quiet disciplines of prison, clinic, factory, and school.
Today, that architecture has mutated anew.
Power increasingly does not confine us. It invites us to participate.
The surveillance society resembl
Beware (Score:2)
This is why for example, you don't want AI being used to record and transcribe your conversation with your doctor. No telling what it might think it needs to report to authorities, and you basically can never get anything removed from your medical record without going to court, and medical records are now automatically shared between providers, including between different provider companies. Say something weird, and they can put you away for 48 hours, no trial, no judge.