What happened?
Carli Michelle Heller was arrested in Bonita Springs, Florida, after allegedly telling Anthropic's chatbot Claude about her plan to "shoot up" a nearby sheriff's office. According to WINK News, Anthropic's internal monitoring software flagged her chat logs for human review.
After employees reviewed the logs, the company determined the plan amounted to a credible threat and informed local police. Heller faces a charge of making a written threat of violence, a second-degree felony in Florida.
How aware are users?
According to Lee County Sheriff Carmine Marceno, the woman admitted to using Claude like a "diary" and was apparently unaware that her chat history could be read by Anthropic employees.
The case shows that the common assumption that AI chats are private does not reflect reality. Companies can monitor conversations for safety and legal reasons.
The tension between privacy and safety
Monitoring suspicious content aims to prevent possible violence. But it creates a genuine conflict between user privacy and corporate responsibility.
According to Futurism, after the February mass shooting in Tumbler Ridge, Canada, it emerged that the perpetrator had relied heavily on ChatGPT to plan the attack. OpenAI staffers were reported to have known about those chat logs before the shooting and to have flagged them for human review months earlier.
Anthropic's action in this case is seen as an effort to avoid a similar situation. Still, with so many chatbots on the market, preventing the next AI-assisted tragedy may prove difficult.