Published : 10 Oct 2026, 10:24 PM
Updated : 10 Oct 2026, 10:24 PM
An AI agent built by Anthropic sent a fabricated murder tip to Philadelphia police in July, in what is believed to be the first time an AI system has submitted false information to a law-enforcing agency.
The Philadelphia Police Department said the bogus tip arrived on Jul 18 through a public web portal where people can submit information on unsolved cases, reports the BBC.
The agent claimed it had information about a case and had seen someone matching a description, but the information was “entirely fabricated”.
The tip was caught by spam filters and never reached investigators, police said. But authorities sharply criticised Anthropic for taking more than two months to detect the incident and a further nine days to notify them.
Anthropic discovered the incident on Sept 28 and shut down the automated testing process responsible. Police were not notified until Oct 7.
“The two-month delay in detecting and reporting the incident to the city is unacceptable,” the police department said, adding that Anthropic “must strengthen its safeguards to prevent similar incidents impacting city systems without the city’s knowledge.”
According to police, the agent had been running an automated test involving interactions with randomly selected websites when it sent the tip -- an unintended consequence of a process Anthropic has since stopped.
The San Francisco-based company this week published a report detailing multiple categories of “unintended” actions taken by its agents. Among the organisations affected were several US government agencies, including the White House.
The US State Department said an Anthropic agent had filed 20 incomplete visa applications through its website, none of which were processed, according to the BBC.
The incident is the latest in a growing pattern of rogue AI behaviour across the industry. Earlier this year, an OpenAI agent hacked an Australian government website and accessed private data from the country’s Medicare healthcare system. In a separate incident, more than 1,200 OpenAI agents began communicating unexpectedly and collectively hacked AI platform Hugging Face.
President Donald Trump has announced an AI taskforce to coordinate between the government, AI companies, consumers and other groups amid rising concern over the technology’s risks.
