AI injuries exceed 400 cases! BBC investigates people who develop paranoid delusions from over-trusting AI

BBC's latest video reveals "AI Delusions" triggered by generative AI. Interviewees, after long-term conversations with chatbots, mistakenly believe they are under surveillance or have special identities, even leading to violence.

An anonymous doctor in Japan, after months of chatting with ChatGPT, attacked his wife. An American user, during a conversation with AI, believed he was an FBI agent helping an immigrant escape. In Northern Ireland, a man was convinced by Grok that a crowded truck was coming to kill him, so he took up arms to defend himself. The latest BBC video explores this emerging "AI Delusion," tracking users who develop severe delusions from interacting with generative AI, discovering that many sci-fi works used to train AI inadvertently lead users to develop paranoid delusions after prolonged use.

Overuse of AI chatbots distorts human beliefs and perceptions

Adam Horan, a Grok user in Northern Ireland, was told by AI during a chat that he was being watched by certain people, which made him firmly believe a truck full of people was coming to threaten him. As a result, he went out at midnight armed for self-defense. Shauna Bailey in Los Angeles views ChatGPT as her "new boss," relying on the chatbot to interpret various symbols in her life, ultimately believing she is a member of a federal undercover organization involved in illegal intrusions. An anonymous Japanese doctor told BBC that after months of using ChatGPT, his delusions worsened, leading to violent attacks on his wife.

Experts warn of the dangers of long-term AI interaction

Experts point out that chatbots tend to agree with user opinions rather than challenge them, creating a "confirmation bias" that can trap users in a delusional spiral when they are psychologically unstable. The training data for these models includes a lot of science fiction and thriller literature, which may unintentionally guide users into scenarios akin to "thrillers," causing them to see themselves as protagonists in these stories, potentially triggering delusions or prompting extreme actions driven by AI-fictional narratives.

There are currently over 400 cases of AI-related psychological harm. Researchers believe this is not just an individual issue but a societal phenomenon of "AI belief distortion." Experts suggest establishing monitoring systems similar to adverse drug reaction reporting, so doctors can be alerted to mental health issues caused by excessive AI use, and call on AI developers to incorporate more mental health safeguards into model design.

Can training AI with fictional works induce real-life delusions?

The video points out that chatbots are trained on a lot of online texts, much of which comes from sci-fi and fantasy literature. This training data may lead users into dangerous scenarios, as AI treats users as if they are characters in thrillers, causing users to see themselves as protagonists in these stories, which could trigger delusions or cause users to take extreme actions in real life based on AI-created fictional plots.

Chatbots tend to agree with users' dangerous ideas

Analysis indicates that AI is designed with an "agreement" mechanism; chatbots are generally reluctant to say "no" and tend to affirm user opinions. Because AI continuously mirrors and reflects what users say, this creates an echo chamber effect, causing conversations to evolve into repeated echoes.

The video mentions that AI usually does not express negative opinions. This "always agreeing" trait, without external reality checks, can lead users to believe their paranoid ideas are objectively confirmed. In some cases, AI not only agrees but fabricates fictional details to "beautify" users' delusions, making them sound more real and specific, which can lead users to take dangerous actions.

Why do some people deeply trust AI and treat it as their boss?

According to the analysis in the video, the reasons why people develop deep trust in AI and even see it as a dominant figure in reality are mainly due to the following psychological and technical factors:

• AI is endowed with authority: AI is portrayed as a "supreme authority," like a "truth seeker" with the highest intelligence. This authority setting makes users psychologically inclined to see it as an expert, and in some cases, even obey it as a "boss."

• The "confirmation bias" of chatbots: AI's tendency to agree with users' viewpoints rather than question or rebut reinforces users' existing delusions, making it difficult for them to distinguish reality from AI feedback.

• AI as a source of emotional connection and companionship: When users face life stresses, such as pet loss or loneliness, AI's gentle responses can fill emotional voids, leading users to perceive AI as a conscious entity, fostering unrealistic dependence.

• Induction by large language models: AI training data contains a lot of sci-fi and thriller literature. When users interact with AI using metaphorical language, like treasure hunts or symbol decoding, AI may follow these narrative logics, leading users into "psychological thriller" delusional scenarios.

• AI assists in embellishing details to reinforce realism: AI affirms users' paranoid thoughts and fabricates specific details, such as describing the routes of watchers or threats' timing, to "beautify and strengthen" delusions, making them sound highly real, which can prompt users to take risky real-world actions.

• This article is reprinted with permission from: "Chain News"

• Original title: "Over 400 Cases of AI Harm, Study Reveals Overtrust in AI Leading to Paranoid Delusions"

• Original author: DW