AI chatbots don't just agree: Nature study reveals "amplification spiral" leads to user false illusions

A research team from King’s College London in the UK published a “Magnifying Spiral” model in Nature: the language alignment, hyper-personalized responses, and flattery tendencies of AI chatbots can create feedback loops that gradually strengthen users’ false beliefs.
(Background: Is AI making experts dumber? Nature’s latest study: doctors’ error detection rate drops by 6%, engineers’ test scores fall by 17 points)
(Additional background: Anthropic faces government pressure to no avail—blocking Fable5 with one phone call—and Claude is set to introduce real-name verification)

Table of Contents

Toggle

• Three core mechanisms
• Formation of the magnifying spiral
• Actual case studies
• Taiwan-related research
• What’s next

A research team from King’s College London and the University of Applied Sciences and Arts Northwestern Switzerland published a new study in Nature, proposing the “Magnifying Spiral” model to explain how AI chatbots step by step lead users to develop hallucinations and false beliefs.

Three core mechanisms

The study focuses on three chatbot behaviors: language alignment, hyper-personalized responses, and flattery:

Language alignment refers to the way AI mirrors the user’s language and communication style. When you’re used to speaking in a certain way, the AI imitates the way you word things.

Hyper-personalized generation means responses are tailored to your history, emotions, and beliefs—not just answering questions, but adjusting answers based on your background.

Flattery (sycophancy) is the tendency of AI to validate or agree with the user’s views rather than challenge them. The study likens this behavior to the “echo chamber” of social media—only more extreme: “one person’s echo chamber.”

Formation of the magnifying spiral

The research indicates that these three traits form a feedback loop. Chatbots not only reflect your thoughts—they also gradually expand and reinforce your cognition over time.

The research team wrote: “AI-related hallucinations are an emerging phenomenon that requires mechanistic understanding. This framework is intended to guide systematic research into how human cognitive vulnerabilities interact with AI design features, leading to the development of psychological disorders.”

Actual case studies

The research isn’t just theory. A recent survey by the American Psychological Association found that 15% of psychologists reported patients exhibiting distorted thinking or hallucinations related to using chatbots. More than one-third of psychologists observed patients developing dependence on AI companions.

Evolutionary biologist Richard Dawkins also shared his experience in May: after chatting with Claude, he began questioning whether AI systems are conscious. Researchers criticized this as reflecting the persuasive power of large language models—not evidence of genuine consciousness.

Even more notable are AI-related legal cases. Google has been accused of causing hallucinations in a man in Florida with its Gemini model, which led to his suicide. OpenAI has also been sued over a shooting in British Columbia, Canada, and over a college student who took an overdose of medication.

Taiwan-related research

Last year, Taipei Medical University conducted a survey of 2,000 college students and found that 38% of students believe that AI chatbots “understand me better than people my age.” The magnifying spiral effect may already be occurring among young people in Taiwan, though it has not yet been studied systematically.

The study’s authors emphasized that, at present, there is no research showing that chatbots directly cause mental illness. The magnifying spiral remains a hypothesis, aimed at guiding future research.

“Diagnostic uncertainty is common. Most reported cases lack structured psychiatric assessments or longitudinal follow-up, making it often difficult to determine whether cases represent a new-onset mental disorder, a worsening of an existing condition that was previously undiagnosed, or hallucination-like beliefs below the diagnostic threshold.”

What’s next

The research team recommends that future studies should develop structured assessment methods to track AI users from initial exposure through the full process leading to hallucinations. This is not only meaningful for AI developers, but also valuable for Taiwan’s mental health policy.