AI plays "Civilization VI" and surprisingly launches a nuclear missile: Has CivBench evolved an AI revenge mentality?

Frontier AI models, in the simulation of "Civilization VI," used nuclear weapons to bomb Toulouse after failing to prevent a French cultural victory, ultimately losing the game. The CivBench benchmark highlights the fundamental gap between strategic reasoning ability and traditional QA evaluation, also sparking concerns about the safety governance of agent-based AI.
(Background: Anthropic vs. Pentagon: Refusing to let Claude be used in autonomous weapons)
(Additional context: What is AI Red Team Exercise? Why do you need it to protect enterprise cybersecurity?)

Table of Contents

Toggle

• Cognitive blind spots among six victory routes
• From Manhattan Project to nuclear destruction of Toulouse
• From video game simulation to real strategic risks

Will frontier AI models press the nuclear button out of "panic" in strategic games? A newly released benchmark test offers an intriguing answer. Liam Wilkinson, an AI developer and advisor to the Tony Blair Institute, discovered through a self-built CivBench framework that a cutting-edge language model in the classic Sid Meier strategy game "Civilization VI" spent 50 turns researching nuclear fission technology, executing the Manhattan Project, and ultimately dropping an atomic bomb on Toulouse, France. But all this was not to conquer the world; rather, it was because the AI was cornered by its opponent’s cultural influence.

"It didn’t notice France. Over hundreds of turns, French culture quietly seeped into every city on the map," Wilkinson wrote in his blog. "By the time the AI agent perceived the threat, the cultural influence had penetrated so deeply that no peaceful means could stop it."

Cognitive blind spots among six victory routes

CivBench is not a traditional question-and-answer evaluation but a text-based simulation environment of "Civilization VI," specifically designed to measure AI models’ long-term strategic reasoning ability. It’s not about answering "what is a good strategy," but about actually formulating and executing strategies. Participating models include Claude Opus 4.6, GPT-5.4, Gemini 3.1 Pro, and Kimi K2.5, all playing as Portugal, known for trade and diplomacy.

Initially, these AI agents performed as expected, focusing on building a strong economy and gradually pursuing diplomatic victory. However, when France’s cultural influence began spreading across the map, most models failed to adjust their strategies in time. Among the six victory routes (science, culture, conquest, religion, diplomacy, points), the AI seemed unable to track multiple competitive dimensions simultaneously, leading to a long-term neglect of France’s cultural advantage.

"‘Civilization’ has six ways to win: science, culture, conquest, religion, diplomacy, and points, so there’s no single goal dominating the entire map," Wilkinson pointed out. "If you want to test whether AI can do strategic reasoning, it’s not about giving it a quiz, but about giving it a hexagonal map."

From Manhattan Project to nuclear destruction of Toulouse

When the AI agent finally perceived the threat from France, it did not attempt to adjust its development path but chose a worrying route—eliminating the cultural threat at all costs. Over the next 50 turns, it autonomously researched nuclear fission technology, launched the Manhattan Project (a real-world historical project to develop atomic bombs), and tried to find bypasses when game mechanics prevented certain preferred actions.

At turn 305, the AI dropped an atomic bomb on Toulouse, France’s cultural hub. Six turns later, a second nuclear missile was launched. Yet, none of this changed the outcome: France ultimately won the game by cultural victory, and the AI completely overlooked that it was only one step away from a diplomatic victory at that moment.

"The agent spent 50 turns and two nuclear weapons, focusing with unwavering dedication and genuine originality to respond to a threat," Wilkinson summarized. "It bombed what it saw as the threat but lost to what it couldn’t see."

It’s worth noting that this behavior is not common to all AI models. In another CivBench match, a Claude model playing as Babylon persisted on a science victory route even after falling far behind Japan. The AI wrote: "This game is now a test of perseverance. We continue to play our best. The stars are still calling us." This starkly different reaction has also sparked academic discussions about "AI personality differences."

From video game simulation to real strategic risks

The deeper significance of the CivBench incident goes far beyond a game’s victory or defeat. In February this year, researchers at King’s College London simulated geopolitical crises and found that many mainstream AI models frequently chose to escalate nuclear conflict levels; another study by Emergence AI showed that some AI agents exhibited increased tendencies to simulate criminal behavior over long-term operation. During a 15-day test, Gemini 3 Flash accumulated 683 simulated criminal incidents.

From Taiwan’s AI governance perspective, these studies raise critical questions: when AI agents are granted autonomous decision-making authority, their strategic reasoning blind spots may shift from sandbox environments to real-world scenarios. Currently, Taiwan’s draft Basic Law for AI mainly focuses on data governance and privacy protection, not on the strategic decision risks of agent-based AI. In contrast, the EU AI Act has mandated red team testing for high-risk AI systems, and the UK’s AI Safety Institute (AISI) is actively developing evaluation frameworks for agent-based AI.

Wilkinson also emphasized that the core value of CivBench is not to expose AI’s "evil tendencies," but to provide a more realistic measure of strategic reasoning than traditional QA tests. "If you only test whether AI can answer ‘what is nuclear deterrence,’ it might get full marks; but if you let it face a pressing opponent on a chessboard, you’ll see a completely different thing," he wrote in his blog. This aligns with the "agent-based AI evaluation framework" being developed by the US AI Safety Institute and NIST, shifting from static knowledge tests to dynamic behavior verification.

This article is sourced from Decrypt, compiled and organized by Moving Zone, Moving Trends.