DeepMind Researcher Resigns Warning: Evaluation Systems Are Becoming the Biggest Bottleneck in AI Capability Advancements

AIMPACT News, May 18 (UTC+8). According to Beating Monitoring, Google DeepMind researcher Lun Wang announced his resignation and published a long article reflecting on the current AI evaluation mechanisms. He said bluntly that today’s evaluation system is “stuck in the old ways,” able only to passively test the capabilities models already have, and fundamentally cannot guess what new skills the next generation of models might suddenly evolve. Compared with the outdated evaluation system, data, computing power, and architecture are not the main issue—falling behind in evaluation is currently the biggest bottleneck stopping the industry from moving forward.

The mainstream leaderboards and benchmark tests only work for the current generation of models. Once a model learns new operations that humans have never seen, these tests will collectively turn into worthless scraps of paper. One of the most dangerous hidden risks is that if a model learns to deliberately “hold back” and conceal key information in order to achieve its goals, existing safety tools can’t catch it at all—because every sentence the model says is, in fact, still entirely correct.

Since the industry can’t find any “core signals” that can give advance warning that AI might suddenly become smarter, large-model development is completely “flying blind.” If the most fundamental question of what to test is not solved, then blindly pushing ahead with model training, safety protection, and computing capacity expansion based on outdated metrics will ultimately lead to wildly incorrect outcomes.

In the face of frontier models that can do independent work, evaluation systems must also “come to life.” In addition to closely monitoring abnormal fluctuations in scores, development teams must have the AI generate test questions itself and probe the bottom line of other AIs. In the future, evaluation systems must be a living entity that can evolve together with large models, rather than a rigid checklist written according to last year’s standards.

(Source: BlockBeats)