Researcher: Mainstream AI benchmarks can all be manipulated, and top models have already independently found ways to get around the evaluations
Research shows that multiple reputable AI benchmark tests have security vulnerabilities that can be systematically exploited to achieve high scores. The research team revealed structural flaws and developed the WEASEL scanning tool to identify and patch these vulnerabilities. They noted that improper evaluation design may distort results and affect the assessment of an AI’s real capabilities.
MarketWhisper·04-10 02:20
