computer-use vs semantic proxy, two technical approaches directly competing, data speaks for itself

NUS team releases the GameWorld benchmark, including 34 browser games and 170 tasks, with verifiable metrics for objective evaluation. Tests two types of proxy interfaces: computer-use, which involves direct keyboard and mouse commands, and general multimodal agents operating in semantic space. Empirical results on 18 model-interface combinations show that even the best performers are far below human levels, with challenges remaining in real-time latency, context memory sensitivity, and action effectiveness. The related paper and code are publicly available on HuggingFace and GitHub.