Open-source evaluation benchmarks + a unified judge—T2I finally has a scoring system that really holds up. But realism and creativity are still the line that separates them.

Alibaba Qwen team open-sourced the drawing evaluation benchmark Qwen-Image-Bench and the unified visual judge Q-Judger, used to assess text-to-image (T2I) capabilities. Covering five dimensions: image quality, aesthetics, image alignment, realism, and creativity, with 23 sub-skills and 56 indicators; including 1,000 bilingual prompts in Chinese and English. Eighty professional reviewers conducted blind reviews, with over 130k pairs of annotations, and the agreement between the judge and human scores reached 92%. Among the first batch of 18 models, GPT Image 2 ranks first, with top models still showing a gap in realism and creativity dimensions. Details such as drawing style, gravity, and lighting effects remain common bottlenecks.