BinEval's trick of breaking down evaluations into true/false questions is quite clever. The room for fabricating scores is directly compressed, and the illusion of being superficially fluent but factually wrong can finally be exposed.

BinEval transforms evaluation into true/false questions. After answering each question, it scores by accuracy, improving transparency and suppressing false reporting. Research shows that its scoring is close to or even surpasses Unieval across multiple datasets, and it is especially good at identifying answers that are superficially fluent but factually wrong. Taking an aircraft interception summary as an example, the prior evaluator gave a full score of 5.0, while BinEval scored 1.57 by answering seven true/false questions, close to the human score of 2.0. Feedback optimization improves format compliance by about 17 percentage points, but it still struggles to meet hard constraints such as word count.