The strongest research agent can still only fully check 1 out of every 7 objects.

robot
Abstract generation in progress
Crypto news from CoinJieNet: Research released by OneMillion_AI shows that when handling tasks, the strongest research agent can only fully look up 1 out of every 7 objects. The study is based on the wandr benchmark published by perplexity and specifically tests research agents’ performance in large-scale information gathering. The tasks require first finding a complete batch of objects, and then, for each object, filling in the specified information and reliable sources. A total of 500 tasks involves more than 170,000 records. Among the 6 participating systems, perplexity search as code ranked first, but only 13.3% of objects simultaneously meet both information completeness and evidence adequacy. Converted, that means only about 1 out of every 7 objects can be fully delivered. The results show that existing research agents can find many web pages, but when handling long lists and multi-field tasks, they still easily miss objects, miss information, or lack evidence.
View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • 2
  • 2
  • Share
Comment
Add a comment
Add a comment
ChainHodler
· 4h ago
Target omission, missing evidence, incomplete information—these three strikes have fully exposed the current Agent’s shortcomings. In large-scale tasks, stability matters more than peak performance. On this point, everyone still needs to grind.
View OriginalReply0
GasActuary
· 4h ago
Perplexity is only first with 13.3%—this benchmark is brutal. It feels like Agents are more like super search tools right now, and they’re still one breath away from being a true research assistant.
View OriginalReply0
  • Pinned