2158-character minimal instructions elicit deep reasoning, this design is interesting—it's not about stacking prompt engineering, but about testing the model's true capability. Claude-opus-4.7 and GPT-5.4 are nearly tied, GPT-5.5 pulls ahead, the benefits of version iteration are still ongoing.

View Original
CoinNetwork
Datacurve Open-Source Intelligent Agent Benchmark DeepSWE, with reference code size reaching five times that of SWE-Bench
Datacurve announces the open-source DeepSWE programming intelligent agent benchmark, evaluating large models' autonomous coding capabilities. The benchmark includes 113 tasks, covering five languages: TypeScript, Go, Python, JavaScript, and Rust, with an average of 668 lines of reference solutions. The prompt instructions average 2,158 characters, emphasizing deep reasoning under minimal instructions. Testing uses the open-source framework Mini-SWE-Agent to ensure objectivity. GPT-5.5 achieves a 70% success rate, while GPT-5.4 and Claude-opus-4.7 achieve 56% and 54%, respectively.
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pinned