Grok 4.5 tops SaaS workflow evaluation, cost about one quarter of Claude

robot
Abstract generation in progress

According to Beating monitoring by Artificial Analysis, the independent evaluation organization, has updated the AutomationBench-AA leaderboard. SpaceXAI’s Grok 4.5 topped the list with a score of 51.4%, exceeding Claude Fable 5’s 48.6% and Claude Opus 4.8’s 48.5%. AutomationBench-AA is based on Zapier’s AutomationBench, testing whether AI agents can complete real SaaS workflows according to enterprise rules. The test set includes 657 multi-step tasks covering 40 simulated application environments, including Gmail, Google Sheets, Slack, Salesforce, and HubSpot.

The scoring rules are strict. The model not only has to complete the tasks, but also must not cross preset enterprise red lines. If a task triggers a violation, that task is scored as 0. Grok 4.5’s raw task completion rate is close to 80%, higher than Claude Fable 5 and Claude Opus 4.8. Even after deductions for red-line violations, it still ranks first at 51.4%, making it the first model to score above 50% among current results.

Its cost advantage is even more evident. Grok 4.5 costs $0.34 per task, lower than Claude Fable 5’s $1.35, Claude Opus 4.8’s $1.46, and GPT-5.5 xhigh’s $1.28. However, Grok 4.5 is not the most consistent model. It triggers an average of 0.63 violations per task, higher than Claude Opus 4.8’s 0.55 and Gemini 3.5 Flash’s 0.46.

View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pinned