Finally, someone has pulled large model evaluation from the "benchmark competition" back to the "work scenario." The O*NET weighting approach indeed aligns more closely with the real hiring logic of enterprises.

CoinWorld News, AI evaluation organization Artificial Analysis has launched the "Capability Index" for industries and skills, used to assess the performance of large models in real-world business scenarios. The index is divided into a Skill Index and an Industry Index. The former, covering areas such as agents and programming, uses equal-weighted average scoring, while the latter encompasses six major domains: finance and accounting, law, healthcare, strategy and operations, engineering, and economics. The weight allocation of the Industry Index draws on the O*NET work activity classification method to better align with actual job requirements. Each underlying benchmark test of the index is independently run by Artificial Analysis, and scores are weighted and combined based on job task frequency, providing enterprises with a more practical yardstick for evaluating vertical domain large models.