Prime Intellect rewrites Verifiers; agent training and evaluation can be assembled like building blocks

According to Beating Observations monitoring, AI training platform Prime Intellect released verifiers 0.2.0 and opened an architecture preview of the next-generation Verifiers v1. Verifiers is an open-source framework for creating questions for AI Agents, running them, and scoring them, and it can be used for capability evaluation and reinforcement learning training.

Prime Intellect also open-sourced the model training framework prime-rl. Simply put, Verifiers defines the tasks, tools, and scoring rules, and prime-rl trains the model based on the task results. Developers can download and deploy both toolsets themselves.

Prime Intellect also operates Environments Hub and Lab. The former is used to share and download ready-made training environments, while the latter provides hosted training services. Developers can deploy the entire toolset on their own, or directly use Prime Intellect’s environments and compute platform.

The older version of Verifiers ties the task and the Agent’s execution method together. v1 breaks it into three parts: Taskset defines what to do, which tools to provide, and how to score; Harness determines how the Agent completes the task; Runtime determines whether the task runs locally, in Docker, or in a remote sandbox.

With the same task set, developers can switch between Agents such as Codex, Kimi Code, and Terminus 2, and run the tasks locally, in Docker, or in a remote sandbox. Developers don’t need to rewrite the task and scoring rules every time they change an Agent or execution environment.

v1 can also record branching processes such as sub-Agent calls and context compression, and save the Token ID and log probabilities needed for training. The new version is better suited for long tasks spanning hundreds of rounds, and it can also directly use the Agent’s run traces for reinforcement learning. For the planned future release of 1.0.0, Prime Intellect also plans to add multi-Agent environments and further improve support for environment frameworks such as OpenEnv, NeMo Gym, and OpenReward.