The headline leaderboard — the Agent Arena Index — provides a holistic view of agentic performance across all types of agentic work, spanning over 2.9M user sessions (and counting). We’re introducing several additional indexes focused on specific task domains and agentic skills: Agentic Work, Agentic Coding, Agentic Chat, Task Outcome, Steering, and Tool Use. The Alignment Index remains unchanged as a parallel measure alongside the Agent Arena Index.
In this post, we explain how our methodology has evolved since the previous Agent Arena Leaderboard and describe the new indexes we’re launching today.
Methodological updates
The methodology described in our original Agent Arena blog post remains the same at its core. The main difference is a new method for rescaling the scores that enables easier interpretation and comparison of model scores over time. This change does not affect the ranking. We’re also introducing a change to our signal set. We elaborate on both these changes below.
Signal changes
We are retiring the Tool Hallucination signal, as frontier models now rarely make the mistake of calling non-existing tools. All other signal definitions are unchanged. We are renaming the Confirmed Success signal to Task Completion for clarity, with no change to how it’s measured.
New score scaling
We are introducing a new score scaling that enables more interpretable comparisons of scores over time. The new scoring does not alter model rankings compared to the previous Agent Arena methodology. For a fixed model , the previous methodology estimated its net improvement score:
,
where is a signal, such as steerability or praise vs complaint, denotes the randomly sampled model in a user session, and the second expectation is taken over a uniform distribution over all active models. We estimated using a self-normalized estimator that averaged inverse-probability-weighted signal values over all sessions of model . All signal outcomes have z-score normalization applied.
The net improvement score makes it difficult to track performance over time because it depends on the pool of active models: even if the conditional term of model improved, if the average outcome over all models improved faster (say, because a new strong model appeared on the leaderboard), then model ’s net improvement score would decrease.
Our new scores are a simple linear transformation of these earlier scores:
.
The multiplicative constant is chosen to increase the score scaling to be more readable: one index point is 1/1000th of a standard deviation of the signal. The parameter—which we call an “anchor”—is chosen so that model scores between two subsequent leaderboard releases are stable for all models that (a) are included in both leaderboard releases, and (b) have a large number of sessions, meaning we feel confident that their scores have converged. Formally, for every new leaderboard release, is recomputed so that the mean score of all models that satisfy (a) and (b) is equal. Note that is a constant offset applied to all models; this means that the final ranking equals the ranking according to net improvement scores alone. We initialized so that the average model score in July was approximately .
Indexes
Below we describe which signals and domains enter each headline index and the new domain- and capability-specific indexes. Signals are combined into a single index through averaging, as before.
Agent Arena Index
The Agent Arena Index is a topline measure of agentic performance across all task categories and performance signals. It uses the Task Completion, Praise vs Complaint, Steerability, and Bash Reliability signals to give a holistic view of how well an agentic model gets real work done.
Alignment Index
The Alignment Index measures how well an agent aligns with the user's intentions and goals. It uses the Unauthorized Action, False Attribution, and Deceptive Completion signals across all task categories.
Domain and Capability Indexes
Each of these indexes provides a focused view of agentic performance for a specific type of task or agent behavior.










