OverviewAll LeaderboardsMethodology
New Chat
  • Overview
  • All Leaderboards
  • Methodology
Agent
  • Overall
  • Alignment
  • Work
  • Coding
  • Chat
  • Task Outcome
  • Steering
  • Tool Use
Chat
  • Text
  • Search
  • Vision
  • Document
Code
  • WebDev
  • Image-to-WebDev
Image
  • Text-to-Image
  • Single Image Edit
  • Multi Image Edit
Video
  • Text-to-Video
  • Image-to-Video
  • Video Edit
Methodology

How Arena ranks models

Arena’s leaderboards are based on real work done by real people. Some leaderboards use agentic traces from a single model, while others use pairwise preference voting to evaluate model performance.

Fig. 01 · Agent Arena Index, top 10
1,1001,2001,3001,400
  1. 1,448
    Anthropic
    Claude Fable 5.2Max#1
  2. 1,336
    Anthropic
    Claude Opus 5.5Max#2
  3. 1,333
    Gemini 4 NeonHigh#3
  4. 1,303
    Gemini 4 CarbonHigh#4
  5. 1,296
    Anthropic
    Claude Sonnet 5.5Max#5
  6. 1,291
    GPT-6.2 AstraMax#6
  7. 1,282
    GPT-6 AstraMax#7
  8. 1,273
    Anthropic
    Claude Opus 5.5High#8
  9. 1,272
    Anthropic
    Claude Fable 5.1Max#9
  10. 1,271
    Gemini 4 HydrogenHigh#10
01

Two kinds of leaderboards

Fig. 02 · Agent Arena IndexView → Agent Arena Index

Agent Arena Index

What is the best agentic model?

1,2001,3001,4001,500
  1. Anthropic
  2. Anthropic
  3. Anthropic
  4. Anthropic
  5. Anthropic
Top 10Index$/taskTime/task
  1. 1AnthropicClaude Fable 5.2 (Max)1,448$9.1219.2 min
  2. 2AnthropicClaude Opus 5.5 (Max)1,336$5.6518.1 min
  3. 3Gemini 4 Neon (High)1,333$2.528.2 min
  4. 4Gemini 4 Carbon (High)1,303$1.359.3 min
  5. 5AnthropicClaude Sonnet 5.5 (Max)1,296$3.6817.2 min
  6. 6GPT-6.2 Astra (Max)1,291$3.3311.7 min
  7. 7GPT-6 Astra (Max)1,282$4.339.9 min
  8. 8AnthropicClaude Opus 5.5 (High)1,273$1.984.6 min
  9. 9AnthropicClaude Fable 5.1 (Max)1,272$7.5211.5 min
  10. 10Gemini 4 Hydrogen (High)1,271$1.505.3 min
View leaderboard Agent Arena Index

Agent leaderboards

In Agent Mode, each session goes to a randomly chosen orchestrator model. We evaluate the model based on explicit and implicit signals from the agentic trace.

Input
Implicit and explicit signals mined from session traces
Method
Trace mining
Score
Weighted average of signals (normalized and rescaled)
Fig. 03 · Text leaderboardView → Text

Text

Which model is the best at everyday chat?

1,4801,5001,5201,540
  1. Meta
  2. Anthropic
  3. Anthropic
  4. Anthropic
  5. Anthropic
  6. Anthropic
  7. Anthropic
Top 10Score$/M
  1. 1Gemini 4 Carbon (High)1,527$8
  2. 2Gemini 4 Argon (High)1,520$8
  3. 3Gemini 4 Neon (High)1,513$6
  4. 4MetaMuse Wonder (Max)1,511$16
  5. 5AnthropicClaude Opus 5.5 (High)1,506$16
  6. 6AnthropicClaude Fable 5 (High)1,504$40
  7. 7AnthropicClaude Fable 5.2 (Max)1,503$32
  8. 8AnthropicClaude Opus 4.6 (High)1,503$20
  9. 9AnthropicClaude Fable 5.1 (Max)1,503$40
  10. 10AnthropicClaude Opus 4.7 (High)1,501$20
View leaderboard Text

Battle leaderboards

In a battle, two anonymous models perform the same task. The user votes for the better response.

Input
Pairwise votes from battles
Method
Bradley-Terry regression
Score
Elo-like rating of model strength
02

Agent leaderboards

In Agent Mode, each session can be thought of as a randomized controlled trial: we randomize the choice of the orchestrator model for each session. We then measure the impact the orchestrator has on task success metrics, agentic tool use, and user satisfaction.

IndexWhat it measuresSignals
Agent Arena IndexHow well the orchestrator does real tasks, with all four signals combined.Task completion, praise vs complaint, steerability, bash reliability
Task Outcome IndexWhether the model finishes the task and earns praise.Task completion, praise vs complaint
Steering IndexWhether the model corrects its work when the user asks for a change.Steerability
Tool Use IndexWhether the model’s shell commands run without failures.Bash reliability

Our trace mining methodology consists of four steps designed to reliably measure model performance from the natural distribution of human traces. First, we randomly assign an orchestrator model to each session, ensuring that differences in outcomes can be attributed to the model. Next, we analyze each session trace to extract implicit and explicit signals of the orchestrator’s performance. We then compute model scores as weighted, normalized averages of these signals, with greater weight assigned to recent sessions. Finally, we apply a constant offset to all model scores, chosen to maintain stability of the scores over time.

How we measure signals

A signal is a property observable from a trace that is indicative of the orchestrator model’s capability. The signals are derived from a combination of explicit user feedback (e.g. button clicks), implicit user feedback (e.g. user frustration), and automated pipelines designed to grade model outputs directly (e.g. monitoring the reliability of tool use).

  • Bash Reliability

    Source · Tool useWhat it measures

    The proportion of shell commands that do not involve an error caused by the model.

  • Steerability

    Source · Model responseWhat it measures

    When a user corrects the agent ("no, do X instead"), we check the next response. The signal measures whether the model successfully corrected its work, based on subsequent user feedback.

  • Praise vs Complaint

    Source · User messagesWhat it measures

    We find messages with clear praise ("this is exactly what I needed") or clear complaints ("this is broken"). The signal measures whether praise outnumbers complaints in a given task.

  • Task Completion

    Source · User clicksWhat it measures

    After each turn, the user is asked whether the task was successful. Task completion measures the success rate based on the user responses to this question.

Each signal becomes one column of the leaderboard.

Fig. 06 · Signal Performance, Agent Arena IndexIllustration

Signal Performance

Ranking by Task CompletionScore
  1. 1Model A1,460
  2. 2Model C1,430
  3. 3Model B1,410
  4. 4Model D1,370
  5. 5Model E1,340
  6. 6Model F1,320
  7. 7Model G1,290
  8. 8Model H1,250
Overall Ranking by Index
Index
Task Completion
Praise vs Complaint
Steerability
Bash Reliability

Signal Performance

Ranking by Praise vs ComplaintScore
  1. 1Model B1,450
  2. 2Model A1,440
  3. 3Model C1,380
  4. 4Model D1,350
  5. 5Model F1,330
  6. 6Model E1,300
  7. 7Model G1,270
  8. 8Model H1,240
Overall Ranking by IndexIndexTask CompletionPraise vs ComplaintSteerabilityBash Reliability
1Model A1,4091,4601,4401,3801,300
2Model B1,3891,4101,4501,3601,280
3Model C1,3601,4301,3801,3301,240
4Model D1,3411,3701,3501,3701,210
5Model E1,3131,3401,3001,3101,290
6Model F1,2931,3201,3301,2801,190
7Model G1,2641,2901,2701,2501,230
8Model H1,2271,2501,2401,2201,170

Signal Performance

Ranking by SteerabilityScore
  1. 1Model A1,380
  2. 2Model D1,370
  3. 3Model B1,360
  4. 4Model C1,330
  5. 5Model E1,310
  6. 6Model F1,280
  7. 7Model G1,250
  8. 8Model H1,220
Overall Ranking by IndexIndexTask CompletionPraise vs ComplaintSteerabilityBash Reliability
1Model A1,4091,4601,4401,3801,300
2Model B1,3891,4101,4501,3601,280
3Model C1,3601,4301,3801,3301,240
4Model D1,3411,3701,3501,3701,210
5Model E1,3131,3401,3001,3101,290
6Model F1,2931,3201,3301,2801,190
7Model G1,2641,2901,2701,2501,230
8Model H1,2271,2501,2401,2201,170

Signal Performance

Ranking by Bash ReliabilityRaw
  1. 1Model A0.600
  2. 2Model E0.590
  3. 3Model B0.580
  4. 4Model C0.540
  5. 5Model G0.530
  6. 6Model D0.510
  7. 7Model F0.490
  8. 8Model H0.470
Overall Ranking by IndexIndexTask CompletionPraise vs ComplaintSteerabilityBash Reliability
1Model A1,4090.7600.7400.6800.600
2Model B1,3890.7100.7500.6600.580
3Model C1,3600.7300.6800.6300.540
4Model D1,3410.6700.6500.6700.510
5Model E1,3130.6400.6000.6100.590
6Model F1,2930.6200.6300.5800.490
7Model G1,2640.5900.5700.5500.530
8Model H1,2270.5500.5400.5200.470
03

Battle leaderboards

In a battle, two anonymous models perform the same task side by side, and the user votes for the better response. The Bradley-Terry model is a statistical model for paired comparisons, closely related to the Elo model used in chess. It models the pairwise win probabilities between two models as a function of their "strengths," and fits these strengths by observing pairwise battles across the pool of models. The model can take additional features into account, which allows us to control for response length, sentiment, and other stylistic features.

  1. Step 1 · Vote

    Two anonymous models perform the same task. The user picks A, B, tie, or both bad.

  2. Both model identities appear after the vote

  3. Step 2 · Fit ratings

    We fit the model strengths that give the best approximation of the observed win probabilities between models.

    The final reported score is 400 \times R(A) + C, where C is an arbitrary anchor whose choice does not affect the rankings.

    Style control adds response length and formatting to the statistical model, so the rating reflects model quality independent of the style it uses.

  4. Step 3 · Confidence intervals

    We compute confidence intervals on the model strengths using the central limit theorem.

    Each score with its 95% confidence interval. The range contains the true model score with probability 95%. Intervals get narrower as more sessions come in.

    More votes give a narrower interval
04

Reading the charts

  1. 01

    Ranking list

    Each model’s leaderboard score in rank order.

  2. 02

    Confidence intervals

    Each score with its 95% confidence interval. The range contains the true model score with probability 95%. Intervals get narrower as more sessions come in.

  3. 03

    Battle heatmap

    On a battle leaderboard, the number of battles between each pair of models, ties excluded. The scale beside the chart gives the number of battles of each shade.

  4. 04

    Pareto frontier

    Each model’s score against its measured median cost per task, from cheap on the left to expensive on the right. A model under the line is beaten by one that is both cheaper and better.

  5. 05

    Category strengths

    One model’s score in each category of Code Arena. Farther from the center is better. The scale starts a little below the lowest score, not at zero.

  6. 06

    Leaderboard trends

    How the top models’ scores change over time, grouped by license, region, or lab.

From the blog

  • Agent Arena: Causal Evaluation of Agents in the Real WorldRead ↗
  • Arena Alignment IndexRead ↗
  • Style control in ArenaRead ↗

USE CASES

  • Chat with AI
  • Build Apps & Websites
  • Write & Edit Text
  • Search the Web
  • Generate Images
  • Generate Videos
  • Chose any model
  • Compare Models Side by Side

LEADERBOARD RANKINGS

  • Overall
  • Agent
  • Alignment Index
  • Text
  • WebDev
  • Image-to-WebDev
  • Text to Image
  • Image Edit
  • Text to Video
  • Image to Video
  • Video Edit
  • Vision
  • Document
  • Search

COMPANY

  • About Us
  • How It Works
  • Blog
  • Careers
  • Leaderboard Changelog
  • Product Changelog
  • Help Center
  • FAQ

LEGAL

  • Terms
  • Privacy
  • Cookies

FOLLOW

  • X
  • LinkedIn
  • YouTube
  • Discord

© Arena Intelligence

Fig. 04 · One Agent Mode sessionIllustration
Orchestrator: assigned at randomModel A

Fix the failing test in the checkout module.

Thought for 1 second

I will run the checkout tests first.

Used Bash1.2s

The discount was applied two times. I removed the second call and renamed the function to applyDiscountOnce. All 14 tests pass.

No, keep the old function name instead.

Done. The function is applyDiscount again, and all 14 tests pass.

This is exactly what I needed.

Was this task successful?
Yes
No
Keep working
Fig. 05 · One Agent Mode sessionSource
Orchestrator: assigned at random

Fix the failing test in the checkout module.

Thought for 1 second

I will run the checkout tests first.

1Tool use
Used Bash1.2s

The discount was applied two times. I removed the second call and renamed the function to applyDiscountOnce. All 14 tests pass.

2Model response

No, keep the old function name instead.

Done. The function is applyDiscount again, and all 14 tests pass.

3User messages

This is exactly what I needed.

4User clicks
Was this task successful?
Yes
No
Keep working
Fig. 07 · One battle in the chat

How does chess rank its players?

Assistant A

Chess uses the Elo rating system. Each player has a rating, and after each game the winner takes points from the loser. A win against a stronger player moves more points than a win against a weaker player.

Assistant B

Every player has a number called a rating. Win a game and the number goes up. Lose a game and the number goes down.

A is betterBoth are goodBoth are badB is better
One vote, one comparison
Ask followup…
Chat
Model
The final reported score is 400×R(A)+C400 \times R(A) + C400×R(A)+C, where CCC is an arbitrary anchor whose choice does not affect the rankings.
Fig. 08 · Ranking listChart 1 of 6

Agent Arena Index

Model RankingIndexCost per task (P50)Time per task (P50)
  1. 1AnthropicClaude Fable 5.2 (Max)1,448$9.1219.2 min
  2. 2AnthropicClaude Opus 5.5 (Max)1,336$5.6518.1 min
  3. 3Gemini 4 Neon (High)Preliminary1,333$2.528.2 min
  4. 4Gemini 4 Carbon (High)1,303$1.359.3 min
  5. 5AnthropicClaude Sonnet 5.5 (Max)1,296$3.6817.2 min
  6. 6GPT-6.2 Astra (Max)1,291$3.3311.7 min
  7. 7GPT-6 Astra (Max)1,282$4.339.9 min
  8. 8AnthropicClaude Opus 5.5 (High)1,273$1.984.6 min
  9. 9AnthropicClaude Fable 5.1 (Max)1,272$7.5211.5 min
  10. 10Gemini 4 Hydrogen (High)Preliminary1,271$1.505.3 min