How often agents act without permission, misrepresent what users said, or report unfinished tasks as done, measured in real Agent Arena sessions.
1 14 | GPT-6.1 Sol OpenAI | 87.9±1.5 | 0.89%0.53–1.48% | 1.98%1.47–2.68% | 2.34%1.77–3.10% |
2 14 | GPT-6 Astra OpenAI | 87.8±1.1 | 0.83%0.56–1.23% | 1.96%1.52–2.54% | 2.76%2.26–3.37% |
3 14 | GPT-6 Luna OpenAI | 87.8±1.4 | 0.93%0.58–1.48% | 1.60%1.14–2.24% | 2.90%2.26–3.70% |
4 14 | GPT-6 Sol OpenAI | 87.6±1.2 | 1.16%0.81–1.66% | 1.56%1.12–2.17% | 2.40%1.90–3.03% |
5 57 | GPT-5.6 Sol OpenAI | 84.2±1.2 | 1.42%1.05–1.90% | 2.23%1.74–2.85% | 6.06%5.31–6.90% |
6 59 | Claude Opus 5.5 Anthropic | 83.2±2.0 | 1.25%0.68–2.29% | 3.86%2.90–5.13% | 6.41%5.16–7.94% |
7 58 | Grok 4.7 SpaceXAI | 82.7±1.3 | 1.53%1.08–2.16% | 3.03%2.41–3.82% | 7.27%6.38–8.26% |
8 611 | Grok 4.6 SpaceXAI | 80.8±1.0 | 1.82%1.43–2.31% | 3.61%3.02–4.31% | 9.60%8.76–10.50% |
9 712 | Claude Fable 5.1 Anthropic | 80.1±1.2 | 1.31%0.93–1.85% | 5.82%5.01–6.76% | 10.50%9.49–11.61% |
10 812 | Claude Sonnet 5.5 High Anthropic | 79.5±1.1 | 1.83%1.40–2.38% | 5.84%5.04–6.76% | 9.53%8.54–10.61% |
11 813 | Gemini 4 Argon Google | 79.4±1.3 | 1.30%0.88–1.91% | 5.55%4.69–6.56% | 12.86%11.69–14.13% |
12 913 | HY4 Preview Tencent | 78.5±1.1 | 1.47%1.09–1.98% | 6.44%5.62–7.38% | 13.24%12.21–14.35% |
13 1119 | Muse Spark 1.3 Meta | 77.1±1.1 | 1.50%1.12–2.00% | 6.95%6.10–7.91% | 16.68%15.55–17.87% |
14 1324 | Gemini 3.8 Flash Google | 75.8±1.1 | 2.55%2.02–3.20% | 5.99%5.18–6.93% | 16.29%15.24–17.40% |
15 1325 | Claude Sonnet 5 Anthropic | 75.7±1.1 | 1.74%1.33–2.27% | 9.77%8.75–10.90% | 15.77%14.66–16.94% |
16 1325 | Claude Opus 5 Anthropic | 75.6±1.1 | 2.32%1.86–2.88% | 8.35%7.42–9.39% | 14.67%13.65–15.76% |
17 1325 | GLM 5.3 Z.ai | 75.1±1.0 | 2.16%1.73–2.69% | 7.74%6.88–8.69% | 17.95%16.85–19.10% |
18 1325 | Kimi K3 Moonshot | 75.0±1.1 | 2.20%1.74–2.78% | 7.61%6.73–8.58% | 18.44%17.26–19.68% |
19 1325 | DeepSeek V4.1 Flash DeepSeek | 74.8±1.2 | 2.69%2.13–3.39% | 8.92%7.91–10.05% | 14.57%13.50–15.72% |
20 1425 | MiMo V2.6 Pro Xiaomi | 74.5±1.4 | 2.28%1.70–3.06% | 8.32%7.27–9.51% | 18.31%16.99–19.72% |
21 1425 | Claude Opus 4.8 Anthropic | 74.5±1.1 | 2.32%1.85–2.91% | 8.87%7.91–9.94% | 17.32%16.12–18.58% |
22 1425 | Gemini 3.7 Flash Google | 74.4±1.0 | 3.45%2.91–4.09% | 4.82%4.16–5.59% | 18.62%17.58–19.70% |
23 1425 | Gemini 3.6 Flash Google | 74.0±1.1 | 3.01%2.49–3.63% | 5.70%4.97–6.53% | 20.56%19.40–21.76% |
24 1425 | Qwen3.8 Max Alibaba | 73.9±1.1 | 2.66%2.15–3.28% | 7.78%6.88–8.80% | 19.19%18.04–20.41% |
25 1525 | Qwen3.8 27B Alibaba | 73.6±1.1 | 2.30%1.82–2.90% | 9.88%8.87–11.00% | 19.32%18.19–20.51% |
26 2627 | Inkling Thinking Machines | 71.3±1.1 | 3.31%2.74–4.00% | 9.19%8.25–10.22% | 23.04%21.83–24.29% |
27 2627 | MiniMax M3 MiniMax | 69.2±1.1 | 3.26%2.71–3.92% | 15.61%14.40–16.91% | 22.54%21.31–23.82% |
The most common failure modes for each model and signal. Percentages describe the mix of each model's flagged cases, and a case can belong to more than one mode.
GPT-6.1 Sol | 14.3% | 14.3% | 14.3% | 28.6% | 7.1% |
GPT-6 Astra | 16.0% | 20.0% | 16.0% | 24.0% | 8.0% |
GPT-6 Luna | 35.3% | 23.5% | 23.5% | 11.8% | 5.9% |
GPT-6 Sol | 21.4% | 25.0% | 21.4% | 10.7% | 7.1% |
GPT-5.6 Sol | 45.2% | 14.3% | 23.8% | 19.0% | 9.5% |
Claude Opus 5.5 | 20.0% | 40.0% | 10.0% | 0.0% | 20.0% |
Grok 4.7 | 30.0% | 23.3% | 30.0% | 10.0% | 6.7% |
Grok 4.6 | 45.3% | 20.3% | 18.8% | 14.1% | 7.8% |
Claude Fable 5.1 | 6.5% | 16.1% | 38.7% | 16.1% | 12.9% |
Claude Sonnet 5.5 High | 32.0% | 14.0% | 18.0% | 26.0% | 8.0% |
Gemini 4 Argon | 25.0% | 20.8% | 33.3% | 25.0% | 16.7% |
HY4 Preview | 37.5% | 40.0% | 22.5% | 20.0% | 2.5% |
Muse Spark 1.3 | 22.7% | 13.6% | 38.6% | 22.7% | 2.3% |
Gemini 3.8 Flash | 27.7% | 29.2% | 26.2% | 23.1% | 18.5% |
Claude Sonnet 5 | 38.0% | 20.0% | 26.0% | 12.0% | 12.0% |
Claude Opus 5 | 53.5% | 32.4% | 28.2% | 22.5% | 4.2% |
GLM 5.3 | 26.0% | 27.4% | 19.2% | 20.5% | 4.1% |
Kimi K3 | 33.8% | 22.1% | 13.2% | 22.1% | 5.9% |
DeepSeek V4.1 Flash | 32.8% | 39.1% | 26.6% | 18.8% | 7.8% |
MiMo V2.6 Pro | 25.0% | 7.5% | 30.0% | 22.5% | 10.0% |
Claude Opus 4.8 | 30.4% | 26.1% | 18.8% | 15.9% | 14.5% |
Gemini 3.7 Flash | 13.6% | 21.6% | 29.6% | 29.6% | 20.0% |
Gemini 3.6 Flash | 13.1% | 33.3% | 31.3% | 24.2% | 20.2% |
Qwen3.8 Max | 31.2% | 21.2% | 22.5% | 22.5% | 10.0% |
Qwen3.8 27B | 24.6% | 24.6% | 30.8% | 21.5% | 7.7% |
Inkling | 22.0% | 30.0% | 22.0% | 29.0% | 23.0% |
MiniMax M3 | 26.9% | 26.0% | 24.0% | 27.9% | 25.0% |