Preprint · 2026
Wayne Chi1,2* I-Hung Hsu2 Li Chen2 Wei-Lin Chiang2 Ion Stoica2,3 Anastasios Nikolas Angelopoulos2 Ameet Talwalkar1 Chris Donahue1 Cho-Jui Hsieh2,4
1 Carnegie Mellon University 2 Arena 3 UC Berkeley 4 UCLA * Work done as part of an internship at Arena.
Datasets
Along with Interaction-as-a-Judge, we are open sourcing ArenaWebdevPreferences and ArenaWebdevCorrectness, two agentic datasets with real user interactions, preferences, and labels. We hope that by releasing this data we can help further advance research into coding agents and computer-use agents.
ArenaWebdevPreferences: 6,390 application pairs across 7 different categories with real user prompts, agent traces, and human preference labels. Agents are tasked with determining user preference between the application pairs.
ArenaWebdevCorrectness: 189 game versions with 362 human-verified requirements. Requirements are labeled and associated with “Broken” or “Working” versions of games. Models are tasked with identifying whether requirements are broken or working.
We evaluate three computer-use methods
Better computer-use methods lead to richer interactions and better evaluations.
Finding 1: Interaction-as-a-Judge improves LLM and reward model alignment to human preferences
We find that Interaction-as-a-Judge improves human alignment compared to code-only and code+screenshot judges. Using Interaction-as-a-Judge, we were able to train a 9B reward model that outperforms even frontier models (70.8% best performance). Additionally, we observe that better computer-use methods lead to better performance. We also find that it’s possible to train a reward model on interactions alone (69.0%).
Finding 2: Interaction-as-a-Judge helps models catch broken requirements
We find that Interaction-as-a-Judge not only catches more broken requirements than a code only LLM-as-a-Judge (63.0% → 71.1%), but it also scales with model capabilities; GPT 5.6 Luna correctly labels 65.2% requirements while GPT 5.6 Terra and GPT 5.6 Sol correctly label 68.0% and 71.1% respectively.
Finding 3: Interaction-as-a-Judge allows for improved inference-time scaling
We ask agents to iteratively repair and verify broken applications, with the idea that better self-verification would lead to more fixes over each iteration. We find that Interaction-as-a-Judge outperforms code-only self-feedback (55.4% → 75.5%).
Finding 4: Interaction-as-a-Judge can be used in reinforcement learning
We also attempt to fine-tune a Qwen-3.5-9B model using an interaction-only reward model signal. We find that RL with an interaction-only reward improves Qwen-3.5-9B’s win rate by the equivalent of over +50 Arena leaderboard points. Additionally, it does so while both reducing build failures and also reducing code bloat.
Ending Remarks
In this work, we study Interaction-as-a-Judge in detail and show how it can be used to improve evaluation, inference-time scaling, and reinforcement learning training. We are particularly excited about the prospect of using Interaction-as-a-Judge to further improve downstream performance and we think that this is just the beginning! As stated in our work, this approach scales with computer-use capabilities and will only improve over time. Some areas of research we think are promising and important:
- Real time computer-use agents: Current models are too slow to interact in real-time. This significantly limits model capabilities for things like game playing. Real-time computer-use will lead to better interactions which will lead to better applications and games.
- Interaction-based rewards: In this work, we did preliminary work to show that reinforcement learning with interaction-only is feasible. We believe that this opens doors to new data avenues where code is not readily available.
Acknowledgements
We thank Arena Intelligence for sponsoring API access, compute, and most importantly providing data to open-source and experiment on.
