Agent Arena: Benchmark Results for Autonomous Coding Agents

ArticleResource

Agent Arena evaluates autonomous AI coding agents on complex real-world tasks using crowdsourced head-to-head comparisons to calculate Elo ratings.

Key points

  1. Crowdsourced benchmark run by Design Arena evaluating autonomous AI agents on real-world problems.
  2. Four-step methodology: task assignment, autonomous execution, side-by-side human voting, and Bradley-Terry Elo scoring.
  3. Evaluation dimensions cover task completion, output quality, efficiency, and robustness.
  4. Rankings (as of December 2025): Factory Droid leads at 1330 Elo, followed by OpenAI Codex (1301), Devin (1263), Claude Code (1242), Cursor (1120), and Gemini CLI (937).

Why it matters

As coding assistants evolve into autonomous agents, side-by-side human evaluations help quantify practical code quality and task completion in real-world environments.

Analysis

By adapting the Elo rating system to autonomous coding tasks, Agent Arena captures nuanced qualitative differences across tools. The gap between top-performing agents like Factory Droid or OpenAI Codex and lower-ranked tools reflects substantial variance in autonomy and execution quality.

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Similar in content, found by the AI search.

Let's talk about your project

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.