agent-fanout: Benchmarking parallel coding agents in iTerm2

Repositoryper-simmons/simmonsbench-agent-fanoutTool

agent-fanout spawns up to six parallel Claude Code instances in iTerm2, having them build tasks simultaneously while collecting token counts, costs, and runtimes for model comparisons.

Key points

  1. Launches up to six color-coded iTerm2 windows using AppleScript, each running `claude --model <MODEL>` in headless mode.
  2. Each window builds a dedicated task into a self-contained `index.html` and signals completion.
  3. A collector script compiles all builds into a local index page and outputs token counts, costs, and elapsed time into `summary.json`.
  4. Supports native Anthropic models (via Claude CLI login/subscription) as well as open models routed through OpenRouter.
  5. Provides the open-source fan-out technique underlying the SimmonsBench benchmark.

Why it matters

Benchmarking code generation models side-by-side often requires manual testing or bespoke test runners. agent-fanout offers an automated, visual desktop workflow to compare output quality, token usage, and cost across providers in parallel.

Analysis

The project solves the inefficiency of testing coding LLMs serially by fanning out tasks across multiple terminal windows on a single machine. In terms of technology, it relies on Bash scripts, AppleScript for iTerm2 window management, Python 3 (stdlib, optional pyobjc), and the Claude Code CLI, with routing support for OpenRouter. It is designed for developers, benchmarkers, and AI practitioners looking to evaluate and compare coding model performance on interactive web tasks. In terms of maturity, the repository is at a very early, experimental stage (4 stars, 3 forks as of late September 2026) and is strictly platform-dependent on macOS and iTerm2.

Mentioned links

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:Gitlawb/openclaude

    OpenClaude: Provider-Agnostic Open-Source Coding Agent CLI

    OpenClaude is an open-source TypeScript CLI tool functioning as an AI coding agent. It standardises terminal-based development workflows across varied local and cloud model providers.

    32.3kstarsTypeScript

    AI & LLMs· Tool

  • Repository:chhoumann/t3-pstack

    t3-pstack: Multi-Model Orchestration for Coding Agents

    t3-pstack adapts Lauren Tan's pstack and Michael Denyer's pstack-claude for T3 Code, dispatching subagents, review panels, and fan-outs across Claude, GPT, and Grok via T3's MCP tools.

    172starsJavaScript

    AI & LLMs· Tool

  • Repository:stablyai/orca

    Orca: Orchestrating Parallel AI Coding Agents via Git Worktrees

    Orca is an open-source Agent Development Environment (ADE) enabling developers to run multiple AI coding agents such as ClaudeCode, Codex, OpenCode, or Pi side-by-side across isolated Git worktrees with companion desktop and mobile apps.

    60.9kstarsTypeScript

    AI & LLMs· Tool

  • Repository:anthropics/claude-code

    Claude Code: Terminal-Based Agentic Coding Tool

    Claude Code by Anthropic is an agentic coding tool that operates in the terminal. It understands codebases, executes routine development tasks, explains complex code, and handles Git workflows using natural language commands.

    144.1kstarsPython

    AI & LLMs· Tool

  • Repository:manaflow-ai/cmux

    cmux: Ghostty-Based macOS Terminal for AI Coding Agents

    cmux is an open source macOS terminal multiplexer based on Ghostty, designed specifically for multitasking with AI coding agents. It features vertical tabs, workspace metadata, and visual notifications when agents require human attention.

    26.8kstarsSwift

    AI & LLMs· Tool

  • Repository:nekuda-ai/WindTunnel

    WindTunnel: Benchmarking Browser Agent Interfaces and WebMCP

    WindTunnel is an open-source benchmark suite designed to compare how browser agents interact with websites. It evaluates WebMCP (direct site-exposed actions) against coordinate-based computer use via screenshots and page structure navigation using DOM and accessibility trees.

    34starsHTML

    AI & LLMs· Research

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.