Token costs are exploding. Not because models are getting more expensive – but because AI agents no longer work for 15 minutes and then stop. They run for days, weeks, in parallel, autonomously, across hundreds of context windows. A single agent run today routinely consumes 40,000 tokens just through system prompt repetition. Ten loop cycles can cause 50 times the token consumption of a linear run.
And the most important insight is not a question of cost: it is not the model that determines the success or failure of an AI agent – but the infrastructure that orchestrates it.
Beyond a minimum capability threshold, a better harness yields more than a better model. LangChain increased the success rate of its coding agent from 52.8% to 66.5% – without a single model upgrade. Only the environment changed.
Technical leaders (CTOs, heads of engineering), software architects, and TYPO3 developers who want to understand why harness engineering is the strategic lever for productive AI agents – and how to prepare their projects for it.
Table of Contents
Evolution
Prompt → Context → Harness Engineering
Token Explosion
Parallel agents, days/weeks runtime
Harness Engineering
Anthropic's architecture, 5 pillars
Platforms
Factory AI, Paperclip, Agent Teams
Quality Criteria
Checklist, evaluation, SWE-bench
Research & Governance
Stanford, MIT, OpenAI, Deloitte
TYPO3
Content Blocks, Schema API, project rules
Conclusion
Three takeaways, next steps
From Prompt Engineering to Harness Engineering
The way we work with AI models has evolved in three stages. Each stage is a response to the limitations of the previous one:
Prompt Engineering was the first discipline. We formulated inputs so cleverly that models produced better outputs – a single, carefully constructed prompt for a single response.
Context Engineering replaced prompt engineering where systems went into production. Andrej Karpathy compares the context window to the RAM of a new operating system: it must be curated, not just filled. Tobi Lütke, CEO of Shopify, defined context engineering as "the art of providing all the context so that the task becomes plausibly solvable." The focus shifted from the single instruction to the dynamic system that assembles instructions, conversation history, tool outputs, and memory.
Harness Engineering goes one step further. It controls not only what is in the context window – but how agents work across multiple context windows, maintain state, recover from errors, and document progress. A harness is the infrastructure that surrounds an agent: memory systems, state management, error handling, tool selection, and context management.
| Feature | Prompt Engineering | Context Engineering | Harness Engineering |
|---|---|---|---|
| Focus | Single prompt | Entire context per call | Infrastructure across sessions |
| Metaphor | Writing a good letter | Curating the RAM of the LLM operating system | Building a working environment for shift workers |
| Time Horizon | A single call | A session | Hours, days, weeks |
| Controlled | Wording of the instruction | What goes into the context window | Memory, state, tools, error recovery |
| Analogy | Asking a good question | Providing the right context | Organising shift handovers |
Why Token Costs Are Now Exploding
The first generation of AI agents worked in a single session: prompt in, response out, done. The current generation works differently. Agents no longer run for 15 minutes – they run for days and weeks, in parallel, across hundreds of context windows.
From Minutes to Weeks
Claude Code supports asynchronous background agents that research, analyse, and generate code in the background while developers work on other tasks. With agent teams (experimental since Opus 4.6), multiple Claude Code sessions orchestrate collaboratively on a shared project – with direct communication between the agents.
Why Tokens Escalate
AI agents consume 3 to 10 times more LLM calls than simple chatbots. A single user request triggers planning, tool selection, execution, and verification. Costs escalate for four reasons:
System Prompt Repetition
The entire system prompt is sent with every API call. A 10-step agent with a 4,000-token system prompt consumes over 40,000 input tokens just through context accumulation.
Output Token Premium
Output tokens cost 3 to 8 times more than input tokens. Agents that generate detailed chain-of-thought reasoning pay this premium at every step.
Loop Cycles
A reflection or ReAct loop multiplies token consumption with every cycle. For identical tasks, research documents up to a 10-fold variance – solely due to different solution paths.
Token Spirals
Agents do not give up when they get stuck. They repeat failed approaches with minimal variations – and each iteration costs full input and output tokens again.
The Figures
The following table summarises current cost data:
| Metric | Value | Source |
|---|---|---|
| Average cost of Claude Code/day | ~$6 per developer | Anthropic |
| 90th percentile Claude Code/day | <$12 | Anthropic |
| Team costs/month (Sonnet 4.6) | $100–200 per developer | Anthropic |
| Token variance for identical tasks | Up to 10x | OpenReview |
| Enterprise agent deployments | $50,000–200,000 p.a. | TechAhead |
Token flow in a multi-agent setup: costs multiply through parallelisation and loop cycles
What Is Agent Harness Engineering?
The term agent harness describes the infrastructure that surrounds and controls an AI agent. Anthropic formalised this concept in November 2025, when the team realised that even frontier models like Opus 4.5 fail at complex projects if run in a loop without a harness.
The Core Problem
Imagine a software project staffed by engineers in shifts – and each new person arrives with absolutely no memory of the previous shift. This is exactly how agents work across context windows. Without a harness, two failure patterns occur:
- One-Shot Attempt: The agent tries to implement everything at once, runs out of context window, and leaves half-finished, undocumented features.
- Premature Completion Report: After a few features, the agent declares the project complete.
Anthropic's Two-Component Solution
Anthropic solves this with a two-part architecture:
Initializer Agent – Session 1
One-off setup: structure, progress file, and initial commit
Coding Agent – Session N
Repeats per session – one feature, one commit
The Initializer Agent sets up the environment in the first session: a comprehensive feature list as JSON (all features initially with "passes": false), an init.sh script to start the dev server, a claude-progress.txt for progress notes, and an initial Git commit.
Each Coding Agent begins its session with a fixed protocol: read progress file and Git logs, start dev server, test basic functionality, then implement a single feature and verify it end-to-end. At the end: Git commit with a descriptive message and progress update.
The Five Pillars of a Harness
External Memory
Information storage and retrieval beyond the context window. Feature lists, progress files, Git history – everything that allows an agent to reconstruct the state of the project.
State Management
Persisting progress across turns, sessions, and context boundaries. Without state management, every agent starts from scratch.
Error Recovery
Intercepting failed tool calls and implementing retry logic. Git-based rollbacks allow the agent to undo faulty changes.
Tool Selection
Which tools are available to the agent and how their interfaces are designed. Princeton research on Agent-Computer Interfaces shows: each tool should perform exactly one action.
Context Management
What goes into the context window and which eviction strategies apply. Server-side compaction, selective context injection, and incremental progress instead of overloading.
Platforms at a Glance
Three approaches demonstrate how differently multi-agent orchestration is implemented today.
Factory AI: Agent-Native Software Development
Factory AI pursues the concept of Agent-Native Software Development with specialised agents called "Droids": a Knowledge Droid for technical research and onboarding, a Code Droid for merge-ready pull requests, a Reliability Droid for incident response and root-cause analyses, and a Product Droid for feature planning and specifications.
The platform integrates into IDEs, browsers, CLIs, and Slack/Teams. For enterprise clients, Factory offers SSO, dedicated compute, and compliance certifications (SOC II, GDPR).
Early users report significant quality issues: code that does not follow best practices and requires manual rework. Token consumption is described as a "black hole" – with entire test credits consumed for a single feature. Basic functions, such as user authentication, showed glaring errors during testing.
Paperclip: Open-Source for "Zero-Human Companies"
Paperclip takes a radically different approach: an open-source orchestration platform for completely autonomous companies. AI agents are organised in a corporate hierarchy – with roles, reporting lines, and job descriptions.
The system is based on heartbeats: agents wake up at defined intervals, check their work, and act. Delegation flows automatically down the organisational chart. Each agent receives a monthly budget with automatic spending caps. Governance is handled via approval gates, budget controls, and complete audit logs.
With over 23,500 GitHub stars, an MIT licence, and a single Node.js process with an embedded PostgreSQL database, Paperclip is deliberately kept simple.
Multi-Agent Frameworks Compared
The following table compares the main multi-agent approaches:
| Feature | Factory AI | Paperclip | Claude Code Agent Teams | AutoGen (Microsoft) |
|---|---|---|---|---|
| Approach | Specialised Droids | Corporate hierarchy | Collaborative sessions | Multi-agent conversations |
| Licence | Proprietary | MIT (Open Source) | Proprietary | MIT (Open Source) |
| Orchestration | Platform-driven | Heartbeat + delegation | Shared task list | Directed graph |
| Budget Control | Token-based | Monthly agent budget | Effort parameter per sub-agent | No native control |
| Maturity | Early (quality criticisms) | Early (active development) | Experimental (Opus 4.6) | Stable (Best Paper ICLR'24) |
| Target Audience | Enterprise teams | Autonomous companies | Developers | Researchers + developers |
How Do You Recognise a Good Harness System?
A harness is not a product you buy – it is an architecture you build. The following criteria separate working systems from token-burning machines:
Incremental Progress
One feature per session. The agent never attempts to implement the entire project at once. Anthropic calls this the decisive factor against the "one-shot trap".
Clean State after Each Session
At the end of each session, the code is merge-ready: no open bugs, proper documentation, and a descriptive Git commit. Just like code a good developer would submit for review.
Automatic Verification
Without explicit testing, agents prematurely mark features as complete. Browser automation (Puppeteer, Playwright) for end-to-end testing is critical – code-based tests alone are not enough.
Structured Progress Files
Feature lists as JSON (harder for the agent to manipulate than Markdown), progress notes, and Git history together form the long-term memory across sessions.
Token Budget Control
Effort parameters per sub-agent (low/medium/high/max), monthly spending caps, and deterministic token budgets prevent uncontrolled cost explosions.
Error Recovery
Git-based rollbacks, retry logic for failed tool calls, and the ability to detect and repair a broken state before implementing new features.
Evaluation: Measure Outcomes, Not Paths
In "Demystifying evals for AI agents", Anthropic recommends a pragmatic start: 20 to 50 tasks, derived from real errors, are sufficient as a basis. The evaluation is based on three pillars according to the Google Cloud framework:
| Pillar | What is measured? | Method |
|---|---|---|
| Agent Success & Quality | Task completion, result quality | Code-based graders (unit tests), model-based graders (LLM judges) |
| Process & Trajectory | Reasoning logic, tool selection | Path analysis, but: accept valid results achieved via unexpected paths |
| Trust & Safety | Reliability under non-ideal conditions | Edge case testing, error injections |
Frontier models often find valid solution paths that designers did not foresee. Measure what the agent produces – not how it gets there. On SWE-bench Verified, the best agents improved from 4.4% to over 71.7% accuracy in just one year.
Microsoft's AXIS framework (ACL 2025) shows another lever: API-first Agent-Computer Interfaces instead of UI-based interaction reduce task completion time by 65 to 70% and cognitive overhead by 38 to 53%.
Research and Governance
The research landscape shows a clear picture: AI agents are rapidly becoming more powerful, but governance is lagging behind.
The SWE-bench Jump
Stanford HAI's AI Index Report 2025 documents one of the fastest performance increases in AI history: on the SWE-bench benchmark for software engineering, AI systems solved just 4.4% of coding problems in 2023 – by 2024, it was 71.7%. A jump of 48.9 percentage points in twelve months.
Enterprise Adoption vs. Governance Gap
The gap between adoption and governance is substantial:
| Metric | Value | Source |
|---|---|---|
| Corporate AI adoption | 78% (2024, vs. 55% in 2023) | Stanford HAI |
| Companies experimenting with AI agents | 62% | Deloitte 2026 |
| Companies with mature agent governance | Only 20% | Deloitte 2026 |
| Companies with measurable bottom-line impact | ~20% | McKinsey 2025 |
| Organisations redesigning workflows around AI | 34% | Deloitte 2026 |
McKinsey sums up the paradox: almost eight out of ten companies use generative AI, but just as many report no significant impact on business success. The reason: most deployments remain superficial – assistance tools rather than deeply integrated agents.
OpenAI's Governance Framework
In December 2023, OpenAI proposed seven practices for governing agentic systems – a framework relevant to any harness design:
- Clear allocation of responsibility – humans are liable for direct damages
- Action Ledgers – transparency regarding agent operations
- Human Approval Gates – human review for critical decisions
- Capability Boundaries – defined limits for system impact
- Staged Deployment – gradual rollout with monitoring
- Reversibility Design – keeping actions reversible where possible
- Shutdown Capabilities – reliable mechanisms to halt operations
Population-Level Coordination
MIT's Ripple Effect Protocol (REP) addresses a problem beyond individual agents: the coordination of entire agent populations. Instead of complete information, agents exchange lightweight "sensitivities" – signals describing how decisions would shift in response to environmental changes. The result: 41 to 100% better coordination in supply chain, preference, and resource scenarios.
Optimising TYPO3 Projects for the Harness
Harness engineering is not just for greenfield projects. Existing TYPO3 codebases can be specifically prepared so that AI agents work more precisely with less context. The complete deep-dive with code examples can be found in our dedicated article.
The strongest levers at a glance:
| Lever | Replaces | AI Benefit |
|---|---|---|
| Content Blocks | TCA scattered across 4+ files | One YAML file instead of four – TCA, SQL, and forms are generated |
| PHP 8.4 Property Hooks | Series of getters/setters | ~85% less boilerplate per property |
| DataHandler as write path | Direct SQL updates | Workspaces, permissions, and FAL relations processed correctly |
| Schema API (v13.2+) | Access to $GLOBALS['TCA'] array | Typed OOP instead of array navigation |
| .cursorrules / AGENTS.md | Implicit team knowledge | Persistent project rules reduce variance between sessions |
| PHPStan + CI gates | Manual code reviews | Mechanical protection for agent-generated code |
Making TYPO3 AI-ready
What the TYPO3 core already does for AI readability – and what you should add to your project. Featuring the Schema API, Content Blocks, property hooks, DataHandler, and harness engineering.
Conclusion
Harness engineering is neither a buzzword nor an optional feature. It is the discipline that determines whether AI agents work productively or burn tokens.
It is not the teams with the most developers that will win in 2026 – but those whose harness architecture reliably orchestrates AI agents.
If you want to audit your existing codebase to find where AI agents are currently being held back, a focused architecture review is the fastest starting point.