Evals vs. Loops: Why Post-Hoc Checks Fail to Fix Agent Workflows

X postHanako11 September 2026Opinion

Hanako breaks down the limitations of conventional evaluations in AI systems, contrasting them with loop and graph architectures. While evals only report failures after execution completes, loops act as gating mechanisms between individual steps to halt compounding errors.

Key points

Evals run after all tasks finish and only report failure, allowing a broken intermediate step to poison subsequent actions.
A loop places the same verification logic between individual steps as a gate, halting execution and retrying before downstream tasks run.
Checks must be programmatically verifiable conditions (such as exit codes or file diffs), rather than impressions like model confidence or zero exceptions raised.
Loops operate inside individual nodes to make single tasks correct, whereas graphs coordinate dependencies, parallelism, and structure between nodes.
Graph architectures rely on four node types: splitters, workers, gates, and pure code nodes that handle deterministic tasks without model overhead.
Return paths must reject only the specific failing unit rather than the entire batch to avoid unnecessarily regenerating valid work.

Why it matters

Building end-of-run dashboards does not prevent multi-step autonomous workflows from cascading when early mistakes occur. Moving verification logic into the control flow allows systems to self-correct in real time, preventing wasted compute and reducing manual oversight.

Analysis

Hanako maps the evolution from Prompts to Context, Harness, Loops, and Graphs. A core observation is that instrumentation without halting capability is merely a report, not active control logic. Furthermore, the article warns against over-relying on LLMs for deterministic data operations, urging developers to reserve models for unbounded reasoning and use code nodes for ranking, deduplication, and parsing.

Original post

Hanako

@hanakoxbt · 11 September 2026

Evals vs. Loops, clearly explained! evals are great, and everyone is building them right now. here is the ceiling: an eval runs after the work is done. it tells you the work was wrong. four steps ran. the second one was wrong. steps three and four ran on its output anyway, t.co/00zgM6bx0u t.co/fEPJmINlgv

89 likes15 replies96 bookmarks8.3k views

View on X

Selected replies (5)

@biscuitweb3 @hanakoxbt loops catch mistakes before they spread
@0xSlyth @hanakoxbt loops act evals report
@0xTimQ @hanakoxbt wow cool and amazing visual!
@rewind02 @hanakoxbt i needed that line
@MegaEvolvedFrog @hanakoxbt What did you use to generate the animation ?

Mentioned links

Links and tools from this entry

Five Layers: A Blueprint and Course for Agent EngineeringLink

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Similar in content, found by the AI search.

X post:Kirill15 Aug 2026
From Loop Engineering to Graph Engineering: Deterministic AI Architectures
Kirill describes an architectural shift in agent development from fragile single-agent loops to Graph Engineering. He argues that system reliability depends not on larger models or more agents, but on deterministic structures using nodes, edges, state verifiers, and knowledge graphs.
27.3kbookmarks2.7mviews
AI & LLMs· Opinion
X post:Mahax6 Aug 2026
Moving Beyond Prompts to Loops and Graphs in AI Systems
Citing Andrew Ng, Mahax highlights the shift from single-shot prompting to agentic loops and graphs. An accompanying guide outlines the architecture of AI agents, self-evaluating feedback loops, and graph-based execution pipelines using Claude.
11.8kbookmarks1mviews
AI & LLMs· Resource
Video
Video:Sean‘s AI Stories1 Aug 2026
Loop vs Graph Engineering for AI Agents: Concepts and Practice with Waku Agent
Sean Chen addresses the debate surrounding loop versus graph engineering in AI agent systems. Presenting an engineering ladder from prompts to graphs, he explains when to apply each pattern and demonstrates the architecture using the open-source Waku Agent project.
AI & LLMs· Tutorial
Video
Video:AI LABS29 Jul 2026
Graph Engineering for AI Agents: Parallel Workflows and Verification
Graph engineering supersedes linear loop engineering by splitting complex tasks across multiple parallel sub-agents. AI Labs outlines the node-and-edge architecture, the risk of cascading failures, and concrete verification patterns implemented with Claude Code.
AI & LLMs· Tutorial
X post:Mahax10 Sept 2026
Graph Engineering: Why Agent Workflows Depend on Graphs
Mahax cites Andrej Karpathy's claim that prompting is disappearing in favour of graphs, and provides a practical guide on graph engineering with Claude to optimise AI agent workflows.
2.4kbookmarks240.8kviews
AI & LLMs· Opinion
X post:nicco21 Sept 2026
Graph Engineering Over Prompting: Why Agent Workflows Require Graph Architectures
A post by nicco highlights remarks by Andrej Karpathy asserting that prompting is an ephemeral interface and that graph architectures represent the true architectural layer for agent workflows. It pairs this with a practical guide by seeco on replacing sequential agent chains with explicit graphs.
529bookmarks58.5kviews
AI & LLMs· Resource