ReasoningBank: Scaling Agent Self-Evolution with Reasoning Memory

ArticleSiru Ouyang, Jun Yan, I-Hung Hsu, et al.Research

ReasoningBank is a memory framework for LLM agents that distils generalisable reasoning strategies from self-judged successful and failed experiences, enabling continuous self-evolution over time.

Key points

  1. Persistent LLM agents typically fail to learn from past interaction history, discarding insights and repeating previous mistakes.
  2. ReasoningBank distils general reasoning strategies from both positive and negative outcomes rather than archiving raw interaction logs.
  3. Memory-Aware Test-Time Scaling (MaTTS) allocates extra compute per task to produce rich, diverse experiences that provide contrastive learning signals.
  4. Across software engineering and web browsing benchmarks, the framework consistently outperforms standard trajectory-storage and success-only memory approaches.

Why it matters

Enabling autonomous agents to evolve directly from operational experience addresses a major barrier to deploying reliable, persistent AI systems in real-world environments.

Analysis

The authors introduce a compelling synergy between test-time scaling and agent memory. Rather than simply scaling search compute at inference or saving verbatim execution histories, extracting high-level reasoning strategies from contrasts between successes and failures allows agents to demonstrate sustained emergent self-improvement.

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Similar in content, found by the AI search.

  • Article:Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, Tu Vu

    WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution

    This research paper introduces WikiSkill, a framework that co-evolves AI agent skills alongside a persistent knowledge base (wiki). By disentangling raw execution experience, accumulated knowledge, and executable skills, WikiSkill enables systematic reuse of insights across iterations.

    AI & LLMs· Research

  • Article:Shuai Shao, Kangning Zhang, Qingyao Li, Shijian Wang, Hao Wang, Wenxiang Jiao, Yuan Lu, Yi Guo, Weiwen Liu, Weinan Zhang

    Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories

    Researchers introduce Harness-R1, a framework that trains a dedicated agent to adapt and patch the runtime harness of LLM agents based on failure logs. Using online reinforcement learning, a separate 9B model learns to generate executable patches for context handling, tool mediation, and error recovery to directly improve task completion.

    AI & LLMs· Research

  • Article:Hongjun Liu, Yifei Ming, Shafiq Joty, Chen Zhao

    HASP: Harnessing LLM Agents with Executable Skill Programs

    A research team introduces HASP (Harnessing LLM Agents with Skill Programs), a framework that converts past agent experiences into executable Program Functions (PFs). Instead of relying on passive textual prompts, PFs act as executable guardrails that trigger on failure-prone states to alter the next action or supply corrective context.

    AI & LLMs· Research

  • Repository:vectorize-io/hindsight

    Hindsight: Agent Memory That Learns

    Hindsight is an MIT-licensed memory system designed to help AI agents learn over time. Rather than merely recalling conversation histories, it aims to overcome the limitations of conventional RAG and knowledge graphs on long-term memory tasks.

    41.3kstarsPython

    AI & LLMs· Tool

  • Article:Devansh

    Stateful Swarms: How Persistent Memory Outperforms Traditional Agent Architectures

    Devansh explores 'Stateful Swarms', an architecture tackling volatile contexts and repetitive document processing through a persistent blackboard state. Instead of repeatedly processing large inputs or losing signal across conversational handoffs, specialized agents populate an append-only, structured memory. Evaluated on the Harvey Legal Agent Benchmark, this setup yielded higher task pass rates at significantly lower operational costs.

    AI & LLMs· Opinion

Let's talk about your project

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.