Remory: Learning Residual Memory for Context Compaction in LLM Agents

Repositoryfudan-generative-vision/RemoryTool

Remory enhances standard text summaries in LLM agents with learned soft memory extracted from execution history. Conditioned on the summary and appended to it, this residual memory enables frozen base models to sustain continuity across context compactions.

Key points

  1. Supplements standard agent compaction summaries with learned soft memory derived from past context history.
  2. The residual memory conditions on the summary and appends to it, leaving the primary foundation model frozen.
  3. Provides pretrained memory checkpoints for Qwen3.8-27B (1.9B memory model) and GLM-5.3-Flash (1.24B memory model).
  4. Deploys via SGLang and Transformers, requiring at least two 80 GB GPUs to serve the Qwen3.8-27B stack.
  5. Includes integration with a native Codex environment (r-codex), supporting `/compact` commands and session resumption.

Why it matters

Long-running LLM agents often suffer from degraded performance when older context is truncated or simplified into plain text summaries. Remory demonstrates that pairing discrete text summaries with learned latent residual states preserves critical operational details without requiring expensive retraining of the foundation model.

Analysis

Developed by Fudan Generative Vision, Remory tackles the degradation of agent context during extended task executions. While standard workflows rely solely on summarising previous conversation history—inevitably shedding granular operational data—Remory trains dedicated memory modules to capture and append residual soft memory. The architecture relies on Python, custom CUDA kernels, and SGLang inference serving using tensor parallelism (--tp 2), alongside Hugging Face Transformers. The target audience comprises AI engineers and researchers building local autonomous agents and coding assistants. In terms of maturity, it is a research-backed open-source release accompanying an arXiv paper (2610.11287), complete with Hugging Face checkpoints and terminal UI integrations, though its substantial hardware requirements (at least 2 × 80 GB GPUs) keep it primarily in the advanced engineering and experimental deployment tier.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 18 October 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:vectorize-io/hindsight

    Hindsight: Agent Memory That Learns

    Hindsight is an MIT-licensed memory system designed to help AI agents learn over time. Rather than merely recalling conversation histories, it aims to overcome the limitations of conventional RAG and knowledge graphs on long-term memory tasks.

    41.3kstarsPython

    AI & LLMs· Tool

  • Repository:rohitg00/agentmemory

    agentmemory: Persistent Memory for AI Coding Agents

    agentmemory is an open-source TypeScript solution providing persistent memory for coding agents and MCP clients. Powered by the iii engine, it builds on Andrej Karpathy's LLM Wiki pattern by introducing confidence scoring, lifecycle management, knowledge graphs, and hybrid search.

    28kstarsTypeScript

    AI & LLMs· Tool

  • Repository:thedotmack/claude-mem

    claude-mem: Persistent Cross-Session Memory for AI Agents

    The open-source project claude-mem (now also branded as Grok Mem) captures agent activity during active sessions, compresses it with AI, and injects relevant context back into future runs.

    93.1kstarsJavaScript

    AI & LLMs· Tool

  • Repository:PrimeIntellect-ai/prime-agent

    Prime Agent: Self-Improving RLM Harness for Long-Running Coding Workflows

    Prime Agent is an open-source coding and research agent designed for complex and long-running autonomous workflows. It combines a Recursive Language Model (RLM) architecture with a Continual Harness to manage context programmatically and persist learned skills across sessions.

    19.9kstarsTypeScript

    AI & LLMs· Tool

  • Repository:Gentleman-Programming/engram

    Engram: Persistent Memory System for AI Coding Agents

    Engram is an open-source, agent-agnostic memory engine written in Go that provides AI coding assistants with persistent memory across sessions.

    6.4kstarsGo

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.