mini-AGI: Byte-Level Continual Learning on 8 GB VRAM Consumer Hardware

Repositoryvolotat/mini-AGIResearch

mini-AGI is an experimental open-source byte-level language model designed to demonstrate continual learning from a single stream of data (batch size 1) on modest 8 GB VRAM hardware. By paging model weights from disk into GPU memory as needed, it decouples parameter capacity from VRAM limits while dynamically scaling its architecture.

Key points

  1. Continual stream learning: Trains from scratch on a single, continuous stream of data without suffering from catastrophic forgetting.
  2. Disk-based weight paging: Parameters reside as files on the local drive and are paged into 8 GB VRAM as required, making storage rather than GPU memory the capacity bound.
  3. Self-adapting architecture: Dynamically grows new capacity (such as experts) when needed and prunes inactive weights.
  4. Unified processing path: Reads, learns, and generates text using the exact same code execution path.
  5. Maturity level: An early-stage experimental toy model; model weights are not yet published while the initial corpus pass continues.

Why it matters

Standard LLMs rely on massive pre-training runs on expensive compute clusters and struggle with catastrophic forgetting when exposed to continuous streams of new information. mini-AGI explores an architectural alternative that brings ongoing adaptation and local training to consumer-grade hardware.

Analysis

Created by developer volotat, mini-AGI is explicitly framed as an experimental proof of concept rather than a frontier model competitor. The tongue-in-cheek name refers to desired AGI attributes—adaptive capacity and continuous learning—rather than raw capability. Its technical strategy of combining byte-level tokens, modular dynamic architecture, and disk-to-VRAM weight paging provides a compelling blueprint for localized, ongoing model training. Since the model is still in its first reading pass and weights are not yet published, its long-term retention and generalisation capabilities under prolonged stream conditions remain to be validated.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 29 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:JustVugg/colibri

    Colibrì: Running Frontier MoE Models via Disk-Streamed Experts

    Colibrì is a lightweight inference engine written in pure C with zero dependencies, designed to run frontier Mixture-of-Experts (MoE) models ranging from 744B to 2.8T parameters on consumer and heterogeneous hardware. It treats VRAM, RAM, and storage as a unified memory hierarchy, streaming experts on demand.

    37.7kstarsC

    AI & LLMs· Tool

  • Repository:vectorize-io/hindsight

    Hindsight: Agent Memory That Learns

    Hindsight is an MIT-licensed memory system designed to help AI agents learn over time. Rather than merely recalling conversation histories, it aims to overcome the limitations of conventional RAG and knowledge graphs on long-term memory tasks.

    41.3kstarsPython

    AI & LLMs· Tool

  • Repository:test-time-training/e2e

    End-to-End Test-Time Training for Long Context in JAX

    This repository provides the official JAX implementation of End-to-End Test-Time Training (TTT) for long contexts. It approaches long-context language modeling as a continual learning challenge, using a standard Transformer with sliding-window attention that compresses input contexts into its weights at test time via next-token prediction.

    693starsPython

    AI & LLMs· Research

  • Repository:MakazhanAlpamys/Soup

    Soup: Single-YAML LLM Fine-Tuning on Consumer GPUs

    Soup (soup-cli) is an open-source tool designed to fine-tune language models via a single YAML configuration file and a unified CLI command. By leveraging layer streaming, it allows users to train 8B models on consumer hardware with as little as 4 GB of VRAM.

    7.5kstarsPython

    AI & LLMs· Tool

  • Repository:FlashML-org/FreeToken

    FreeToken: Local Serving of 290B+ MoE Models on Consumer Hardware

    FreeToken is an open-source inference engine designed to run massive Mixture-of-Experts (MoE) models on desktop PCs and laptops. Using adaptive CPU-GPU co-execution, semantic caching, and dynamic VRAM management, it enables models like DeepSeek-V4-Flash and GLM-5.2 to run locally without datacentre infrastructure.

    11.7kstarsPython

    AI & LLMs· Tool

  • Repository:fudan-generative-vision/Remory

    Remory: Learning Residual Memory for Context Compaction in LLM Agents

    Remory enhances standard text summaries in LLM agents with learned soft memory extracted from execution history. Conditioned on the summary and appended to it, this residual memory enables frozen base models to sustain continuity across context compactions.

    236starsPython

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.