Local Markdown Compiler Replaces LLM Wikis Using Pure Python

LinkEmmimal P AlexanderTool

Emmimal P Alexander replaced an agentic LLM-based personal wiki with a pure Python compiler. The tool processes raw text notes into a linked markdown wiki deterministically, without API calls, embeddings, or third-party dependencies.

Key points

  1. Critique of Karpathy-style agentic wikis: probabilistic models cause unnecessary token expenses, latency, and inconsistent link structures on static files.
  2. Zero third-party dependencies, relying exclusively on the Python standard library without vector databases or embedding APIs.
  3. Four-stage pipeline: regex metadata extractor, link graph builder, section-aware rewriter preserving manual notes, and a structural linter.
  4. Algorithmic fix: replacing quadratic pairwise regex comparisons with a word-indexed phrase matcher reduced execution time on 5,000 files from 107 seconds to under 500 ms.
  5. Deterministic, cross-platform results verified with identical outputs across Linux and Windows environments.

Why it matters

Teams often default to probabilistic AI agents for tasks that are fundamentally deterministic parsing and graph problems. This project illustrates how traditional algorithms reduce cost, latency, and maintenance overhead for local personal knowledge bases.

Analysis

Alexander's approach highlights that compiling local structured text is a parsing problem rather than a reasoning problem. Employing LLMs to reorganise owned local text creates non-determinism, as link structures can shift without changes to the underlying files. The primary trade-off is the reliance on strict lexical matching: the compiler misses semantic synonyms that embeddings would detect. For users with defined terminology, this trade-off delivers immediate speed, zero operational cost, and full reproducibility.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Link:agentwikis.com

    Agent Wikis: Maintained Knowledge Bases for Humans and AI Agents

    Agent Wikis transforms fast-moving information sources such as repositories, changelogs, transcripts, and community Q&A into structured Markdown knowledge bases. Designed for both humans and AI agents, the content is accessible via an llms.txt entry point.

    AI & LLMs· Resource

  • Repository:AgriciDaniel/claude-obsidian

    claude-obsidian: Local-First AI Knowledge Management for Obsidian and Claude Code

    claude-obsidian is an MIT-licensed open-source project that connects Claude Code with Obsidian into a self-organizing knowledge graph. Implementing Karpathy's LLM Wiki pattern, it maintains a structured vault made entirely of local plain Markdown files.

    14.6kstarsPython

    AI & LLMs· Tool

  • Article:Wyndo

    LLM Wiki: Building an AI-Powered Second Brain in Obsidian

    Inspired by an idea from Andrej Karpathy, author Wyndo outlines a workflow using Claude Code and Obsidian Skills to automatically compile, cross-link, and maintain a personal knowledge base.

    AI & LLMs· Tutorial

  • Article:Agrici Daniel

    claude-obsidian: Self-Organising Knowledge Base with Claude Code

    Agrici Daniel introduces claude-obsidian, a free open-source plugin for Claude Code. It implements Andrej Karpathy's LLM Wiki pattern to automatically compile and link Obsidian vaults from raw documents without requiring a vector database.

    AI & LLMs· Tool

  • Repository:alexeyban/EKOS

    EKOS: Enterprise Knowledge Compiler for AI Agents

    EKOS (Enterprise Knowledge Operating System) is an open-source Rust-based platform designed to observe, compile, and serve enterprise knowledge from disparate sources via the Model Context Protocol (MCP) for AI agents.

    10starsRust

    AI & LLMs· Tool

  • Link:portablellm.wiki

    Portable LLM Wiki: Portable Personal Context for LLMs via URL and MCP

    Portable LLM Wiki is a tool that turns bios, resumes, and personal notes into a structured wiki. It exposes this context to LLMs like ChatGPT, Claude, and tools like Cursor through a simple URL, a QR code, or the Model Context Protocol (MCP).

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.