TurboFieldfare: Gemma 4 26B-A4B Inference in 2 GB RAM on Apple Silicon

Repositorydrumih/turbo-fieldfareTool

TurboFieldfare is a dedicated Swift and Metal runtime designed to run Google's Gemma 4 26B-A4B on Apple Silicon Macs within an approximately 2 GB RAM budget. It avoids loading the full 14.3 GB model into memory by keeping only the 1.35 GB core and KV cache in RAM while streaming the required mixture-of-experts weights from SSD on demand.

Key points

  1. Enables local inference for the 26-billion-parameter Gemma 4 26B-A4B model on entry-level Macs with 8 GB RAM.
  2. Operates within ~2 GB of RAM by streaming required MoE experts token-by-token from NVMe storage.
  3. Resident RAM footprint is restricted to the 1.35 GB shared core and the FP16 KV cache.
  4. Built from scratch using Swift 6.2 and Metal 4, avoiding wrappers around MLX or llama.cpp.
  5. Provides a native macOS application, CLI tools, a decode service executable, and local server integration.

Why it matters

Running capable large language models locally is often blocked by unified memory limitations on standard consumer laptops. Selective expert streaming directly from fast SSDs allows developers and privacy-focused users to execute 20B+ parameter models on hardware previously limited to much smaller architectures.

Analysis

Developed by drumih, the repository is tailored specifically for Gemma 4 MoE architectures rather than being a generalized inference engine. With over 6,600 GitHub stars, 103 documented kernel experiments, and native macOS tooling under the Apache-2.0 license, it demonstrates strong technical depth, targeting users on recent macOS releases.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Link:drumih

    TurboFieldfare: Running Gemma 4 26B in 2 GB RAM on Apple Silicon

    TurboFieldfare is an open-source inference engine written in Swift and Metal that executes 4-bit Gemma 4 26B-A4B-IT on Apple M-series Macs using roughly 2 GB of RAM. While the model weights require 14 GB of disk space, the engine keeps only the shared model base and KV cache in memory and streams routed expert layers on demand directly from the SSD.

    AI & LLMs· Tool

  • Repository:JustVugg/colibri

    Colibrì: Running Frontier MoE Models via Disk-Streamed Experts

    Colibrì is a lightweight inference engine written in pure C with zero dependencies, designed to run frontier Mixture-of-Experts (MoE) models ranging from 744B to 2.8T parameters on consumer and heterogeneous hardware. It treats VRAM, RAM, and storage as a unified memory hierarchy, streaming experts on demand.

    37.7kstarsC

    AI & LLMs· Tool

  • Repository:ashhart/TensorFold

    TensorFold: LLM Inference Engine for Apple Silicon and Modern NVIDIA GPUs

    TensorFold is an open-source inference engine designed to serve large language models on Apple Silicon (via MLX/Metal) and newer NVIDIA GPUs (Compute Capability 8.9+). It provides OpenAI- and Anthropic-compatible endpoints with model-tailored kernels and speculative draft verification.

    1.1kstarsPython

    AI & LLMs· Tool

  • Repository:google-ai-edge/LiteRT-LM

    LiteRT-LM: Google's Open-Source Inference Framework for On-Device LLMs

    LiteRT-LM is an open-source inference orchestration layer from Google designed to run large language models directly on edge devices. It leverages GPU and NPU acceleration and powers on-device generative AI features in Chrome, Pixel Watch, and Chromebook Plus.

    6.4kstarsC++

    AI & LLMs· Tool

  • Repository:PipeNetwork/kimi-k3-mlx

    kimi-k3-mlx: Apple Silicon Port of the 2.78T Multimodal MoE Kimi-K3

    kimi-k3-mlx is an Apple MLX port of Moonshot AI's native multimodal MoE model Kimi-K3 (2.78T total parameters). Because the model exceeds the memory capacity of any single Apple Silicon machine, the repository includes a custom streaming converter, vision components, and expert-pruning tools.

    337starsPython

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.