RepositoryJustVugg/colibriTool
Frontier MoE models typically demand enterprise-grade GPU clusters. Colibrì bypasses massive VRAM requirements through intelligent tiering, opening access to cutting-edge research, privacy-conscious local runs, and low-budget AI deployments.
Colibrì addresses the prohibitive hardware requirements of frontier MoE models. Architecturally, it adopts a minimalist approach: pure C, zero external dependencies, and single-file implementations per model family. In terms of maturity, boasting over 37,000 GitHub stars and version 1.12.1 with dedicated dashboards, it has evolved well beyond an initial proof of concept into an actively maintained research engine. However, the repository explicitly notes that throughput depends heavily on I/O speeds and offers no fixed latency SLA. It is aimed at ML engineers, systems researchers, and developers seeking to experiment with multi-hundred-billion parameter models on commodity hardware.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Repository:FlashML-org/FreeToken
FreeToken is an open-source inference engine designed to run massive Mixture-of-Experts (MoE) models on desktop PCs and laptops. Using adaptive CPU-GPU co-execution, semantic caching, and dynamic VRAM management, it enables models like DeepSeek-V4-Flash and GLM-5.2 to run locally without datacentre infrastructure.
11.7kstarsPython
Repository:drumih/turbo-fieldfare
TurboFieldfare is a dedicated Swift and Metal runtime designed to run Google's Gemma 4 26B-A4B on Apple Silicon Macs within an approximately 2 GB RAM budget. It avoids loading the full 14.3 GB model into memory by keeping only the 1.35 GB core and KV cache in RAM while streaming the required mixture-of-experts weights from SSD on demand.
6.6kstarsSwift
Link:drumih
TurboFieldfare is an open-source inference engine written in Swift and Metal that executes 4-bit Gemma 4 26B-A4B-IT on Apple M-series Macs using roughly 2 GB of RAM. While the model weights require 14 GB of disk space, the engine keeps only the shared model base and KV cache in memory and streams routed expert layers on demand directly from the SSD.
Repository:inclusionAI/Ling
InclusionAI has open-sourced Ling, a family of Mixture-of-Experts (MoE) large language models. The suite is offered in two sizes: Ling-lite with 16.8 billion total parameters (2.75 billion activated) and Ling-plus with 290 billion total parameters (28.8 billion activated).
267starsPython
Repository:PipeNetwork/kimi-k3-mlx
kimi-k3-mlx is an Apple MLX port of Moonshot AI's native multimodal MoE model Kimi-K3 (2.78T total parameters). Because the model exceeds the memory capacity of any single Apple Silicon machine, the repository includes a custom streaming converter, vision components, and expert-pruning tools.
337starsPython
Link:inclusionAI
inclusionAI has introduced Ling-3.0-tiny, a compact MoE model with 7.9B total parameters activating only 1.3B parameters per token. Utilizing a hybrid architecture of linear and latent attention mechanisms, it aims to deliver efficient reasoning and agentic workloads on local hardware.
Parts of this content were created with the assistance of AI.
