Repository:FlashML-org/FreeToken
FreeToken: Local Serving of 290B+ MoE Models on Consumer Hardware
FreeToken is an open-source inference engine designed to run massive Mixture-of-Experts (MoE) models on desktop PCs and laptops. Using adaptive CPU-GPU co-execution, semantic caching, and dynamic VRAM management, it enables models like DeepSeek-V4-Flash and GLM-5.2 to run locally without datacentre infrastructure.
11.7kstarsPython
From Video: AI Weekly Updates: Open-Source Model Surge, Agent Harnesses, and Meta's AI Reality Check