X postHamid MaeiAnnouncement
Running a 27-billion-parameter model locally usually requires specialised or high-end GPU hardware. Reducing the footprint to under 6 GB allows capable frontier-style models to run on standard consumer devices without relying on external cloud APIs.
Ternary Bonsai 2 27B highlights the speed at which extreme model compression and ternary techniques are maturing. PrismML demonstrates that a 9x compression ratio does not necessarily incur severe quality penalties, particularly for complex tasks like agentic coding and tool use. The Apache 2.0 release provides developers with a strong candidate for local agent setups.
Hamid Maei
@HamidMaei · 17 September 2026
Really proud of this one. At @PrismML, we took Qwen3.8 27B, made it 9× smaller, and brought it down to just 5.9 GB—while retaining 98.2% of its benchmark performance. Kind of wild that a model this capable can run locally. t.co/gVOnULRS39
358 likes25 replies97 bookmarks21k views
View on XSummary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
X post:Udi Wertheimer
Udi Wertheimer highlights PrismML's release of Ternary Bonsai 2 27B, a model based on Qwen3.8 27B compressed to 5.9 GB while reportedly retaining 98.2% of its full-precision benchmark performance.
1.5kbookmarks260.1kviews
Link:prismml.com
PrismML has introduced the Bonsai model family, offering ultra-dense AI models based on research from Caltech. The flagship Bonsai 27B model runs locally on smartphones in a 1-bit variant requiring only 3.9 GB of memory.
Article:trendforce.com
Caltech spinout PrismML has compressed Alibaba’s open-source Qwen 3.6 model from around 54 GB to under 4 GB to run directly on the iPhone 17 Pro. The breakthrough has reportedly led to meetings with Apple as it aims to expand on-device AI capabilities.
Link:Qwen
The Qwen team has released Qwen3.8-27B, a compact, dense vision-language model under the Apache-2.0 licence. The 27-billion-parameter model pairs Gated DeltaNet with conventional gated attention, handles images and videos natively, and introduces flexible reasoning controls.
Link:Unsloth AI
Unsloth AI has published GGUF quantizations for the Qwen3.8-27B model under an Apache-2.0 licence. Utilizing their new Dynamic V3.0 quantization format, Unsloth claims over 10% higher top-1 accuracy at identical sizes compared to alternative quants. Qwen3.8-27B is a 27-billion parameter dense vision-language model featuring native text, image, and video comprehension alongside granular thinking controls.
Repository:Tencent/AngelSlim
AngelSlim is an open-source Python toolkit by Tencent designed for large-scale model compression and inference acceleration. It covers quantization, speculative decoding, and sparse attention to make massive models like Hunyuan and Qwen viable on constrained hardware setups.
1.6kstarsPython
Parts of this content were created with the assistance of AI.
