PrismML Announces Ternary Bonsai 2 27B: A 6 GB Quantised Model

X postUdi WertheimerOpinion

Udi Wertheimer highlights PrismML's release of Ternary Bonsai 2 27B, a model based on Qwen3.8 27B compressed to 5.9 GB while reportedly retaining 98.2% of its full-precision benchmark performance.

Key points

  1. PrismML announced Ternary Bonsai 2 27B under the Apache 2.0 licence.
  2. Based on Qwen3.8 27B, the 5.9 GB model is nine times smaller than its full-precision equivalent.
  3. According to PrismML, it retains 98.2% of aggregate benchmark performance, showing gains in agentic coding, multimodal reasoning, and tool use.
  4. Udi Wertheimer claims it is nearly as capable as Opus 4.5 and could run on high-end hardware like iPads with 16 GB or more memory.
  5. Community feedback in the thread highlights trade-offs, citing slower inference speeds and saturated benchmark results in practical coding.

Why it matters

Ternary quantisation shrinks substantial model weights down to sizes manageable by edge devices, potentially lowering the barrier to running advanced local models on consumer-grade hardware.

Analysis

While Wertheimer enthusiastically compares the small footprint to top-tier frontier models like Opus 4.5, initial real-world user tests mentioned in the replies caution that benchmark parity does not always translate to flawless practical coding. Regardless, compressing a 27B parameter architecture to under 6 GB marks a notable milestone in efficient local inference.

Original post

Udi Wertheimer

@udiWertheimer · 18 September 2026

this is crazy. 6gb model and it’s almost as powerful as opus 4.5 you should be able to run this on a high-end ipad t.co/dOhiTXSfDz

2.6k likes37 replies1.5k bookmarks260.1k views

View on X
More posts in the thread (4)
  1. @TheDesertLynx what's your hardware?
  2. @TheDesertLynx kimi k3 on 128gb??
  3. @KryptoKenCPU if it’s like 16gb or more, yeah…
  4. @bensig i tried the previous version. i don’t remember the numbers but it didn’t feel exceptionally slow
Selected replies (5)
  • @TheDesertLynx @udiWertheimer Get a mini PC before they get too expensive. I've gone fully-local and it's great.
  • @TheDesertLynx @udiWertheimer Yep. Not going to be fast, but there are inference engines that let you do it on as low as 64gb
  • @TheDesertLynx @udiWertheimer Strix Halo 128GB. Running DeepSeek V4 Flash mainly. Giving Kimi K3 another go tonight.
  • @xolandar @udiWertheimer Used the abliterated model and it's awful Just asked for a story of Batman consuming meth it went into an infinite loop Also ggufs only 23tok/s on a M3 Max 36gb so poor performance
  • @dev_in_tech @udiWertheimer Maybe I am missing it, where is the comparison with opus4.5 ?

Mentioned links

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • X post:Hamid Maei

    Ternary Bonsai 2 27B: Highly compressed Qwen3.8 model reduced to 5.9 GB

    PrismML has announced Ternary Bonsai 2 27B, a compressed version of Qwen3.8 27B. It is nine times smaller than the full-precision original, requiring only 5.9 GB of storage while retaining 98.2% of aggregate benchmark performance.

    97bookmarks21kviews

    AI & LLMs· Announcement

  • X post:Trevor Wood

    Qwen3.8-27B Abliterated Released for Apple Silicon via MLX

    Trevor Wood and PocketAI Model Lab have released MLX-optimised checkpoints of Qwen3.8-27B for Apple Silicon with built-in refusal mechanisms removed ('abliterated'), provided across multiple quantisation formats from 2-bit to BF16.

    3.2kbookmarks199kviews

    AI & LLMs· Announcement

  • Link:Unsloth AI

    Unsloth Releases Dynamic V3.0 GGUF Quants for Qwen3.8-27B

    Unsloth AI has published GGUF quantizations for the Qwen3.8-27B model under an Apache-2.0 licence. Utilizing their new Dynamic V3.0 quantization format, Unsloth claims over 10% higher top-1 accuracy at identical sizes compared to alternative quants. Qwen3.8-27B is a 27-billion parameter dense vision-language model featuring native text, image, and video comprehension alongside granular thinking controls.

    AI & LLMs· Tool

  • X post:0xSero

    LLM Benchmark Performance at 3-Bit Quantisation

    A post on X highlights surprisingly high benchmark performance for a language model running at 3-bit quantisation (exl-3bits), achieving a 78.6% score alongside 100 tokens per second at 1400W peak power.

    36bookmarks11kviews

    AI & LLMs· Discussion

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.