DeepSeek Releases DeepSeek-V4-Pro-DSpark Featuring Speculative Decoding

Linkdeepseek-aiResource

DeepSeek has published the model card and weights for DeepSeek-V4-Pro-DSpark on Hugging Face. The release equips the DeepSeek-V4-Pro MoE model with an attached speculative decoding module (DSpark) for accelerated inference.

Key points

  1. DeepSeek-V4-Pro-DSpark uses the existing DeepSeek-V4-Pro checkpoint combined with a speculative decoding module based on DeepSpec.
  2. The DeepSeek-V4 series features V4-Pro (1.6T total, 49B activated parameters) and V4-Flash (284B total, 13B activated parameters), both supporting a 1M token context window.
  3. Architectural updates include Hybrid Attention (CSA and HCA), Manifold-Constrained Hyper-Connections (mHC), and the Muon optimizer.
  4. In a 1M token context setting, DeepSeek-V4-Pro reportedly requires 27 per cent of single-token inference FLOPs and 10 per cent of the KV cache compared to DeepSeek-V3.2.
  5. The model supports three reasoning modes (Non-think, Think High, Think Max) and is distributed under an MIT licence.

Why it matters

Running frontier MoE models over massive context windows creates severe bottlenecks in memory and inference latency. The combination of speculative decoding via DSpark and aggressive attention compression significantly reduces both KV-cache demands and FLOP requirements for open-source 1M-token workloads.

Analysis

This release underscores an industry shift towards sparse MoE models where only a fraction of total parameters are activated per token (49B out of 1.6T). Rather than relying purely on scaling, the focus lies on practical serving efficiency: integrated speculative decoding, hybrid attention mechanisms, and mixed FP4/FP8 precision allow deployment via serving engines like vLLM and SGLang. The omission of standard Jinja templates in favour of custom Python encoding scripts also indicates specialized token-handling requirements for complex reasoning modes.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Article:DeepSeek AI

    DeepSeek Launches DeepSeek-V4 Preview as Open Source with 1M Context

    DeepSeek has open-sourced the preview release of DeepSeek-V4. The line-up features DeepSeek-V4-Pro and DeepSeek-V4-Flash, standardising a 1M context window and introducing architectural innovations to cut compute and memory overhead.

    AI & LLMs· Announcement

  • X post:Mia

    DeepSeek Releases Official DeepSeek-V4-Flash-0731 Weights on Hugging Face

    DeepSeek has officially published the weights for DeepSeek-V4-Flash-0731 under an MIT licence on Hugging Face. The release supersedes the preview version, features enhanced agentic capabilities, and incorporates an attached speculative decoding module.

    336bookmarks153.4kviews

    AI & LLMs· Announcement

  • Link:Tencent Hy Team

    Tencent Releases Hy4-preview MoE Model Under Apache 2.0 License

    The Tencent Hy Team has released Hy4-preview, an open-source Mixture-of-Experts (MoE) model featuring 770B total parameters and 49B activated parameters per token. The flagship model supports a 1M token context window and is tuned for software engineering, office analytics, and scientific research.

    AI & LLMs· Announcement

  • Link:deepseek-ai

    NVIDIA Build lists DeepSeek-V4-Flash-0731 MoE model

    DeepSeek-V4-Flash-0731 is a sparse Mixture-of-Experts language model featuring 13B active parameters and a 1-million-token context window. The NVIDIA Build model card outlines technical specifications, architecture details, and coding benchmark performances.

    AI & LLMs· Resource

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.