Running DeepSeek-v4-Flash on a Single DGX Spark

RepositoryMiaAI-Lab/DeepSeek-v4-Flash-One-DGX-SparkTool

An open-source launcher and Docker recipe for running DeepSeek V4 Flash 0731 (EXL3) on a single NVIDIA DGX Spark with 128 GiB unified memory.

Key points

  1. Operates on a single DGX Spark with tensor-parallel 1, removing the need for a two-node TP2 cluster required by the official FP4 build.
  2. Deploys the 3.0 bpw EXL3 build (0xSero/deepseek-v4-flash-0731-spark) using the sparkinfer (formerly b12x) kernel stack.
  3. Features DSpark K5 speculative decoding with a K64 draft model and nvfp4_ds_mla compressed KV cache using B12X_MLA_SPARSE.
  4. Delivers 44–47 tokens per second decode speed with a KV cache pool of up to 439,622 tokens and context lengths up to 384k.
  5. Includes long-prefill starvation guards, tuned CUDA graphs, and applies upstream kernel backports as read-only bind mounts.

Why it matters

Running massive context-length models usually demands distributed multi-node clusters. This project demonstrates how combining custom quantisation, kernel optimisations, and speculative decoding allows serving deep-context models on a single unified-memory GPU.

Analysis

The project solves the high-entry hardware barrier of official DeepSeek V4 Flash deployments by packing execution into a single 128 GiB DGX Spark node. It leverages a tailored Docker environment, EXL3 quantisation, and DSpark speculative decoding. It targets ML engineers and researchers running DGX Spark hardware (GB10, SM121). In terms of maturity, it functions as a highly targeted, bench-tested community tool rather than a generic framework, complete with needle-in-a-haystack stress tests up to 370k context tokens and custom kernel backports.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • X post:Mia

    Running DeepSeek v4 Flash on a Single NVIDIA DGX Spark

    MiaAI-Lab published an open-source launcher allowing DeepSeek V4 Flash 0731 (EXL3) to run on a single NVIDIA DGX Spark node instead of requiring a multi-node setup.

    71bookmarks9.8kviews

    AI & LLMs· Tool

  • Repository:MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks

    Serving GLM-5.3 Flash EXL3 on Dual NVIDIA DGX Spark Kits

    Mia's AI Lab provides an optimised vLLM deployment overlay for GLM-5.3-Flash using 4 bpw EXL3 quantisation tailored for 2x NVIDIA GB10 systems.

    431starsPython

    AI & LLMs· Tool

  • Repository:eugr/spark-vllm-docker

    spark-vllm-docker: Optimised vLLM Docker Setups for DGX Spark

    spark-vllm-docker is an open-source tool providing Docker configurations and startup scripts to run vLLM on DGX Spark systems, supporting both single-node setups and multi-node clusters via Ray or PyTorch distributed mode.

    2.2kstarsShell

    AI & LLMs· Tool

  • Article:Elite Test Engineering

    Serving DeepSeek V4 Flash on Dual DGX Spark Nodes

    A detailed technical setup report documents serving DeepSeek-V4-Flash (284B MoE) across two DGX Spark nodes powered by Grace Blackwell chips using Ray and vLLM. It highlights workarounds for unified memory monitoring, NCCL RDMA deadlocks, and CUDA graph issues, as well as a 45% speedup using native speculative decoding.

    AI & LLMs· Tutorial

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.