Running DeepSeek v4 Flash on a Single NVIDIA DGX Spark

X postMiaTool

MiaAI-Lab published an open-source launcher allowing DeepSeek V4 Flash 0731 (EXL3) to run on a single NVIDIA DGX Spark node instead of requiring a multi-node setup.

Key points

  1. Runs the DeepSeek V4 Flash 0731 model (3.0 bpw EXL3) on one NVIDIA DGX Spark (GB10, 128 GiB unified memory) using Tensor Parallel 1 (TP1).
  2. Employs the sparkinfer kernel stack (formerly b12x) with DSpark K5 speculative decoding and a K64 draft model.
  3. Supports long-context inference up to 384,000 tokens using a compressed NVFP4 KV cache (nvfp4_ds_mla).
  4. Validated in needle-in-a-haystack stress tests at 320k and 370k tokens with exact recall and zero preemptions.
  5. Delivers decode throughput of 44–47 tokens per second at 384k context.
  6. Provides a self-contained Docker setup with kernel backports fixing prior NVFP4 dual-cache prefill bugs.

Why it matters

While the official FP4 build required TP2 across two DGX Spark systems, this launcher halves the hardware requirement, making ultra-long context local inference possible on a single node.

Analysis

The setup operates at around 94% utilization of the host's 128 GiB unified memory. By applying kernel backports to resolve NVFP4 dual-cache prefill issues and tuning CUDA graphs, it avoids eager fallback and NaN errors during deep prefill. While prefill speeds degrade from ~1,024 tok/s to ~350–614 tok/s past 300k tokens, the test confirms deterministic recall without truncation across nearly 440,000 pooled KV tokens.

Original post

Mia

@MiaAI_lab · 3 August 2026

Run DeepSeek v4 Flash on single DGX Spark: t.co/dEO6LAray2

66 likes5 replies71 bookmarks9.8k views

View on X

Mentioned links

Links and tools from this entry

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark

    Running DeepSeek-v4-Flash on a Single DGX Spark

    An open-source launcher and Docker recipe for running DeepSeek V4 Flash 0731 (EXL3) on a single NVIDIA DGX Spark with 128 GiB unified memory.

    328starsPython

    AI & LLMs· Tool

  • Article:Elite Test Engineering

    Serving DeepSeek V4 Flash on Dual DGX Spark Nodes

    A detailed technical setup report documents serving DeepSeek-V4-Flash (284B MoE) across two DGX Spark nodes powered by Grace Blackwell chips using Ray and vLLM. It highlights workarounds for unified memory monitoring, NCCL RDMA deadlocks, and CUDA graph issues, as well as a 45% speedup using native speculative decoding.

    AI & LLMs· Tutorial

  • Repository:MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks

    Serving GLM-5.3 Flash EXL3 on Dual NVIDIA DGX Spark Kits

    Mia's AI Lab provides an optimised vLLM deployment overlay for GLM-5.3-Flash using 4 bpw EXL3 quantisation tailored for 2x NVIDIA GB10 systems.

    431starsPython

    AI & LLMs· Tool

  • X post:Mike Gannotti

    Qwen3.8-Flash-Next on a Single NVIDIA DGX Spark Outscores Dual-Spark Setup in Benchmark

    A benchmark report from SMF Clearinghouse shows MiaAI's Qwen3.8-Flash-Next (NVFP4) scoring 87.3% on smf-bench Official A running on a single NVIDIA DGX Spark (GB10). The single-node serve outperformed the previous dual-Spark DeepSeek V4 Vision-Exp setup in overall accuracy, achieving a perfect score in coding.

    17bookmarks2.3kviews

    AI & LLMs· News

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.