RepositoryMiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-SparksTool
Running massive MoE architectures locally demands extreme memory efficiency and low-level kernel optimisations. This overlay demonstrates how tailored GPU-driven execution and speculative decoding can make inference viable on dual GB10 systems.
The project serves AI infrastructure teams and practitioners running local large-scale inference on NVIDIA GB10 hardware. By replacing host loops with custom CUDA routines and tuning speculative draft attention, it achieves high prefill efficiency. It operates as an advanced deployment recipe with curated container images and benchmark data.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Repository:MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark
An open-source launcher and Docker recipe for running DeepSeek V4 Flash 0731 (EXL3) on a single NVIDIA DGX Spark with 128 GiB unified memory.
328starsPython
X post:Mia
MiaAI-Lab published an open-source launcher allowing DeepSeek V4 Flash 0731 (EXL3) to run on a single NVIDIA DGX Spark node instead of requiring a multi-node setup.
71bookmarks9.8kviews
Repository:eugr/spark-vllm-docker
spark-vllm-docker is an open-source tool providing Docker configurations and startup scripts to run vLLM on DGX Spark systems, supporting both single-node setups and multi-node clusters via Ray or PyTorch distributed mode.
2.2kstarsShell
Article:Elite Test Engineering
A detailed technical setup report documents serving DeepSeek-V4-Flash (284B MoE) across two DGX Spark nodes powered by Grace Blackwell chips using Ray and vLLM. It highlights workarounds for unified memory monitoring, NCCL RDMA deadlocks, and CUDA graph issues, as well as a 45% speedup using native speculative decoding.
Article:Inferact
This guide details running vLLM on NVIDIA's DGX Spark (GB10) desk-side workstation. Leveraging its 128 GB unified memory pool, the system is designed for local inference with NVFP4 MoE models such as Nemotron-3-Super.
X post:Mia
Mia (@MiaAI_lab) reports running Inkling-Small across two NVIDIA DGX Spark systems with a 1M token context window using SGLang and dspark speculative drafts.
106bookmarks9.5kviews
Parts of this content were created with the assistance of AI.
