spark-vllm-docker: Optimised vLLM Docker Setups for DGX Spark

Repositoryeugr/spark-vllm-dockerTool

spark-vllm-docker is an open-source tool providing Docker configurations and startup scripts to run vLLM on DGX Spark systems, supporting both single-node setups and multi-node clusters via Ray or PyTorch distributed mode.

Key points

  1. Optimised Docker images and launch scripts for vLLM on DGX Spark hardware.
  2. Supports single-node and multi-node cluster deployments using Ray or vLLM native PyTorch distributed mode.
  3. Networking support for InfiniBand and RDMA (NCCL), including direct dual-Spark connections, QSFP/RoCE switches, and 3-node mesh topologies.
  4. High-performance model loading via fastsafetensors and InstantTensor.
  5. Pre-built nightly runner images on DockerHub (eugr/spark-vllm:latest) alongside flags for custom wheel or source builds (vLLM, FlashInfer).
  6. Dedicated runbooks and development guidelines designed for AI agent-assisted operations.

Why it matters

Deploying distributed LLM inference across specialised accelerator nodes often requires complex tuning of inter-node networking, drivers, and runtime libraries. This project automates and standardises the setup on DGX Spark environments, accelerating time-to-deployment for multi-node inference.

Analysis

Targeted at ML infrastructure engineers running DGX Spark setups, the repository provides preconfigured environments for multi-node cluster inference. With over 2,200 stars, automated nightly testing pipelines, and explicit runbooks for automated agents, it demonstrates solid community adoption and operational maturity despite not being an official NVIDIA project.

Mentioned links

Found in

Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.

Related entries

Similar in content, found by the AI search.

  • Repository:MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark

    Running DeepSeek-v4-Flash on a Single DGX Spark

    An open-source launcher and Docker recipe for running DeepSeek V4 Flash 0731 (EXL3) on a single NVIDIA DGX Spark with 128 GiB unified memory.

    328starsPython

    AI & LLMs· Tool

  • Repository:MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks

    Serving GLM-5.3 Flash EXL3 on Dual NVIDIA DGX Spark Kits

    Mia's AI Lab provides an optimised vLLM deployment overlay for GLM-5.3-Flash using 4 bpw EXL3 quantisation tailored for 2x NVIDIA GB10 systems.

    431starsPython

    AI & LLMs· Tool

  • Link:NVIDIA

    NVIDIA DGX Spark Developer Hub: Workflows for GB10 Grace Blackwell Superchip

    NVIDIA has launched the DGX Spark Developer Hub, offering playbooks and guides for running AI agent workflows on the NVIDIA GB10 Grace Blackwell Superchip. The hub features quickstarts for tools such as Open-WebUI, ComfyUI, and vLLM alongside multi-Spark networking setups.

    AI & LLMs· Resource

  • Repository:tonyd2wild/The-Sparky-Command-Center

    Sparky Command Center: Dependency-Free Monitoring for GPU Fleets and LLM Inference

    Sparky Command Center is a lightweight, zero-dependency web dashboard designed to monitor DGX Spark fleets and custom Linux GPU rigs. It collects hardware metrics over SSH and scrapes live inference performance data from vLLM and llama.cpp without external databases.

    40starsPython

    AI & LLMs· Tool

Back to the knowledge base

Let's talk about your project

Get in touch

Locations

  • Mattersburg
    Johann Nepomuk Bergerstraße 7/2/14
    7210 Mattersburg, Austria
  • Vienna
    Ungargasse 64-66/3/404
    1030 Wien, Austria

Parts of this content were created with the assistance of AI.