RepositoryNousResearch/DisTrOResource
Traditional distributed large language model training relies heavily on ultra-fast, local data centre interconnects. Drastically reducing inter-GPU communication overhead opens the door to training large models over public internet connections.
DisTrO tackles the communication bottleneck in distributed deep learning. Technologically anchored in DeMo optimization and Rust, the framework is aimed at AI researchers and infrastructure engineers looking to train models outside high-bandwidth clusters. In terms of maturity, it has demonstrated scale by training models up to 40 billion parameters and deploying production code, though this repository mainly acts as a central hub linking to papers, runs, and related codebases.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Link:Nous Research Team
Nous Research has unveiled Psyche, an open infrastructure designed to train large language models across distributed, heterogeneous hardware. By combining the DisTrO optimizer with Solana-based coordination, the architecture drastically reduces communication overhead to unlock idle global compute.
Article:Kyt Dotson
AI startup Nous Research has raised $50 million in a Series A round led by Paradigm. The company leverages the Solana blockchain to coordinate distributed AI model training across contributed consumer and enterprise GPUs.
Link:Nous Research
Nous Research develops open-source language models and distributed training infrastructure. Its research focuses on model architecture, data synthesis, fine-tuning, and reasoning.
Repository:deepseek-ai/DeepSpec
DeepSpec is an open-source codebase by DeepSeek for training and evaluating draft models used in speculative decoding. The suite covers the entire pipeline across data preparation, multi-GPU model training, and performance benchmarking.
7.1kstarsPython
Repository:NVIDIA/Model-Optimizer
NVIDIA Model Optimizer (ModelOpt) provides state-of-the-art model optimization techniques such as quantization, pruning, distillation, and speculative decoding. It accepts PyTorch, Hugging Face, and ONNX models and outputs optimized checkpoints ready for downstream engines like TensorRT-LLM, vLLM, and SGLang.
5kstarsPython
Repository:eugr/spark-vllm-docker
spark-vllm-docker is an open-source tool providing Docker configurations and startup scripts to run vLLM on DGX Spark systems, supporting both single-node setups and multi-node clusters via Ray or PyTorch distributed mode.
2.2kstarsShell
Parts of this content were created with the assistance of AI.
