RepositoryTencent/AngelSlimTool
Running massive frontier models is financially and logistically challenging due to memory and compute bottlenecks. AngelSlim provides practical tools to dramatically reduce model footprint and accelerate token generation, enabling deployment of massive multi-hundred-gigabyte models across standard or mixed GPU setups.
AngelSlim addresses inference latency and memory footprints by combining multiple compression paradigms under one roof. Rather than focusing solely on post-training quantization, it integrates prefill acceleration (Stem sparse attention), disaggregated speculative decoding (AngelSpec), and Megatron-Core distillation workflows. It is aimed at ML engineers, systems researchers, and teams deploying large models in production. With over 1,600 GitHub stars, accompanying research papers, and regular updates for cutting-edge architectures like Hy4 and Qwen3-MoE, the repository demonstrates active maintenance and practical engineering maturity.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Repository:NVIDIA/Model-Optimizer
NVIDIA Model Optimizer (ModelOpt) provides state-of-the-art model optimization techniques such as quantization, pruning, distillation, and speculative decoding. It accepts PyTorch, Hugging Face, and ONNX models and outputs optimized checkpoints ready for downstream engines like TensorRT-LLM, vLLM, and SGLang.
5kstarsPython
Link:Tencent Hy Team
The Tencent Hy Team has released Hy4-preview, an open-source Mixture-of-Experts (MoE) model featuring 770B total parameters and 49B activated parameters per token. The flagship model supports a 1M token context window and is tuned for software engineering, office analytics, and scientific research.
Repository:Tencent-Hunyuan/Hy4-preview
The Tencent Hy Team has introduced Hy4-preview, a new flagship Mixture-of-Experts (MoE) model featuring 770B total parameters with 49B activated per token. The architecture incorporates MoE routing, Gated DeepSeek Sparse Attention, and a native MTP layer for speculative decoding.
344starsPython
Repository:deepseek-ai/DeepSpec
DeepSpec is an open-source codebase by DeepSeek for training and evaluating draft models used in speculative decoding. The suite covers the entire pipeline across data preparation, multi-GPU model training, and performance benchmarking.
7.1kstarsPython
X post:Hamid Maei
PrismML has announced Ternary Bonsai 2 27B, a compressed version of Qwen3.8 27B. It is nine times smaller than the full-precision original, requiring only 5.9 GB of storage while retaining 98.2% of aggregate benchmark performance.
97bookmarks21kviews
Repository:MiaAI-Lab/GLM-5.3-Flash-EXL3-2x-DGX-Sparks
Mia's AI Lab provides an optimised vLLM deployment overlay for GLM-5.3-Flash using 4 bpw EXL3 quantisation tailored for 2x NVIDIA GB10 systems.
431starsPython
Parts of this content were created with the assistance of AI.
