Repository:NVIDIA/Model-Optimizer
NVIDIA Model Optimizer: Unified Library for Model Compression and Inference Acceleration
NVIDIA Model Optimizer (ModelOpt) provides state-of-the-art model optimization techniques such as quantization, pruning, distillation, and speculative decoding. It accepts PyTorch, Hugging Face, and ONNX models and outputs optimized checkpoints ready for downstream engines like TensorRT-LLM, vLLM, and SGLang.
5kstarsPython
From Video: Trending GitHub Projects: OpenBao, Magnitude, herdr, and Other Open-Source Tools
