Repository:NousResearch/DisTrO
DisTrO: Distributed AI Training Over The Internet
DisTrO (Distributed Training Over-The-Internet) by Nous Research is a family of low-latency distributed optimizers designed to reduce inter-GPU communication requirements by three to four orders of magnitude.
1.1kstars