VideoVideo:CURT
Practical Agent RL: Stabilising Async Training Across Heterogeneous GPUs
Jason Mancuso from Modal argues that treating LLMs as swappable backends is an uncompetitive strategy for production agents. Teams already possess the necessary RL ingredients: harnesses, evaluations, and production traces. Using GRPO, masked importance sampling, and score centering, asynchronous RL remains stable across fragmented GPUs and quantisation levels, cutting rollout costs by 15%.