🏷️ DiLoCo

Decoupled DiLoCo: Making Large-Scale AI Training More Resilient

Google DeepMind introduces Decoupled DiLoCo, a method that decouples synchronization dependencies in distributed trainin…