Tweeted By @PyTorch
We just assessed the effectiveness of the DDP, Pipe and FSDP distributed training techniques available via PyTorch with different model sizes and network configurations. See the results here. https://t.co/vN191zRclh
— PyTorch (@PyTorch) March 17, 2022