Tweeted By @PyTorch
TorchShard is a lightweight engine for slicing a PyTorch tensor into parallel shards. It can reduce GPU memory and scale up the training when the model has massive linear layers or huge classes. Read more below:https://t.co/DBLXYTCmEO
— PyTorch (@PyTorch) July 16, 2021