Tweeted By @Tim_Dettmers
The key insight is the following: In the small dataset regime, it is all about dataset augmentation. The analog in computer vision is that you get much better results, particularly on small datasets, if you do certain dataset augmentations. This also regularizes the model.
— Tim Dettmers (@Tim_Dettmers) April 8, 2020