Tweeted By @ak92501
Early Convolutions Help Transformers See Better
— AK (@ak92501) June 29, 2021
pdf: https://t.co/5XTWUDzFag
abs: https://t.co/Faq0Yi18Bi
convolutional stem in ViT dramatically increases optimization stability and also improves peak performance (by ∼1-2% top-1 accuracy on ImageNet-1k) pic.twitter.com/q0gq67AyuF