Transformed CNNs: recasting pre-trained convolutional layers with self-attention
— AK (@ak92501) June 11, 2021
pdf: https://t.co/NNk9Ngp90o
abs: https://t.co/RM4nyVA3N6
+2.2% top-1 on ImageNet-1k for a ResNet50-RS as well as substantially improved robustness +11% top-1 on ImageNet-C pic.twitter.com/0z6GRrgjC7
AdaMatch: A Unified Approach to Semi-Supervised
— AK (@ak92501) June 10, 2021
Learning and Domain Adaptation
pdf: https://t.co/9BUJQK3SdZ
abs: https://t.co/G1AOXAopye
a general method designed to boost accuracy on domain shifts when given access to unlabeled data from the new domain pic.twitter.com/n8PCfST3ql
Happy to introduce CoAtNet: combining convolution and self-attention in a principled way to obtain better capacity and better generalization.
— Mingxing Tan (@tanmingxing) June 10, 2021
88.56% top-1 with ImageNet21K (13M imgs), matching ViT-huge with JFT (300M imgs).
Paper: https://t.co/AQE33LuzSr pic.twitter.com/YEly0cSaTp
DETReg: Unsupervised Pretraining with Region Priors for Object Detection
— AK (@ak92501) June 9, 2021
pdf: https://t.co/rLPSu27w7A
abs: https://t.co/9aaGSJeeqB
project page: https://t.co/mnCDiWBftJ
unsupervised pretraining approach for object detection with transformers using region priors pic.twitter.com/lTH36cBcvk
Shuffle Transformer: Rethinking Spatial Shuffle for
— AK (@ak92501) June 8, 2021
Vision Transformer
pdf: https://t.co/qkqHXKWpLu
abs: https://t.co/VpAssMY0Ec pic.twitter.com/JQYVlUPeoO
Your periodic reminder that @JennyBryan has collected a bunch of resources for free, open licensed images you can use for talks: https://t.co/IqCAmX9KvX
— Amelia McNamara (@AmeliaMN) June 7, 2021
When Vision Transformers Outperform ResNets without Pretraining or Strong Data Augmentations
— AK (@ak92501) June 4, 2021
pdf: https://t.co/GYknaVoNAM
abs: https://t.co/kaUxIdMVNQ
+5.3% and +11.0% top-1 accuracy on ImageNet for ViT-B/16 and MixerB/16, with the simple Inception-style preprocessing pic.twitter.com/EI1ZSUccUn
You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection
— AK (@ak92501) June 2, 2021
pdf: https://t.co/LThmFQ2a6g
abs: https://t.co/XhhOip5bOw
github: https://t.co/crirLeiGGI
series of object detection models based on the naïve Vision Transformer pic.twitter.com/Ml38kzqdtt
StyTr^2: Unbiased Image Style Transfer with Transformers
— AK (@ak92501) June 1, 2021
pdf: https://t.co/H3OraPsolh
abs: https://t.co/PM8dZiCuct pic.twitter.com/8ld0m4SDyN
Less is More: Pay Less Attention in Vision Transformers
— AK (@ak92501) June 1, 2021
pdf: https://t.co/ydo2bFvxsH
abs: https://t.co/baTSDrBpEd
hierarchical vision transformer pays less attention in
early stages to ease huge computational cost of self-attention modules over high-resolution
representations pic.twitter.com/6J5xdAO0mc
Drawing Multiple Augmentation Samples Per Image
— AK (@ak92501) May 28, 2021
During Training Efficiently Decreases Test Error
pdf: https://t.co/zwS8DT9Km0
abs: https://t.co/8xxFUi2nay
ImageNet SOTA of 86.8% top-1 accuracy after just 34 epochs of training with an NFNet-F5 using the SAM optimizer pic.twitter.com/AhDVnLSef4
CogView: Mastering Text-to-Image Generation via Transformers
— AK (@ak92501) May 28, 2021
pdf: https://t.co/SOIqFhHleW
abs: https://t.co/nmhinD6kXK
demo: https://t.co/QULIeN628P
a 4-billion-parameter Transformer with VQ-VAE tokenizer, achieves a new SOTA FID on blurred MS COCO pic.twitter.com/hvXheo5cYP
