Thinking Like Transformers
— hardmaru (@hardmaru) June 16, 2021
RNNs have direct parallels in finite state machines, but Transformers have no such familiar parallel. This paper aims to change that. They propose a computational model for the Transformer in the form of a programming language.https://t.co/OuPBSrS1EJ pic.twitter.com/zYyAcH2zQd
Keep CALM and Improve Visual Feature Attribution
— AK (@ak92501) June 16, 2021
pdf: https://t.co/I90yEPbjse
abs: https://t.co/JKIlsnNnGn
github: https://t.co/MP1F8LXvY0
identifies discriminative attributes for image classifiers more accurately than CAM and other visual attribution baselines pic.twitter.com/50DBMoVYai
We just released the multilingual C4 dataset! 1.4TB of German? 1.6TB of Spanish? 3.6TB of Russian 😲? We got it all. Check out the announcement at https://t.co/T6wcPDl5O7!
— AllenNLP (@ai2_allennlp) June 16, 2021
BEIT: BERT Pre-Training of Image Transformers
— AK (@ak92501) June 16, 2021
pdf: https://t.co/WiFZIiErLt
abs: https://t.co/Ld2067ltiV
large-size BEIT obtains 86.3% only using ImageNet-1K, even outperforming ViT-L with supervised
pre-training on ImageNet-22K (85.2%) pic.twitter.com/abMaWZ1aZ8
I haven't written anything in *a while* because I've been heads down at @noteable_io so it's great to finally get back to it and get into a topic that I've been obsessing over the last year (and longer outside of startup-land): How we've siloed ourselves.https://t.co/YZFXbE0cE5
— Elijah Meeks (@Elijah_Meeks) June 15, 2021
Pre-Trained Models: Past, Present and Future
— AK (@ak92501) June 15, 2021
pdf: https://t.co/8X6QC2ohp5
abs: https://t.co/CRqSZElwua pic.twitter.com/PPzScQr418
Break-It-Fix-It: Unsupervised Learning for Program Repair
— AK (@ak92501) June 15, 2021
pdf: https://t.co/rs199G6b7M
abs: https://t.co/rjRD6m9fWg
outperforms sota methods, obtaining 90.5% repair accuracy on GitHub Python (+28.5%) and 71.7% on DeepFix (+5.6%) pic.twitter.com/c7AtQWMVpp
Styleformer: Transformer based Generative Adversarial Networks with Style Vector
— AK (@ak92501) June 15, 2021
pdf: https://t.co/jNVLty3unL
abs: https://t.co/SEK0ko63E7
github: https://t.co/hQanKidsZ8
outperforms GAN-based generative models, including StyleGAN2-ADA with fewer parameters on CIFAR-10 pic.twitter.com/bs3JmTJtdz
pytorch-widedeep, deep learning for tabular data IV: Deep Learning vs LightGBM by Javier Rodriguez Zaurin in @TDataScience https://t.co/7HtGdDBLhm
— Jeremy Howard (@jeremyphoward) June 14, 2021
MlTr: Multi-label Classification with Transformer
— AK (@ak92501) June 14, 2021
pdf: https://t.co/wqvd89AtJq
abs: https://t.co/H2n64N5OGa pic.twitter.com/yXFM5caLMK
SimSwap: An Efficient Framework For High Fidelity
— AK (@ak92501) June 14, 2021
Face Swapping
pdf: https://t.co/l2aWTrM1CP
abs: https://t.co/ZSuDnRLUuF
github: https://t.co/deYKr8rhLY pic.twitter.com/cBuaXySkd9
This map of 95,647 (pre-pandemic) flight routes shows a view of the world based on population density and economic power. Source: https://t.co/mOgbzT1FUP pic.twitter.com/NJjAq72JL1
— Simon Kuestenmacher (@simongerman600) June 13, 2021
