Pre-Trained Models: Past, Present and Future
— AK (@ak92501) June 15, 2021
pdf: https://t.co/8X6QC2ohp5
abs: https://t.co/CRqSZElwua pic.twitter.com/PPzScQr418
Pre-Trained Models: Past, Present and Future
— AK (@ak92501) June 15, 2021
pdf: https://t.co/8X6QC2ohp5
abs: https://t.co/CRqSZElwua pic.twitter.com/PPzScQr418
Break-It-Fix-It: Unsupervised Learning for Program Repair
— AK (@ak92501) June 15, 2021
pdf: https://t.co/rs199G6b7M
abs: https://t.co/rjRD6m9fWg
outperforms sota methods, obtaining 90.5% repair accuracy on GitHub Python (+28.5%) and 71.7% on DeepFix (+5.6%) pic.twitter.com/c7AtQWMVpp
Styleformer: Transformer based Generative Adversarial Networks with Style Vector
— AK (@ak92501) June 15, 2021
pdf: https://t.co/jNVLty3unL
abs: https://t.co/SEK0ko63E7
github: https://t.co/hQanKidsZ8
outperforms GAN-based generative models, including StyleGAN2-ADA with fewer parameters on CIFAR-10 pic.twitter.com/bs3JmTJtdz
MlTr: Multi-label Classification with Transformer
— AK (@ak92501) June 14, 2021
pdf: https://t.co/wqvd89AtJq
abs: https://t.co/H2n64N5OGa pic.twitter.com/yXFM5caLMK
SimSwap: An Efficient Framework For High Fidelity
— AK (@ak92501) June 14, 2021
Face Swapping
pdf: https://t.co/l2aWTrM1CP
abs: https://t.co/ZSuDnRLUuF
github: https://t.co/deYKr8rhLY pic.twitter.com/cBuaXySkd9
MuZero removed simulators in MBRL vs AlphaGo. VQ Models for Planning generalize to partial observable & stochastic environments. How?
— Oriol Vinyals (@OriolVinyalsML) June 11, 2021
1. Discretize states w/ VQVAE
2. Train a LM over states
3. Plan w/ MCTS using the LM
Led by @yazhe_li & @sherjilozair https://t.co/thvB6Ke1EA pic.twitter.com/tsXGcrweTZ
Today, we’re introducing TextStyleBrush, the first self-supervised AI model that replaces text in existing images of both scenes and handwriting — in one shot — using just a single example word: https://t.co/0QfLraAQvV pic.twitter.com/FNDJxNC20S
— Facebook AI (@facebookai) June 11, 2021
Transformed CNNs: recasting pre-trained convolutional layers with self-attention
— AK (@ak92501) June 11, 2021
pdf: https://t.co/NNk9Ngp90o
abs: https://t.co/RM4nyVA3N6
+2.2% top-1 on ImageNet-1k for a ResNet50-RS as well as substantially improved robustness +11% top-1 on ImageNet-C pic.twitter.com/0z6GRrgjC7
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training
— AK (@ak92501) June 11, 2021
pdf: https://t.co/nvO4RUj540
abs: https://t.co/vjEeRlX9Ud
a largescale pre-trained model for symbolic music understanding, sota performance on four evaluated symbolic music understanding tasks pic.twitter.com/cF7I2vwjx0
The development of hundreds AI/ML/Stats algorithms for diagnosing and prognosticating COVID-19 patients has been a disaster (with very, very few exceptions). Hopefully the worst example of research waste I will ever see pic.twitter.com/IFRew5OCZb
— Maarten van Smeden (@MaartenvSmeden) June 10, 2021
AdaMatch: A Unified Approach to Semi-Supervised
— AK (@ak92501) June 10, 2021
Learning and Domain Adaptation
pdf: https://t.co/9BUJQK3SdZ
abs: https://t.co/G1AOXAopye
a general method designed to boost accuracy on domain shifts when given access to unlabeled data from the new domain pic.twitter.com/n8PCfST3ql
Happy to introduce CoAtNet: combining convolution and self-attention in a principled way to obtain better capacity and better generalization.
— Mingxing Tan (@tanmingxing) June 10, 2021
88.56% top-1 with ImageNet21K (13M imgs), matching ViT-huge with JFT (300M imgs).
Paper: https://t.co/AQE33LuzSr pic.twitter.com/YEly0cSaTp