Restormer: Efficient Transformer for High-Resolution Image Restoration
— AK (@ak92501) November 19, 2021
abs: https://t.co/ioFgvW3TcA
propose an efficient Transformer model by making several key designs in the building blocks such that it can capture long-range pixel interactions pic.twitter.com/Ud1YQlVyAG
Swin Transformer V2: Scaling Up Capacity and Resolution
— AK (@ak92501) November 19, 2021
abs: https://t.co/vBm66uyUBZ
scaling Swin Transformer up to 3 billion parameters and making it capable of training with images of up to 1,536×1,536 resolution, sets new records on four representative vision benchmarks pic.twitter.com/2lvEvCOZ35
XLS-R is our new model for cross-lingual self-supervised speech learning trained on 436K hours of public speech in 128 languages and with up to 2B params.
— Michael Auli (@MichaelAuli) November 18, 2021
Paper: https://t.co/o0HuU7azeu
Blog: https://t.co/w9Ta8ZCLj5
Code/models: https://t.co/hIUFJLPupbhttps://t.co/jjZrgoz686 pic.twitter.com/iENwHgoTuj
(Of course) XGBoost does not always win on tabular datasets. Made a HW where students got to tinker w. hyperparam tuning techniques (grid search, randomized search, Hyperopt, Optuna, successive halving) and algos (GBMs and everything in scikit-learn + mlxtend). Top-10 results: pic.twitter.com/aq88Uw5dkF
— Sebastian Raschka (@rasbt) November 16, 2021
Great paper and thread!
— Andrej Karpathy (@karpathy) November 13, 2021
- 😮that super simple MSE loss works vs. BEiT-style dVAE (multi-modal) cross-entropy
- <3 efficiency of asymmetric encoder/decoder
- 👏detailed training recipes
- +1 v curious about dataset size scaling
- bit of lack of commentary on test-time protocol https://t.co/MQFAvrqBvr
Dense Unsupervised Learning for Video Segmentation
— AK (@ak92501) November 12, 2021
abs: https://t.co/zKV94Pn7s5
github: https://t.co/8WQyRw3jw9 pic.twitter.com/nST5odpRMU
Are Transformers More Robust Than CNNs?
— AK (@ak92501) November 11, 2021
abs: https://t.co/UIPYvRPTrk pic.twitter.com/M3KojPHARj
Data Augmentation Can Improve Robustness
— AK (@ak92501) November 10, 2021
abs: https://t.co/9Tt4vZUcy3
github: https://t.co/VwXyAQWPeo
against ℓ∞ norm-bounded perturbations of size ϵ=8/255, model reaches 60.07% robust accuracy without using any external data pic.twitter.com/S6gMbh7WZh
NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework
— AK (@ak92501) November 9, 2021
abs: https://t.co/DqwSqfuvgn pic.twitter.com/6lCs6HmLKh
The Deep Learning Devil is in the Details. I love this work from @IrwanBello and collaborators in which they show how training "tricks" improve ~3% absolute accuracy on ImageNet, progress equivalent to years of developments and research!
— Oriol Vinyals (@OriolVinyalsML) November 5, 2021
Paper: https://t.co/oA6MDfq1XB pic.twitter.com/auvM5yrt01
Today on the AI2 Blog: An overview of our model Delphi as a nascent step toward machine ethics, a look into the stress testing of this research prototype, the lessons learned, and reflection on why working on imperfect solutions is better than inaction:https://t.co/ObjUwdVD5N
— Allen Institute for AI (@allen_ai) November 4, 2021
VLMO: Unified Vision-Language Pre-Training with
— AK (@ak92501) November 4, 2021
Mixture-of-Modality-Experts
abs: https://t.co/Rv9o8aFIdI
introduce Mixture-of-Modality-Experts Transformer,
where each block contains a pool of modality-specific experts and a shared self attention layer pic.twitter.com/4k0YFlvgsR
