New code walkthrough on https://t.co/m6mT8SrKDD: gradient centralization, a simple trick that can markedly speed up model convergence. Implemented in Keras in literally 10 lines of code.
— François Chollet (@fchollet) June 18, 2021
Don't miss it: https://t.co/4Gz2GamFmW pic.twitter.com/Aa3GRKliAi
We're open-sourcing XCiT, a new Transformer-based #computervision model with linear (not quadratic) complexity. XCiT, created in partnership w/ @inria researchers, processes high-res images extremely efficiently & delivers strong performance. Code & models https://t.co/7aRHfNxOp6 pic.twitter.com/ZsYgEmfM7R
— Facebook AI (@facebookai) June 18, 2021
Skip-Sideways is an efficient approximation of backprop for training video networks. It allows training on potentially infinite video sequences, being independent of the video length: https://t.co/OXG2AgKvlb (1/) pic.twitter.com/JxnJk9FzIN
— DeepMind (@DeepMind) June 18, 2021
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better https://t.co/A24eWzFrTg
— /MachineLearning (@slashML) June 18, 2021
Multi-head or Single-head? An Empirical Comparison for Transformer Training
— AK (@ak92501) June 18, 2021
pdf: https://t.co/89iVdeAWcY
abs: https://t.co/gXjRZVKr5m pic.twitter.com/p4oi7RgnIn
Scaling Laws for Acoustic Models
— AK (@ak92501) June 18, 2021
pdf: https://t.co/VUk1ufSuya
abs: https://t.co/1Cx2nsTJzD
demonstrate that acoustic models trained with an auto-predictive coding loss behave as if they are subject to similar scaling laws pic.twitter.com/w36CmGIdun
Announcing the Data-Centric AI competition! I’m excited to invite you to participate in this new competition format, and see how you can improve an AI system only by refining the data it depends on! https://t.co/9S52L3BiZv pic.twitter.com/PriTEj8Fh0
— Andrew Ng (@AndrewYNg) June 17, 2021
🤗Transformers v4.7.0 was just released with 🖼️DETR by @facebookai!
— Hugging Face (@huggingface) June 17, 2021
DETR is an Object Detection model that can take models from timm by @wightmanr as a backbone.
Contributed by @NielsRogge, try it out: https://t.co/0AOf3P7QaC
v4.7.0 launches with support for PyTorch v1.9.0! pic.twitter.com/1LAYy4cN2W
We’ve just open-sourced AugLy, a new #Python library that will help AI researchers use data augmentations to evaluate and improve the robustness of their machine learning models. Read more: https://t.co/w1FBLKMUFh pic.twitter.com/Rqrd9f8JYJ
— Facebook AI (@facebookai) June 17, 2021
Watching Too Much Television is Good: Self-Supervised Audio-Visual Representation Learning from Movies and TV Shows
— AK (@ak92501) June 17, 2021
pdf: https://t.co/hBN7Um0RMm
abs: https://t.co/cRXjYZzUhy pic.twitter.com/t56CsoEYPx
This is stellar work in collaboration with folks in Zurich. Do neural networks really make overconfident predictions in general? For new architectures, this is not true. It may not even be true consistently for CNNs! @FrancesAnnHubis @XiaohuaZhai @neilhoulsby @MarioLucic_ https://t.co/zkB1zzmzkp
— Dustin Tran (@dustinvtran) June 16, 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units by @facebookai in @PyTorch on @Gradio Hub using @huggingface models
— AK (@ak92501) June 16, 2021
paper: https://t.co/OzIO3tL351
github: https://t.co/g691cqN7Fb
gradio demo: https://t.co/46qkxR59n4 pic.twitter.com/G79R9z2X6X
