Wow.@OpenAI has trained a model that can create new images - you just describe them in English. 🤯https://t.co/qxDreBM2Qx pic.twitter.com/TWYwkfeblY
— Jeremy Howard (@jeremyphoward) January 5, 2021
Wow.@OpenAI has trained a model that can create new images - you just describe them in English. 🤯https://t.co/qxDreBM2Qx pic.twitter.com/TWYwkfeblY
— Jeremy Howard (@jeremyphoward) January 5, 2021
Amazing to see progress in adapting language models to unseen languages, like Tibetan script.https://t.co/sM8orQshL9 https://t.co/bgduV56Wtq
— hardmaru (@hardmaru) January 4, 2021
Exciting new dataset, and don't miss the appendix for this 🔥 https://t.co/Eeb2ZSVthP pic.twitter.com/1Df8akE1J2
— Jacob Eisenstein (@jacobeisenstein) January 2, 2021
By that measure, MSR's model is somewhat better than T5 or RoBERTa, but it still falls back on stereotypes substantially more often than humans. We know that LMs pick up stereotypes from their training data, and that's not something we can easily counteract. Proceed with caution.
— Prof. Sam Bowman (@sleepinyourhat) December 30, 2020
More progress on the SuperGLUE NLU leaderboard (https://t.co/ipzFoqJiyU), from an MSR team including @AllenLao and @JianfengGao0217, with a larger version of their DeBERTa: https://t.co/8WzBjgk16q pic.twitter.com/PLjVVAxCrO
— Prof. Sam Bowman (@sleepinyourhat) December 30, 2020
- How Transformers work in deep learning and NLP: an intuitive introduction https://t.co/3jCQ302heb
— /MachineLearning (@slashML) December 26, 2020
🤗Transformers are starting to work with structured databases!
— Hugging Face (@huggingface) December 17, 2020
We just released 🤗Transformers v4.1.1 with TAPAS, a multi-modal model for question answering on tabular data from @googleAI.
Try it out through transformers or our inference API: https://t.co/cJWxi7mB68 pic.twitter.com/s0oU0UFwW8
"Transformer are .. more effective at machine translation than RNN models, but ... most of these quality gains were from the transformer encoder, and that the transformer decoder was not significantly better than the RNN decoder."https://t.co/lGiWB9abZN
— Sasha Rush (@srush_nlp) December 17, 2020
(not sure this is CW)
[XLSR-53: Multilingual Self-Supervised Speech Transformer]
— Alexis Conneau (@alex_conneau) December 17, 2020
We're happy to release XLSR-53: a wav2vec 2.0 model pre-trained on 56k hours of speech in 53 languages from MLS, CommonVoice and BABEL datasets!
Model: https://t.co/WjOOO8k3yd
Updated paper: https://t.co/zyz4Z35lKn
1/N
New Preprint: Diff Pruning (https://t.co/c4yTd7s47W) by Demi Guo / Yoon Kim Lab (😀)
— Sasha Rush (@srush_nlp) December 15, 2020
How many BERT parameters do you really need to change during fine-tuning? Turns out the answer is 0.5%
Allows new task adaption by shipping extremely small param diff's pic.twitter.com/ZkSQfvBUvg
🥳 We now have Tsinghua's CPM, a Chinese GPT-3-like model with 2.6B parameters available from 🤗 model hub! 来试试吧！https://t.co/jnw0VH3mqG@Tsinghua_Uni @zibuyu9
— Hugging Face (@huggingface) December 11, 2020
A bunch of researchers in China have built 'CPM', a Chinese language GPT-style model. Paper here: https://t.co/frVcm7TNhC It's notable to not see analysis of bias (when compared to LM papers in the West). Can people point me to good studies of Chinese language bias?
— Jack Clark (@jackclarkSF) December 3, 2020