Powerful LMs such as GPT-3 and T5 have impressive ability to answer questions by continuing a textual prompt. However, how can we know when an LM knows the answer with confidence, and when it's making a random guess? Our new preprint asks this: https://t.co/Gupb7WP9zS 1/N pic.twitter.com/aahY9P4hZU
— Graham Neubig (@gneubig) December 2, 2020
I was made aware of two papers that are similar and preceded both OpenAI papers. I think these add more data points to scaling behavior for language (and also vision). These should be shared more widely! https://t.co/ZCXYCt3DgN https://t.co/QNn8KznjXe
— Tim Dettmers (@Tim_Dettmers) November 29, 2020
I succumbed to threats and wrote my 2019–20 faculty report. Hot papers last year: Electra: Pre-training text encoders as discriminators https://t.co/UXrVN5dDEx, Stanza: A Python toolkit for many languages https://t.co/ZEHuBQPQcp & Universal Dependencies v2 https://t.co/xqZSwCcpJZ pic.twitter.com/Y3K36qVaiv
— Christopher Manning (@chrmanning) November 29, 2020
I am curious why people are not talking more about the OpenAI scaling law papers. For me, they seem very significant. What I heard so far: "Too complicated. I don't understand and I don't care", "NLP is not physics". Other criticism? Any insights why people ignore it?
— Tim Dettmers (@Tim_Dettmers) November 28, 2020
Want to know if a 🤗Transformers model has a slow/fast tokenizer or an implementation in your favorite backend? A new table (auto-magically updated) just made its way in the docs! pic.twitter.com/pEj9OIAYbU
— Sylvain Gugger (@GuggerSylvain) November 25, 2020
Any JAX/Flax lovers out there? Ever wanted to use 🤗 Transformers with all the awesome features of JAX? Well you're in luck! 😍
— Hugging Face (@huggingface) November 24, 2020
We've worked with the Google Flax team to enable support for BERT and RoBERTa! 🚀
Check it out 👇https://t.co/zdNUqabOLw pic.twitter.com/qMakbVzbxj
I posted my #SustaiNLP2020 slides on "More is Less? Non-parametric Language Models and Efficiency" (https://t.co/N9wynDRcHB), where I overview and survey non-parametric models in NLP, and introduce our new work on sparse prototypes for text generation (https://t.co/BhNTr9zkM2)! https://t.co/cELvzdMDe7 pic.twitter.com/AMz0GaUEWZ
— Graham Neubig (@gneubig) November 20, 2020
The powerful mT5 model including all pre-trained models is now part of Transformers.
— Hugging Face (@huggingface) November 20, 2020
mT5 was trained on 101 languages and yields SOTA on many multilingual tasks.
Thanks, @lintingxue, Noah Constant, @ada_rob, @colinraffel for releasing the weights!
➡️ https://t.co/sWdM7VNeLw. pic.twitter.com/mOj9hEkpCG
These tutorial slides on "High Perf NLP" are really impressive. Every slide is current to the minute. Amazing set of diagrams.https://t.co/o0o4SY6chR
— Sasha Rush (@srush_nlp) November 19, 2020
(@gabriel_ilharco @Tim_Dettmers @IuliaTurc @kentonctlee Felipe Ferreira Cesar Ilharco) pic.twitter.com/TdfVx9lBsU
If you like ELECTRA, check out Electric, our new #emnlp2020 work that brought over energy-based model perspective ;) Electric produces fast pseudo-likelihood to improve speech recognition & translation reranking. Github: https://t.co/ErnTR4llRL & paper https://t.co/N2DzPJ3ksb pic.twitter.com/UfPYPnUDXS
— Thang Luong (@lmthang) November 16, 2020
These text-to-image results look pretty cool.https://t.co/CPkC56bxst https://t.co/TbAaYZ9pwl pic.twitter.com/GNShK9NSmp
— hardmaru (@hardmaru) November 10, 2020
A very nice notebook/blog-post from @PatrickPlaten on how to create and train (to really nice results) an Encoder-Decoder model using pre-trained Bert/Roberta/GPT2/etc models as encoders and as decoders 👇 https://t.co/bZAUTaDv5g
— Thomas Wolf (@Thom_Wolf) November 9, 2020
