"Language models as knowledge bases?" they asked: https://t.co/O7BCcy0V6r
— Graham Neubig (@gneubig) September 26, 2019
"A cat has four kidneys", replied GPT-2. pic.twitter.com/yMizwyOSpU
"Language models as knowledge bases?" they asked: https://t.co/O7BCcy0V6r
— Graham Neubig (@gneubig) September 26, 2019
"A cat has four kidneys", replied GPT-2. pic.twitter.com/yMizwyOSpU
📢 @GitHub is releasing a large dataset for natural language processing and machine learning. It's a large parallel corpus of code and natural language, with benchmarks on IR tasks. Leaderboard is hosted by @weights_biases: https://t.co/rSKbvkfX4S
— Hamel Husain (@HamelHusain) September 26, 2019
I'm still looking through the enormous number of interesting NLP submissions to #ICLR2020, but I'm really excited to see *two* new pretraining methods that outperform XLNet/RoBERTa on NLU tasks with far fewer parameters/FLOPS: https://t.co/j2IarJd2lC https://t.co/O6VnnhuDsm
— Sam Bowman (@sleepinyourhat) September 26, 2019
HuggingFace has ported their very popular Transformer library to TensorFlow 2.0 / Keras.
— François Chollet (@fchollet) September 26, 2019
33 state-of-the-art pretrained NLP models (8 architectures) for 102 languages.
The TF 2.0 conversion unlocks TPU support, TF.js export, TFX serving, and more.https://t.co/wpqouBJ8Lw pic.twitter.com/tXqL0pJEim
Install TensorFlow 2.0 and/or PyTorch, and just
— Thomas Wolf (@Thom_Wolf) September 26, 2019
*pip install transformers*
with @LysandreJik @julien_c @SanhEstPasMoi @ClementDelangue and all the @huggingface team
h/t: @fchollet @random_forests
Repo: https://t.co/iNdvPcRFPa
Release notes: https://t.co/lPJAV5cIJx
🤗Transformers 2.0💥
— Thomas Wolf (@Thom_Wolf) September 26, 2019
State-of-the-art NLP in TensorFlow 2.0/PyTorch
8 architectures
33 trained models
102 lang.
Seamlessly pick the right framework for training, eval, deploy
Train on TPU ⏩ finetune/test in PyTorch ⏩ serve w. TFX
🍒Keras magic: train SOTA model in 10 lines👇 pic.twitter.com/K7BNdxDBQh
I've been trying to use a language generation model (GPT-2) to make sketches.
— Robbie Barrat (@DrBeef_) September 26, 2019
The process is heavily inspired by Sol LeWitt's - I use GPT-2 to generate a set of rules describing a drawing; then based on to my interpretation of those rules; I make a processing sketch. pic.twitter.com/cVasjUGrd5
Our two-step, self-supervised approach to extractive question answering (QA) first trains a model to generate questions, then uses those questions to train a standard extractive QA model.https://t.co/FDSiMAyhuo pic.twitter.com/7JQStT0915
— Facebook AI (@facebookai) September 24, 2019
Notebooks implementing real-time trigger word detection in Keras. Neat work :) https://t.co/m9K26H06ss
— François Chollet (@fchollet) September 23, 2019
One of the biggest tool gaps in ML right now is tin building utilities to more easily inspect and understand data.
— Emmanuel Ameisen (@mlpowered) September 20, 2019
I gave a talk about just this at:https://t.co/0MxmhAF3zl
It also quotes your great data in industry vs data in academia slide in the conclusion @karpathy https://t.co/DxLqUIDXam
To be noticed: the French Bidirectional LM trained with a #QRNN architecture and the #SentencePiece tokenizer got better performance than the one with #AWDLSTM architecture and the #spaCy tokenizer. All notebooks/models parameters/vocab on line at https://t.co/vQl8GO2Lbk
— Pierre Guillou (@pierre_guillou) September 20, 2019
We've fine-tuned GPT-2 using human feedback for tasks such as summarizing articles, matching the preferences of human labelers (if not always our own). We're hoping this brings safety methods closer to machines learning values by talking with humans. https://t.co/ok9jeMP5zj
— OpenAI (@OpenAI) September 19, 2019