PyTorch BERT models are now 4x faster, thanks to @nvidia https://t.co/enSldGFAAC
— PyTorch (@PyTorch) December 14, 2018
PyTorch BERT models are now 4x faster, thanks to @nvidia https://t.co/enSldGFAAC
— PyTorch (@PyTorch) December 14, 2018
So it turns out that Language Modelling with Approximate Outputs (LMAO) idea I've been working on for spaCy works well! So well that the work must have been finished before I even started 🤷♂️Great paper by Kumar and Tsevtkov 👉 https://t.co/TLJ9a42c0K
— Matthew Honnibal (@honnibal) December 14, 2018
Combating the spread of 'fake news' and 'influencer fraud' - Researchers show how data science techniques can find Twitter ‘amplification bots’ by @matthewhughes https://t.co/CbWNdXRY1u pic.twitter.com/KtxlFTNyVe
— Kaggle (@kaggle) December 12, 2018
This is a *great* walkthru of implementing a new research paper from scratch with fastai and @PyTorch . And the results are impressive - fast and accurate NLP models! :) https://t.co/q0kHF404Tb
— Jeremy Howard (@jeremyphoward) December 11, 2018
"What are the 3 biggest open problems in NLP?"
— Sebastian Ruder (@seb_ruder) December 11, 2018
We had asked experts a few simple but big questions for the NLP session at the @DeepIndaba. We're now happy to share the full responses from Yoshua Bengio, @redpony, @RichardSocher and many othershttps://t.co/0vk3ot3Hav pic.twitter.com/Xy9L1TWlk1
Flair – a very simple framework for state-of-the-art NLP.
— ML Review (@ml_review) December 10, 2018
By @ZalandoTech, @alan_akbik
– Powerful NLP library. SoTA NER, PoS, sense disambiguation & classification.
– Text embedding library
– Builds directly on Pytorchhttps://t.co/ykQTFMcD99 pic.twitter.com/mEFKdSxB5K
Learning More with Less by @DataLives https://t.co/hYMQN3DDZ4
— Jeremy Howard (@jeremyphoward) December 10, 2018
Great to see folks doing additional ablation experiments beyond those we already did in the ULMFiT paper. This work shows that even just a little unlabeled data helps a lot!
The "Cross-View Training" paper by Clark et al came out two weeks before BERT, and I think it's got lost in the noise. It's worth paying attention to: their self-training trick is very elegant, and seems easy to apply and extend. https://t.co/0kjTCuun00
— Matthew Honnibal (@honnibal) December 10, 2018
Forward & Backward Character Language Models for Conjoined Word Separation with fastai by Martin Boyanov https://t.co/t6RO6RMc6B
— Jeremy Howard (@jeremyphoward) December 6, 2018
code2vec: Learning Distributed Representations of Code #POPL2019
— ML Review (@ml_review) December 6, 2018
By @urialon1@omerlevy_ @yahave
Demohttps://t.co/OvpSLOtTCp
ArXivhttps://t.co/ZQsImneOsg
Githubhttps://t.co/ylECFYOhwn pic.twitter.com/30qn1hUvTW
The LM (a sequence model) used in ULMFit was trained on an English corpus with stopwords intact. So by throwing away the stopwords you’re creating (or worsening) a covariate shift.
— Delip Rao (@deliprao) November 30, 2018
Why? The word “the”, for e.g, might appear in all documents. Similarly “a”, “an” ... As a consequence the inverted index blows up in size. And not just the construction cost, but also the retrieval cost goes up. Simple solution from the 70s: just drop the high frequency words.
— Delip Rao (@deliprao) November 30, 2018