In @Stanford's Conversational Question Answering Challenge, machines are measured by their ability to understand text passages and answer interconnected questions. To date, Microsoft researchers are the only team to reach human parity in model performance: https://t.co/6T8LlWhOis
— Microsoft Research (@MSFTResearch) May 3, 2019
Finally, here is a gist showing how to code top-k and nucleus sampling in PyTorch:https://t.co/aDOlWLI3aq
— Thomas Wolf (@Thom_Wolf) May 3, 2019
[9/9]
Last in this recent trend of work is https://t.co/D3IOZ8CMNQ in which @universeinanegg & co show that the distribution of words in BS/greedy decoded texts is very different from the one in human texts.
— Thomas Wolf (@Thom_Wolf) May 3, 2019
Clearly BS/greedy fail to reproduce distributional aspects of human text [7/9] pic.twitter.com/7blkmtLPjB
First, there was growing evidence that beam-search is highly sensitive to the length of the output. Best results are obtained when the output length is predicted from the input before decoding (https://t.co/imPuU2t6hp, https://t.co/WkxOsscAe7 at EMNLP 2018) [4/9]
— Thomas Wolf (@Thom_Wolf) May 3, 2019
Lilian Weng of OpenAI deep dives into two prominent pre-trained language models - ULMFiT and OpenAI GPT - and describe how they work in detail. https://t.co/PAm3yLPkyO #AI #ML
— Mariya Yao (@thinkmariya) May 2, 2019
Explacy by @tylerneylon gives you a very nicely formatted dependency tree and a table including part-of-speech tags and lemmas.https://t.co/hJnqKEHqO3 pic.twitter.com/ALW7WdgvZC
— Ines Montani 〰️ (@_inesmontani) April 29, 2019
#NLProc reveals fundamental pattern of human thinking — Word Frequency Patterns show that humans process common and uncommon words differently, with important consequences for NLP: https://t.co/8nXpTida4B#BigData #DataScience #MachineLearning #AI #DataMining #TextAnalytics pic.twitter.com/XssqaOQurc
— Kirk Borne (@KirkDBorne) April 28, 2019
Finished uploading all videos for 2019 edition of CMU CS11-747 "Neural Networks for NLP": https://t.co/4qk88kWsJt
— Graham Neubig (@gneubig) April 26, 2019
Like other offerings (e.g. Stanford CS224n) it covers basics, but it's also a grad course with more topics, so it might be a good choice if you want to go deeper!
This is the most terrifying ML application I've seen in a while:https://t.co/M047AVDLh9
— Joel Grus ☕ (@joelgrus) April 23, 2019
Using Transformer models to generate Hacker News comments from titles.
— hardmaru (@hardmaru) April 23, 2019
Fun demo: https://t.co/pGfnObA9gX pic.twitter.com/j0YQIb3Uuu
“Experimental results on the PTB, WikiText-2, and WikiText-103 show that our method achieves perplexities between 20 and 34 on all problems, i.e. on average an improvement of 12 perplexity units compared to state-of-the-art LSTMs.” 🔥 https://t.co/VB7C0KdRp8
— hardmaru (@hardmaru) April 23, 2019
Transformer/LSTM hybrids!!
— Thomas Lahore (@evolvingstuff) April 23, 2019
Language Models with Transformers
"we explore effective Transformer architectures for language model, including adding additional LSTM layers to better capture the sequential context while still keeping computation efficient"https://t.co/KVWjpsACwO pic.twitter.com/4B96N4Sa57
