Large NLP models are expensive and opaque, but maybe it doesn't have to be that way. This exciting work with Omar Khattab and @ChrisGPotts uses retrieval to set SotA results in hard NLP tasks at low cost. Our Baleen paper will be a spotlight at NeurIPS. https://t.co/vGp0YD52CU
— Matei Zaharia (@matei_zaharia) October 6, 2021
Microsoft Turing Universal Language Representation model, T-ULRv5, has topped the XTREME leaderboard for evaluating cross-lingual generalization. Learn how recent advances enable T-ULRv5 to not only excel beyond previous models, but also train 100x faster: https://t.co/gX4ooVYGFX
— Microsoft Research (@MSFTResearch) September 28, 2021
FewNLU: Benchmarking State-of-the-Art Methods
— AK (@ak92501) September 28, 2021
for Few-Shot Natural Language Understanding
pdf: https://t.co/IhdNEnGWo8
abs: https://t.co/YO2hqCTWnk pic.twitter.com/eF7icGRVbA
Amusing! Object detection cast naively into language modeling framework + borrowing many of the tips&tricks.
— Andrej Karpathy (@karpathy) September 24, 2021
- random object ordering seems fine ✅
- coords, class labels flattened into a single softmax 😂
- sequence augmentation is the most gnarly part, almost as yucky as nms 😬 https://t.co/FxSz5UbpxY
Have you wondered why object detection, unlike classification, has so many sophisticated algorithms?
— Ting Chen (@tingchenai) September 23, 2021
With Pix2Seq (https://t.co/ygsG3aAIbG), we simply cast object detection as a language modeling task conditioned on pixels!
(with @srbhsxn, Lala Li, @fleet_dj, @geoffreyhinton) pic.twitter.com/aTYZ5IvJc9
Primer: Searching for Efficient Transformers for Language Modeling
— AK (@ak92501) September 20, 2021
abs: https://t.co/JM9v7pNoSI
github: https://t.co/xhA7uGyC7H
Experiments show Primer’s gains over Transformer increase as compute scale grows and follow a power law with respect to quality at optimal model sizes pic.twitter.com/CXq1yYMfUA
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA
— AK (@ak92501) September 13, 2021
abs: https://t.co/A3s6zdW0Iu
a simple yet effective method that Prompts GPT3 via the use of Image Captions. Using only 16 examples, PICa surpasses the supervised sota by an absolute +8.6 points on the OK-VQA dataset pic.twitter.com/9HKybsk1qu
Efficient Nearest Neighbor Language Models
— AK (@ak92501) September 10, 2021
pdf: https://t.co/uEtibkYA1L
abs: https://t.co/K0hBVhtvlk pic.twitter.com/ZKd0vnBDCs
Datasets: A Community Library for Natural Language Processing
— AK (@ak92501) September 8, 2021
abs: https://t.co/xEpY9oQ2a5
github: https://t.co/HvY6Nlf41c
650+ unique datasets, 250+ contributors, and has helped support a variety of novel crossdataset research projects and shared tasks pic.twitter.com/AdlB21Hu2c
PermuteFormer: Efficient Relative Position Encoding for Long Sequences
— AK (@ak92501) September 7, 2021
abs: https://t.co/S0bSxCDoc2
experiments show that PermuteFormer uniformly improves the performance of Performer with almost no computational overhead and outperforms vanilla Transformer on most of the tasks pic.twitter.com/PcmcRPrOtC
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
— AK (@ak92501) September 3, 2021
pdf: https://t.co/Fk81CFNMBa
abs: https://t.co/vyIHZQsPLK pic.twitter.com/DvvUQIzRuQ
If you ever find yourself in a situation where you need to evaluate whether a word or sentence is plausible English according to GPT-2, lm-scorer does the job nicely https://t.co/DPUqQyQNvh
— Brendan Dolan-Gavitt (@moyix) September 2, 2021
