Tweeted By @srush_nlp
"When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute" by Tao Lei @taolei15949106 - Outstanding Paper at EMNLP https://t.co/7IR25d9Sz2
— Sasha Rush (@srush_nlp) October 30, 2021
(Tao's work is always must read. Combines algorithmic cleverness with practical engineering and experiments.)