Tweeted By @ak92501
A Dot Product Attention Free Transformer
— AK (@ak92501) October 6, 2021
abs: https://t.co/d19I4MHPRb
an efficient variant of Transformers (Vaswani et al., 2017) that eliminates the query-key dot product in self attention pic.twitter.com/LKrO2uzlpk