Tweeted By @gneubig
MEGA is a new method for modeling long sequences based on the surprisingly simple technique of taking the moving average of embeddings.
— Graham Neubig (@gneubig) September 23, 2022
Excellent results, outperforming strong competitors such as S4 on most tasks! Strongly recommend that you check it out: https://t.co/Y07MSd0hhc https://t.co/9xbmaF5XCr pic.twitter.com/pX01RctxDN