Tweeted By @ericjang11
A beautiful music transformer visualization of the final attention heads from @ashVaswani 's talk on "attention is all you need" at RAAIS 2019 https://t.co/XwSIKr4nm3
— Eric Jang 🇺🇸🇹🇼 (@ericjang11) December 26, 2021
The model learns to attend to periodic tokens when doing things like tremelos pic.twitter.com/znWCv5WWsU