Tweeted By @GoogleAI
Introducing a novel mixture-of-experts routing algorithm, called Expert Choice, that can achieve optimal load balancing between experts while allowing heterogeneous token-to-expert mapping. Learn how it’s done at https://t.co/EDomelBL53 pic.twitter.com/Oyaa0wEMlX
— Google AI (@GoogleAI) November 16, 2022