https://t.co/r3rVYtlepN (on which I'm a minor co-author) shows results for both transfer learning and multi-task learning for different languages.
— Jeff Dean (@JeffDean) January 20, 2019
tl:dr: languages without much training data improve a lot, languages with lots of training data improve a little.