Tweeted By @JulianSlzr
[3/4] Unsupervised MLM scores from BERT narrow the human gap on the BLiMP minimal pairs set (@a_stadt, @sleepinyourhat), suggesting left-to-right bias in GPT-2 has an outsized effect.
— Julian Salazar (@JulianSlzr) May 15, 2020
(Yes, we ran these experiments when the dataset came out, 1 week before the ACL deadline 😌) pic.twitter.com/I5ng2NobSd