Tweeted By @arimorcos

Are all negatives created equal in contrastive instance discrimination?

In new work led by Tiffany Cai, we show that only the hardest 5% of negatives per query are both necessary and largely sufficient for self-supervised learning.

Tweetprint time!https://t.co/ijyrMb4zGG pic.twitter.com/dIeR7DptVF

— Ari Morcos (@arimorcos) October 15, 2020