Tweeted By @hardmaru
Stiffness: A New Perspective on Generalization in Neural Networks
— hardmaru (@hardmaru) March 18, 2019
They measure how stiff a network is by looking at how a small gradient step on one example affects the loss on another. Stiffness is useful for diagnosing and characterizing generalization. https://t.co/v7vFuxIKct pic.twitter.com/18nJLmHJ4b