Tweeted By @jeremyphoward

"Pretty nice" is quite the understatement. This is a wonderful in-depth discussion of the weird interactions between batchnorm, weight decay, and learning rate, including a fascinating experiment that shows that you can entirely replace weight decay with learning rate changes, https://t.co/DuCaJRRp91

— Jeremy Howard (@jeremyphoward) April 11, 2019