Tweeted By @hardmaru
ZerO Initialization: Initializing Neural Networks with only Zeros and Ones
— hardmaru (@hardmaru) November 11, 2022
A fully deterministic initialization scheme which sets the weights to only 0s and 1s can achieve SOTA on various datasets including ImageNet. Maybe random weights are unnecessary.https://t.co/t6u3S6Dj71 pic.twitter.com/XgszDvat6T