Tweeted By @iamtrask

Hypothesis - major Deep Learning discoveries in the last ~decade fall into two categories:

1) better learning bias (usually by strategically re-using weights)

Convnets, RNNs, LSTMs, GRUs, Attention, CapsNet, Dropout...

2) more training data

GANs, GPUs, word2vec, ImageNet...

— Trask (@iamtrask) September 17, 2018