Think you understand weight decay? Think again. We found three distinct mechanisms by which it achieves a regularization effect, depending on the architecture and optimization algorithm. New paper w/ @Guodzh, Chaoqi Wang, and Bowen Xu.https://t.co/BXN9tedEPz
— Roger Grosse (@RogerGrosse) October 30, 2018
Check out the #conll2018 best papers:
— Sebastian Ruder (@seb_ruder) October 29, 2018
- Artetxe et al. reveal that word embeddings capture more information than typically assumed. https://t.co/vvN4sVtXYR
- Barrett et al. show that human attention is a good inductive bias for attention. https://t.co/dvNLj9HGjE https://t.co/QHRZdqRVnX
This is the first published study on the use of SentencePiece + ULMFiT. It's a really great combination, and there's many more opportunities to do research like this on effective use of subword units on NLP tasks. https://t.co/JUWbZY5bCA
— Jeremy Howard (@jeremyphoward) October 27, 2018
The Five Major Objectives of the M4 Forecasting Conference:
— Spyros Makridakis (@spyrosmakrid) October 27, 2018
New York CIty, December 10 and 11 https://t.co/dC6vpmRjBC
For information about the comparisons of ML and Stat forecasting methods for the M3 Competition https://t.co/S3BpRgtxUW
for the M4 https://t.co/fYp0Mn2fCn pic.twitter.com/I74LnHOOUI
Based on this research work:
— Jeff Dean (@JeffDean) October 27, 2018
AutoAugment: Learning Augmentation Policies from Data
Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, Quoc Lehttps://t.co/O3rZLcECzg
Reversible RNNs: reduce memory costs of GRU and LSTM networks by 10-15x without loss in performance. Also 5-10x for attention-based architectures. New paper with Matt MacKay, Paul Vicol, and Jimmy Ba, to appear at NIPS. https://t.co/Bx2v6mOn7O
— Roger Grosse (@RogerGrosse) October 26, 2018
Neighbourhood Consensus Networks. More deep learning for image-to-image dense correspondence. https://t.co/F1OHg7lI3q #computervision #machinevision #machinelearning #nips2018 pic.twitter.com/f4sQASupfg
— Tomasz Malisiewicz (@quantombone) October 25, 2018
Are you interested in summarization? @tbsflk compiled the results on the most common datasets (CNN/DailyMail, Gigaword, DUC04 Task 1) from 2015-2018. 👏🏻https://t.co/xdPsLwrGqy
— Sebastian Ruder (@seb_ruder) October 25, 2018
Playing Mortal Kombat with TensorFlow.js https://t.co/taRHHqblOC
— François Chollet (@fchollet) October 25, 2018
Do Deep Generative Models Know What They Don't Know? https://t.co/dxqoLX8f8m
— DeepMind (@DeepMindAI) October 24, 2018
One of the best RL papers to date. https://t.co/zyn24gq0JK
— hardmaru (@hardmaru) October 24, 2018
1st place solution in #Kaggle TGS Salt Identification Challenge https://t.co/XcxW4LxNPP
— Alexandr Kalinin (@alxndrkalinin) October 22, 2018
- encoder: ResNeXt50/ResNet34
- center: Feature Pyramid Attention
- decoder: conv3x3, T conv, scSE + hyper columns
- loss: BCE, Lovász
- 2-level pseudo-labeling
- 2 model final ensemble
