Homepage
Close
Menu

Site Navigation

  • Home
  • Archive(TODO)
    • By Day
    • By Month
  • About(TODO)
  • Stats
Close
by RogerGrosse on 2019-03-08 (UTC).

Excited to release our paper on Self-Tuning Networks, a way of adapting regularization hyperparameters online during training. This is the work of Matt MacKay, Paul Vicol, and @jonLorraine9, to appear at ICLR 2019.https://t.co/5svBdWVjhu

— Roger Grosse (@RogerGrosse) March 8, 2019
research
by RogerGrosse on 2019-03-08 (UTC).

Interestingly, the hyperparameters seem to equilibrate over a shorter timescale than the weights, allowing us to learn a schedule. E.g., start with low dropout, then crank it up once the network starts overfitting. Works better than any fixed value! pic.twitter.com/Mw3mp7ph3f

— Roger Grosse (@RogerGrosse) March 8, 2019
research

Tags

learning tutorial misc nlp rstats gan ethics research dataviz survey python tool security kaggle video thought bayesian humour tensorflow w_code bias dataset pytorch cv tip application javascript forecast swift golang rl jax julia gnn causal surey diffusion
© Copyright Philosophy 2018 Site Template by Colorlib