The more I work in prod, the more I realize how hard it is to do prob & stats right.
— Chip Huyen (@chipro) July 8, 2021
Splitting data is easy. Sampling data correctly is hard.
Writing metrics is easy. Understanding what metrics measure is hard.
Monitoring numbers is easy. Interpreting them is very, very hard.
But a rough auto-scalable “template” for a healthy & efficient labeling workflow is slowly emerging along the lines of a finite state machine with a number of slots for specific roles, points of checks and balances and supporting infrastructure. Kinda. Maybe.
— Andrej Karpathy (@karpathy) July 8, 2021
Introduction to Modern Statistics is an open-source book that provides an introduction to modern statistics https://t.co/6w7A67WnG2
— Nathan Yau (@flowingdata) July 8, 2021
We made it easier to train a new tokenizer on a given corpus with 🤗Transformers and 🤗Tokenizers — check out this new example done with @LucileSaulnier!https://t.co/cKhFRWO2bf pic.twitter.com/JH18uHZ1Ga
— Sylvain Gugger (@GuggerSylvain) July 8, 2021
Map shows how Mexico City engulfed Lake Texcoco, from the 16th century to the year 2000. Source: https://t.co/SCqjSxs58H pic.twitter.com/Tac8ZdiXWd
— Simon Kuestenmacher (@simongerman600) July 8, 2021
Rethinking Positional Encoding
— AK (@ak92501) July 7, 2021
pdf: https://t.co/J6wrROIYLk
abs: https://t.co/eIRZEaAXg7
github: https://t.co/FHxaSYLt0m pic.twitter.com/I2nBemH2b3
Should this be "/=" instead of "*="? Thanks for sharing this!
— Brian Bartoldson (@bartoldson) July 6, 2021
In a few equations how it relates to grad clipping. This is the most important part of LAMB/LARS to understand.
— Tim Dettmers (@Tim_Dettmers) July 6, 2021
Grad clip:
if norm(grad) > max_norm:
grad *= norm(grad)/max_norm
Update clip:
if norm(update) > max_unorm:
update *= norm(update)/max_unorm
Solving Machine Learning Problems
— AK (@ak92501) July 6, 2021
pdf: https://t.co/jf7EeapP4Z
abs: https://t.co/uv1JlDLw5M
accuracy of 96% for open-response questions and 97% for mcq, compared with MIT students’ average of 93%, achieving grade A performance in the course, all in real-time pic.twitter.com/NwF3SgMsTv
How does perception of objects arise? Objects undergo huge changes in appearance due to deformation, perspective change, & dynamic occlusion. We prove from first principles that it’s possible, without learning, to perceive invariant objects despite this. https://t.co/oTWSUmuzbk
— Doris Tsao (@doristsao) July 6, 2021
The first step to mastering something is always to *have faith in your capability to learn new skills*. No matter how hard or mystifying the task seems.
— François Chollet (@fchollet) July 5, 2021
The pipeline myth won't die. Google execs are really trying to convince us that they just need to train more students, and it doesn't matter how many senior leaders they wrongly fire.
— Rachel Thomas (@math_rachel) July 5, 2021
