Arguably AI/ML research was held up pre-deep learning by the mistaken belief that empirical questions required foundational answers. Now, as AI researchers look beyond supervised learning (causality, robustness, distribution shift) the folly’s turned converse. #feelthelearn
— Zachary Lipton (@zacharylipton) August 24, 2019
Part 1 TLDR. For supervised learning, the "throw spaghetti at the wall model" works. And the preoccupation with using theoretically-guaranteed methods was arguably misplaced. If anything theory was more useful for insight, not guarantees you'd ever actually use. (5/n)
— Zachary Lipton (@zacharylipton) August 24, 2019
More ambitious problems (causality, robustness, extrapolation beyond support) don't provide the safety net (validation set) that brute force requires (no after-the-fact-guarantees). Suddenly, just as we've accepted empiricism, we find ourselves needing foundational theory (6/n)
— Zachary Lipton (@zacharylipton) August 24, 2019
