Many people in engineering believe that to understand something, it is necessary and sufficient to have a low-level mathematical description of that thing. That you need to "know the math behind it". In nearly all cases, it is neither sufficient nor at all necessary - far from it
— François Chollet (@fchollet) October 26, 2018
The same is true of backprop in deep learning -- knowing how to code up backprop by hand gives you no useful knowledge wrt deep learning, and inversely, developing powerful mental models for deep learning does not in any way require knowing the algorithmic details of backprop
— François Chollet (@fchollet) October 26, 2018
Similar to how, say, you can always reinvent the Pythagorean theorem on the fly if you think about geometry through the lens of vector products, or how you don't need to memorize the quadratic formula if you understand what an equation is and the general process for solving them
— François Chollet (@fchollet) October 26, 2018
