Tweeted By @gdb
ML bugs are so much trickier than bugs in traditional software because rather than getting an error, you get degraded performance (and it's not obvious a priori what ideal performance is).
— Greg Brockman (@gdb) May 14, 2022
So ML debugging works by continual sanity checking, e.g. comparing to various baselines.