The unambiguously correct place to examine your training data is immediately before it feeds into the network. Take the raw x,y batch tuple, ship it back to CPU, unrender, visualize. V often catches bugs with data augmentation, label preprocessing, samplers, collation, etcetc.
— Andrej Karpathy (@karpathy) November 17, 2020