Tweeted By @TedPetrou
Have a huuuuuge dataset and want to read in a random fraction of the data with pandas saving enormous amounts of memory?
— Ted Petrou (@TedPetrou) November 22, 2019
Set skiprows to a function that tests a random value against some constant. The below function skips 99% of rows and keeps the first for the column header. pic.twitter.com/tKKR2dyMqy