Upsampling for the whole dataset or for each mini-batch

Asked Jan 08 '20 at 17:15

Active May 25 '20 at 03:06

Viewed 219 times

I am trying to train my convNet on a very large unbalanced dataset. It will be quite difficult to load the data altogether to memory and do the upsampling on the whole dataset.

Instead, I want to load the data in mini-batches and do the upsampling for each of those mini-batches. Will that be a valid way to do resampling and training?

I appreciate any feedback!

asked Jan 08 '20 at 17:15

Chong Sun

1

Do you want to use `smote` or repeat minority class as your upsampling method? What framework do you use? – Szymon Maszke Jan 08 '20 at 18:10
I am using SMOTE algorithm with imblearn package in Python – Chong Sun Jan 08 '20 at 18:30

Upsampling for the whole dataset or for each mini-batch

0 Answers0