pytorch ImageFolder datasize does not match with the original imgs number?

Question

I have 21,522 items, totalling 609.7 MB in train folder

train
  -1(3600 png)
  -2(3600 png)
   .
   .
  -6

train_trans = transforms.Compose([
        vision.transforms.Resize(target_size),
        vision.transforms.RandomHorizontalFlip(),
        vision.transforms.RandomRotation(20),
        #CIFAR10Policy(),
        vision.transforms.ToTensor(),
        vision.transforms.Normalize(
            [0.485, 0.456, 0.406], 
            [0.229, 0.224, 0.225])
])
valid_trans = transforms.Compose([
        vision.transforms.Resize(target_size),
        vision.transforms.RandomResizedCrop(target_size, scale=(0.8,1.0)),
        vision.transforms.RandomHorizontalFlip(),
        vision.transforms.ToTensor(),
        vision.transforms.Normalize(
            [0.485, 0.456, 0.406], 
            [0.229, 0.224, 0.225])
])
test_trans = transforms.Compose([
        vision.transforms.Resize((size,size)),
        vision.transforms.RandomResizedCrop(target_size, scale=(0.8,1.0)),
        vision.transforms.ToTensor(),
        vision.transforms.Normalize(
            [0.485, 0.456, 0.406], 
            [0.229, 0.224, 0.225])
])

but when I load the dataset with

train_dataset = torchvision.datasets.ImageFolder(root= path + 'train/',transform=trans)

somehow

print(len(train_dataset)) = 4109

somehow it does not load the entire image files

I don't think there's any way to debug this with the information given. The only guess I have is that maybe the images in your folder don't all have acceptable extensions, i.e. one of these: `IMG_EXTENSIONS = ('.jpg', '.jpeg', '.png', '.ppm', '.bmp', '.pgm', '.tif', '.tiff', '.webp')` — jodag, Dec 08 '19 at 18:05

score 0 · Answer 1 · answered Dec 10 '19 at 13:41

If you want to access all the images in a given folder, the path given should be one level above.

For example, when you give the path as root= path + 'train/' , then the ImageFolder looks at only the folders within the ../train/ folder, and not the image files in that folder. So, either organize the files in folders within the train folder, or make sure you have only the train folder in the root folder and provide the path to the root folder.

pytorch ImageFolder datasize does not match with the original imgs number?

1 Answers1