Split images when creating custom dataset function

Question

I have 5000x5000X3 sized images, I want to split image into multiple smaller images. I have tried to create dataset with splitted images. But it occupies more space and is very tedious task to manage those images. Then I tried to create a pipeline where images are split while training. But confused with how to implement it as pytorch custom dataset class.

import os
import pandas as pd
from torchvision.io import read_image

class CustomImageDataset(Dataset):
    def __init__(self, annotations_file, img_dir, transform=None, target_transform=None):
          self.img_labels = pd.read_csv(annotations_file)
          self.img_dir = img_dir
          self.transform = transform
          self.target_transform = target_transform

   def __len__(self):
         return len(self.img_labels)

  def __getitem__(self, idx):
        img_path = os.path.join(self.img_dir, self.img_labels.iloc[idx, 0])
        image = read_image(img_path)
        label = self.img_labels.iloc[idx, 1]
        if self.transform:
            image = self.transform(image)
        if self.target_transform:
            label = self.target_transform(label)
        return image, label

Answer 1

Please first look at this: https://stackoverflow.com/a/72642001/9560771 and make sure if you like to use online augmentations and offline augmentations

If you like offline you must save the small images onto the disk.

Else, using online you can use transforms.RandomResizedCrop(XX) to randomly crop small images from the input image