Getting image path through a torchvision dataloader using local images

Question

I want to use a dataloader in my script.

normaly the default function call would be like this.

dataset = ImageFolderWithPaths(
    data_dir,
    transforms.Compose([
            transforms.ColorJitter(0.1, 0.1, 0.1, 0.1),
            transforms.Resize((img_size_XY, img_size_XY)),
            transforms.ToTensor(),
            transforms.Normalize(_mean , _std)
        ])
)

dataloader = torch.utils.data.DataLoader(
    dataset,
    batch_size=batch_size,
    shuffle=False,
    num_workers=2
)

and to iterate through this dataloader i use

for inputs, labels , paths in _dataloader:
    break

now i need to collect the path for each image.

i found in github this code: (https://gist.github.com/andrewjong/6b02ff237533b3b2c554701fb53d5c4d)

class ImageFolderWithPaths(datasets.ImageFolder):
    """Custom dataset that includes image file paths. Extends
    torchvision.datasets.ImageFolder
    """

    # override the __getitem__ method. this is the method that dataloader calls
    def __getitem__(self, index):
        # this is what ImageFolder normally returns 
        original_tuple = super(ImageFolderWithPaths, self).__getitem__(index)
        # the image file path
        path = self.imgs[index][0]
        # make a new tuple that includes original and the path
        tuple_with_path = (original_tuple + (path,))
        return tuple_with_path

# EXAMPLE USAGE:
# instantiate the dataset and dataloader
data_dir = "your/data_dir/here"
dataset = ImageFolderWithPaths(data_dir) # our custom dataset
dataloader = torch.utils.DataLoader(dataset)

# iterate over data
for inputs, labels, paths in dataloader:
    # use the above variables freely
    print(inputs, labels, paths)

But this code does not take transforms into account, like in my original code.

Can anybody help in how I should go about making it work with that?

`StandardTransform Transform: Compose( ColorJitter(brightness=[0.9, 1.1], contrast=[0.9, 1.1], saturation=[0.9, 1.1], hue=[-0.1, 0.1]) Resize(size=(384, 384), interpolation=bilinear, max_size=None, antialias=None) ToTensor() Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) )` — asdf qwer, Feb 25 '22 at 17:10
And you are not getting any errors when calling an instance from the dataset? — Ivan, Feb 25 '22 at 17:16
with the initial code it works like it should. When i add the code from github it doesent work but i suppose its due to the missing trasforms which means that the images do not get resized. ( or the github code doesent work, but according to the comments it does) — asdf qwer, Feb 25 '22 at 17:28
The code with the example in github worked with you? I tried it but I have some issues. — Hamzah, Feb 26 '22 at 08:10
no thats what i am saying that it didnt work. But i dont know if it is my implementation faulty or if just the algorithm is broken — asdf qwer, Feb 26 '22 at 09:02

Hamzah · Accepted Answer · 2022-02-26T10:07:10.067

Since ImageFolderWithPaths inherits from datasets.ImageFolder as shown in the code from GitHub and datasets.ImageFolder has the following arguments including transform: (see here for more info)

torchvision.datasets.ImageFolder(root: str, transform: Optional[Callable] = None, target_transform: Optional[Callable] = None, loader: Callable[[str], Any] = , is_valid_file: Optional[Callable[[str], bool]] = None)

Solution: you can use your transformations directly when you instantiate ImageFolderWithPaths.

import torch
from torchvision import datasets
from torch.utils.data import DataLoader 

class ImageFolderWithPaths(datasets.ImageFolder):

    def __getitem__(self, index):
  
        img, label = super(ImageFolderWithPaths, self).__getitem__(index)
        
        path = self.imgs[index][0]
        
        return (img, label ,path)


# put here your root directory not subfolders directory
# subfolders should be names of classes or encodings
root_dir = "training" 

transform = transforms.Compose([transforms.Resize((32, 32)), 
                                transforms.ToTensor()]) # my transformations.

dataset = ImageFolderWithPaths(root_dir,transform=transform) # add transformation directly

dataloader = DataLoader(dataset)


for inputs, labels, paths in dataloader:
    print(inputs.shape, labels, paths)

# output
torch.Size([1, 3, 32, 32]) tensor([0]) ('training\\0\\1.jpg',)
torch.Size([1, 3, 32, 32]) tensor([0]) ('training\\0\\1000.jpg',)
torch.Size([1, 3, 32, 32]) tensor([0]) ('training\\0\\10005.jpg',)
torch.Size([1, 3, 32, 32]) tensor([0]) ('training\\0\\10010.jpg',)

I also edited the code from github because there is no torch.utils.DataLoader but torch.utils.data.DataLoader.

Getting image path through a torchvision dataloader using local images

1 Answers1