printing image paths from the dataloader in pytorch

Question

I am trying to learn One-shot learning with pytorch. I am experimenting with this Siamese Network in Pytorch example. Using that notebook as a guide, I simply would like to print out the image file paths for each pair of images, in addition to the dissimilarity scores.

From what I've been reading, it looks like I need to make some alterations to the dataloader in order to achieve this, as indicated here.

I do not have much experience yet in all of this. I would appreciate some guidance. I imported the altered dataloader (as in this gist) into my code.

The altered dataloader:

import torch
from torchvision import datasets

class ImageFolderWithPaths(datasets.ImageFolder):
    """Custom dataset that includes image file paths. Extends
    torchvision.datasets.ImageFolder
    """

    # override the __getitem__ method. this is the method that dataloader calls
    def __getitem__(self, index):
        # this is what ImageFolder normally returns 
        original_tuple = super(ImageFolderWithPaths, self).__getitem__(index)
        # the image file path
        path = self.imgs[index][0]
        # make a new tuple that includes original and the path
        tuple_with_path = (original_tuple + (path,))
        return tuple_with_path

and the example usage:

data_dir = "your/data_dir/here"
dataset = ImageFolderWithPaths(data_dir) # our custom dataset
dataloader = torch.utils.DataLoader(dataset)

# iterate over data
for inputs, labels, paths in dataloader:
    # use the above variables freely
    print(inputs, labels, paths)

my code:

from pytorch_image_folder_with_file_paths import ImageFolderWithPaths

folder_dataset_test = dset.ImageFolder(root=Config.testing_dir)
siamese_dataset = SiameseNetworkDataset(imageFolderDataset=folder_dataset_test,
                                        transform=transforms.Compose([transforms.Resize((100,100)),
                                                                      transforms.ToTensor()
                                                                      ])
                                       ,should_invert=False)

test_dataloader = DataLoader(siamese_dataset,num_workers=6,batch_size=1,shuffle=True)

dataiter = iter(test_dataloader)
x0,_,_ = next(dataiter)

for i in range(10):
    _,x1,label2 = next(dataiter)

    concatenated = torch.cat((x0,x1),0)

    output1,output2 = net(Variable(x0).cuda(),Variable(x1).cuda())
    euclidean_distance = F.pairwise_distance(output1, output2)
    imshow(torchvision.utils.make_grid(concatenated),'Dissimilarity: {:.2f}'.format(euclidean_distance.item()))

for inputs, labels, paths in test_dataloader:
    print(inputs, labels, paths)

I get the paired images with the dissimilarity score, but I don't get the paths; I get

tensor([[[[0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          ...,
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.]]]]) tensor([[[[0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          [0., 0., 0.,  ..., 0., 0., 0.],
          ...,

...etc.

Thanks

You are not using `ImageFolderWithPaths` in your code anywhere, just importing it. — Anubhav Singh, Jul 10 '19 at 06:58
Right, right. So... I'd use here `siamese_dataset = SiameseNetworkDataset(imageFolderDataset ` then? — shawn, Jul 10 '19 at 14:55
Yeah, I tested the above code with `SiameseNetworkDataset(imageFolderDataset)` and it's working perfectly, returning paths also. — Anubhav Singh, Jul 10 '19 at 14:59

score 1 · Answer 1 · answered Jul 10 '19 at 16:05

Thank you Anubhav Singh for sorting me out.

This works:

from pytorch_image_folder_with_file_paths import ImageFolderWithPaths

folder_dataset_test = ImageFolderWithPaths(root=Config.testing_dir)
siamese_dataset = SiameseNetworkDataset(imageFolderDataset=folder_dataset_test,
                                        transform=transforms.Compose([transforms.Resize((100,100)),
                                                                      transforms.ToTensor()
                                                                      ])
                                       ,should_invert=False)

test_dataloader = DataLoader(siamese_dataset,num_workers=6,batch_size=1,shuffle=True)

dataiter = iter(test_dataloader)
x0,_,_ = next(dataiter)

for i in range(10):
    _,x1,label2 = next(dataiter)

    concatenated = torch.cat((x0,x1),0)

    output1,output2 = net(Variable(x0).cuda(),Variable(x1).cuda())
    euclidean_distance = F.pairwise_distance(output1, output2)
    imshow(torchvision.utils.make_grid(concatenated),'Dissimilarity: {:.2f}'.format(euclidean_distance.item()))

for paths in folder_dataset_test:
    # use the above variables freely
    print(paths)

Incidentally, I'm working in Google Colab which doesn't allow me to edit files directly, so for the dataloader, I made a new cell and used %%writefile to get it into my notebook:

%%writefile pytorch_image_folder_with_file_paths.py

import torch
import torchvision.datasets as dset

class ImageFolderWithPaths(dset.ImageFolder):
    """Custom dataset that includes image file paths. Extends
    torchvision.datasets.ImageFolder
    """

    # override the __getitem__ method. this is the method that dataloader calls
    def __getitem__(self, index):
        # this is what ImageFolder normally returns 
        original_tuple = super(ImageFolderWithPaths, self).__getitem__(index)
        # the image file path
        path = self.imgs[index][0]
        # make a new tuple that includes original and the path
        tuple_with_path = (original_tuple + (path,))
        return tuple_with_path

Wait, I tell a lie. Yes, this prints the paths, but its printing all the paths, not the paths to the two images being compared for similarity. — shawn, Jul 10 '19 at 16:18

printing image paths from the dataloader in pytorch

1 Answers1