1

Ive been trying to integrate the to(device) inside my dataloader using to(device) as seen in https://github.com/pytorch/pytorch/issues/11372

I defined it on FashionMNIST in the following way:

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
batch_size = 32
trainset = datasets.FashionMNIST('~/.pytorch/F_MNIST_data/',
                                 download=True,
                                 train=True,
                                 transform=transforms.ToTensor())
rain_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=False, collate_fn=lambda x: default_collate(x).to(device))

But i get the following error: AttributeError: 'list' object has no attribute 'to' It seems that the output of default collate is a list of length 2 with the first element being the image tensor and the second the labels tensor (since its the output of next(iter(train_loader)) with collate_fn=None), so I tried with the following defined function:

def to_device_list(l, device):
    return [l[0].to(device), l[1].to(device)]
train_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=False, collate_fn=lambda x: to_device_list(x, device))

And I got the following error: AttributeError: 'tuple' object has no attribute 'to'

Any help please on how to do it?

talonmies
  • 70,661
  • 34
  • 192
  • 269
JNYC
  • 31
  • 1
  • 5

1 Answers1

2

The fashion mnist dataset returns a tuple of img and target, where the img is tensor and target is int value for class.

Now, your dataloader takes batch size samples from dataset class to get list of samples. Note, this list of samples is now, List[Tuple[Tensor, int]](using typing annotation here). Then it calls, collate function to convert List[Tuple[Tensor, int]] into List[Tensor], where this list has 2 tensors. The first tensor is stacked array of images of size [32, 1, 28, 28], where 32 was batch size and second tensor is tensor array of int values(class labels).

The default_collate function, just converts array of structures to structures of array.

Now, when you use collate_fn=lambda x: default_collate(x).to(device), notice that default_collate returns you a list of tensors. So calling .to on list wont work and should be called on all elements of the list.

Solution Use

collate_fn=lambda x: list(map(lambda x: x.to(device), default_collate(x))))

The map function transfers each element of list(from default_collate) to cuda, and finally, call list, since map is evaluated lazy in python3.

saurabheights
  • 3,967
  • 2
  • 31
  • 50
  • For anyone facing this issue, I would recommend to use debugging tool in pycharm or other ide, or just pudb. Its very easy to pinpoint the issue that way. – saurabheights Apr 26 '21 at 11:24