Pytorch - Can not slice torchvision MNIST dataset

Question

In Pytorch, when using torchvision's MNIST dataset, we can get a digit as follows:

from torchvision import datasets, transforms
from torch.utils.data import DataLoader, Dataset, TensorDataset

tsfm = transforms.Compose([transforms.Resize((16, 16)),
                           transforms.ToTensor(),
                           transforms.Normalize((0.1307,), (0.3081,))])

mnist_ds = datasets.MNIST(root='../../../_data/mnist',train=True,download=True,
                          transform=tsfm)

digit_12 = mnist_ds[12]

Though it is possible to slice on many datasets, we cannot slice on this one:

>>> digit_12_to_14 = mnist_ds[12:15]
ValueError: Too many dimensions: 3 > 2.

This is due to a Image.fromarray() in the getItem().

Is it possible to use MNIST dataset without using a Dataloader?

PS: The reason why I would like to avoid using Dataloader is that sending batches one at a time to the GPU slows down the training. I prefer to send the entire dataset to the GPU at once. For this I need to have access to the whole transformed dataset.

Does this answer your question? [How do you alter the size of a Pytorch Dataset?](https://stackoverflow.com/questions/44856691/how-do-you-alter-the-size-of-a-pytorch-dataset) — iacob, Mar 15 '21 at 11:28

score 4 · Answer 1 · answered Mar 15 '21 at 14:03

4

You can use torch.utils.data.Subset() to get an index based slice of a torch Dataset e.g:

import torch.utils.data as data_utils

indices = torch.arange(12,15)
mnist_12to14 = data_utils.Subset(tr, indices)

answered Mar 15 '21 at 14:03

iacob

20,084
6
92
119

1

Great find, thanks :) – Andrei-Cristian Rad Jan 31 '22 at 19:21

Jatentaki · Answer 2 · 2019-01-18T18:17:18.913

2

The Dataset interface only requires that

All subclasses should override __len__, that provides the size of the dataset, and __getitem__, supporting integer indexing in range from 0 to len(self) exclusive.

which clearly does not mention slicing - the slicing behavior of other datasets is an extra feature. If you want to get the whole data at once, you can look up the implementation and just use the mnist.data and mnist.targets tensors defined towards the end of __init__.

If you want to have transformed data, you can use

data = [mnist_ds[i] for i in range(len(mnist_ds))]
xs = torch.stack([d[0] for d in data], dim=0)
ys = torch.stack([d[1] for d in data], dim=0)

or transform the mnist.data tensor all at once (though that will not work with the torchvision.transform transforms).

edited Jan 18 '19 at 18:17

answered Jan 18 '19 at 17:43

Jatentaki

11,804
4
41
37

Thanks for the clarification about slicing. mnist.data and mnist.targets are not transformed. One should use __getIem()__ to get transformed images – u2gilles Jan 18 '19 at 18:10
My bad, updated. Essentially, use the approach of Fabio Perez – Jatentaki Jan 18 '19 at 18:17

u2gilles · Answer 3 · 2019-01-24T07:00:57.790

I found 2 solutions so far to convert torchvision MNIST dataset to tensors. The first one is derived from Fábio Perez comment :

print("\nFirst...")
st = time()
x_all_ts = torch.tensor([mnist_ds[i][0].numpy() for i in range(0, len(mnist_ds))])
t_all_ts = mnist_ds.train_labels
print(f"{time()-st}   images:{x_all_ts.size()}  targets:{t_all_ts.size()} ")

print("\nSecond...")
st = time()
mnist_dl = DataLoader(dataset=mnist_ds, batch_size=len(mnist_ds))
x_all_ts2, t_all_ts2 = list(mnist_dl)[0]
print(f"{time()-st}   images:{x_all_ts2.size()}  targets:{t_all_ts2.size()} ")


First...
19.573785066604614   images:torch.Size([60000, 1, 16, 16])  targets:torch.Size([60000]) 
Second...
16.826476573944092   images:torch.Size([60000, 1, 16, 16])  targets:torch.Size([60000])

Please let me know if you find better ones.

Pytorch - Can not slice torchvision MNIST dataset

3 Answers3