I'm doing a model of a CNN network which takes images that are located in a directory "images", and each image is located inside that directory in a directory named as the ID of the image (ex. image 1 is in: images/1/1.png).
class SSMDataset(Dataset):
songs_list = []
def __init__(self, root_dir, transform=None):
"""lista de nombre de canciones"""
self.root_dir = root_dir
self.songs_list = []
self.labels_list = []
for (dirpath, dirnames, filenames) in os.walk(self.root_dir):
for f in filenames:
if f.endswith('.png'):
o = {}
o['img_path'] = dirpath + '/' + f
self.songs_list.append(o)
elif f.endswith('.csv'):
o['label_path'] = dirpath + '/' + f
self.labels_list.append(o)
self.transform = transform
def __len__(self):
return len(self.songs_list)
def __getitem__(self, index):
img_path = self.songs_list[index]['img_path']
image = Image.open(img_path)
fp = image.fp
image.load()
fp.closed
image = np.array(image)
image = image[np.newaxis, :, :]
label_path = self.labels_list[index]['label_path']
label = np.genfromtxt(label_path)
if self.transform is not None:
for t in self.transform:
image, labels = t(image, label)
return image, label
When I train the model, at the epoch 23 and always in taht epoch the following error raises:
Epoch: 23 39% 136/350 [00:32<00:37, 5.67it/s, accuracy=0.221, loss=1.97]
Traceback (most recent call last): File dataloader.py line 560, in next File dataloader.py line 560, in File line 55, in getitem File Image.py line 2543, in open
OSError: [Errno 24] Too many open files: C:\Users\...\images\1296 /1296.png
Process finished with exit code 1
I have made proof with different libraries as cv2, matplotlib...to open the image, I have use the command "with Image.open(img_path) as im:" to make sure the image is closed after it is taken but it always raises the same error and in the same epoch. I also included:
import torch.multiprocessing
torch.multiprocessing.set_sharing_strategy('file_system')
in the code but the error continues to appear at the same epoch. Any idea of how could it be solved? Maybe is it a Windows problem...?
