Couldn't close file error when using pytorch enumerate

Question

I seem to get an error when calling the pytorch enumerate method. From googling just the error message it seems to be a problem with my operating system, macOS catalina. Is there a solution to this that I just couldn't find and/or is there an alternative to using enumerate? (I am running scripts someone else wrote and am not familiar with DataLoaders in pytorch and couldn't find much information about the enumerate method). Here is the full error message:

libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Couldn't close file
Traceback (most recent call last):
  File "background_analysis.py", line 313, in <module>
    validation_classifier(fit_tree, "../events/" + data + "/dataframes/bbbb_SR.h5", data_name=data, method_name=method, region='SR', epochs=15)
  File "/Users/Trevor/Desktop/Research/toy4b/python/validation.py", line 18, in validation_classifier
    model.runEpoch(print_all_epochs=True, fit_validation=True, data_name=data_name, method_name=method_name, region=region)
  File "fvt_scripts/model_train.py", line 1270, in runEpoch
    self.validate()
  File "fvt_scripts/model_train.py", line 1132, in validate
    self.evaluate(self.validation, doROC)
  File "fvt_scripts/model_train.py", line 1111, in evaluate
    for i, (J, O, D, Q, y, w) in enumerate(results.evalLoader):
  File "/Users/Trevor/Desktop/Research/myenv/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 279, in __iter__
    return _MultiProcessingDataLoaderIter(self)
  File "/Users/Trevor/Desktop/Research/myenv/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 746, in __init__
    self._try_put_index()
  File "/Users/Trevor/Desktop/Research/myenv/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 861, in _try_put_index
    index = self._next_index()
  File "/Users/Trevor/Desktop/Research/myenv/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 339, in _next_index
    return next(self._sampler_iter)  # may raise StopIteration
  File "/Users/Trevor/Desktop/Research/myenv/lib/python3.7/site-packages/torch/utils/data/sampler.py", line 202, in __iter__
    if len(batch) == self.batch_size:
  File "/Users/Trevor/Desktop/Research/myenv/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 6750) is killed by signal: Unknown signal: 0. 
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Couldn't close file
Abort trap: 6

Does it work with `DataLoader`'s `num_workers` argument set to `0`? — Szymon Maszke, Jun 18 '20 at 09:14

score 0 · Answer 1 · answered Sep 09 '20 at 11:02

I found that someone is using the file and may cause the Couldn't close file problem. For my code that is a pytorch example code, which trained a resnet18 after open a picture by matplotlib, this is the code I commented out the showing part and the error gone:

def imshow(inp, title=None):
    inp = inp.numpy().transpose((1, 2, 0))
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    inp = std * inp + mean
    inp = np.clip(inp, 0, 1)

    plt.imshow(inp)
    if title is not None:
        plt.title(title)
    plt.pause(0.001)  # pause a bit so that plots are updated


#inputs, classes = next(iter(dataloaders['train']))
# Make a grid from batch
#out = torchvision.utils.make_grid(inputs)
#imshow(out, title=[class_names[x] for x in classes]) #stop showing the pic

Couldn't close file error when using pytorch enumerate

1 Answers1