2

Goal: Override FastAI appending models/ prefix and .pth suffix to the weights file path.

Version: fastai==2.7.7

Note: this behaviour does not happen to the model file path.

How can I override this behaviour?

Turning:

model.load('data/models/my_weights.pth')

Into:

FileNotFoundError: [Errno 2] No such file or directory: 'models/data/models/my_weights.pth.pth'

Code:

def inference(model_params: ModelParams, device: str, dataloader: DataLoader, id: str):
    model = load_learner(model_params.model, cpu=device=='cpu')
    model.load('data/models/my_weights.pth')

Traceback:

Traceback (most recent call last):
  File "/home/me/BitBucket/project/app/container/main.py", line 145, in <module>
    prediciton = inference(model_params, device, dataloader, id)
  File "/home/me/BitBucket/project/app/container/main.py", line 116, in inference
    model.load('data/models/my_weights.pth')
  File "/home/me/miniconda3/envs/venv/lib/python3.9/site-packages/fastai/learner.py", line 387, in load
    load_model(file, self.model, self.opt, device=device, **kwargs)
  File "/home/me/miniconda3/envs/venv/lib/python3.9/site-packages/fastai/learner.py", line 51, in load_model
    state = torch.load(file, map_location=device)
  File "/home/me/miniconda3/envs/venv/lib/python3.9/site-packages/torch/serialization.py", line 791, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/me/miniconda3/envs/venv/lib/python3.9/site-packages/torch/serialization.py", line 271, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/me/miniconda3/envs/venv/lib/python3.9/site-packages/torch/serialization.py", line 252, in __init__
    super().__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'models/data/models/my_weights.pth.pth'
DanielBell99
  • 896
  • 5
  • 25
  • 57
  • [GitHub Issue](https://github.com/fastai/fastai/issues/3914) – DanielBell99 May 11 '23 at 09:27
  • 1
    What is `model_params.model` exactly? I've checked fastai and torch code, none of them change input path. There is something to notice. `model.load(path)` calls the `load(path)` method from `fname`. `fname` is the first input of `load_learner (fname, cpu=True)`. So we can conclude the bug is from your `model_params.model`'s `load` method. – Ali Ent May 15 '23 at 14:25
  • 1
    `model_params.model` is just the file path for the model .pkl. This path does not have this problem. Ohh interesting ok, I'll wrap my head around this more and get back here tomorrow – DanielBell99 May 15 '23 at 15:34
  • Passing the string literal directly `model.load('data/models/my_weights.pth')` still throws: `FileNotFoundError: [Errno 2] No such file or directory: 'models/data/models/my_weights.pth.pth'` – DanielBell99 May 16 '23 at 14:20
  • Seems like in the class, the path is appened with .pth. This can not be prevented, exept for altering the libary. ` file = join_path_file(file, self.path/self.model_dir, ext='.pth')` – Christian Schaffner May 22 '23 at 10:19
  • Can you please provide more code for this @DanielBell99 I cannot reproduce the error and by adding print functions to the libraries ***DO NOT DO IT UNLESS 1000% SURE*** I cannot seem to find anywhere that changes the path. – jett8998 May 22 '23 at 11:08

0 Answers0