Why does a saved PyTorch model in eval mode return different results every time inference is ran?

Question

In this case, I tried all the usual steps to make sure there was no stochastic element to the inference code itself i.e. the model shouldn't change it's predictions based on the inputs. See for details of this:

Why am I getting different results after saving and loading model weights in pytorch?

score 0 · Answer 1 · answered Feb 23 '23 at 13:40

In my case, it was the data. The input to the model was changing on each run. At training time, I had set randomized data transforms which were appropriate for training. But, of course, this introduced randomness as I prepared the data for inference. So I changed

data_transforms = {
"train": transforms.Compose(
    [
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
    ]
),

to:

data_transforms = {
"train": transforms.Compose(
    [
        transforms.Resize(256),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),

which ensured consistency.

Why does a saved PyTorch model in eval mode return different results every time inference is ran?

1 Answers1