PyTorch model output different dimension when using DataParallel

Question

I implemented my PyTorch model with DataParallel for multi-GPU training. However, it seems that the model doesn't consistently output the right dimension. In the training loop, it seems that the model gave the correct output dimension for the first two batches, but it failed to do so for the third batch and caused an error when calculating the loss:

I also tried to use the solution from this post but it didn't help.

Please provide more details (code, etc.). – GoodDeeds Sep 12 '22 at 06:13 — GoodDeeds, Sep 12 '22 at 06:13

score 1 · Accepted Answer · answered Sep 12 '22 at 06:34

1

It seems like you are left with only one sample for the last batch. Try setting drop_last=True in your Dataloader: This will discard the last "not-full" batch.

answered Sep 12 '22 at 06:34

Shai

111,146
38
238
371

could you check this question please? ty https://stackoverflow.com/questions/74380417/how-to-use-multiple-gpu-at-the-same-time-when-using-models-with-pytorch-and-hugg – Furkan Gözükara Nov 09 '22 at 20:05

PyTorch model output different dimension when using DataParallel

1 Answers1