Does torch.manual_seed include the operation of torch.cuda.manual_seed_all?

Question

Does torch.manual_seed include the operation of torch.cuda.manual_seed_all?

If yes, we can just use torch.manual_seed to set the seed. Otherwise we should call both functions.

score 3 · Accepted Answer · answered May 18 '21 at 11:18

3

Yes, torch.manual_seed() does include CUDA:

You can use torch.manual_seed() to seed the RNG for all devices (both CPU and CUDA):

answered May 18 '21 at 11:18

iacob

score 1 · Answer 2 · answered May 18 '21 at 07:25

1

random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)

Makes me believe these are all and only the required seeds.

answered May 18 '21 at 07:25

Gulzar

1

There may be [other sources of randomness](https://stackoverflow.com/questions/67511658/training-pytorch-models-on-different-machines-leads-to-different-results/67511954#67511954) in PyTorch models, depending on the specific modules/functions used. – iacob May 18 '21 at 12:23

score 1 · Answer 3 · answered Oct 20 '22 at 03:01

Yes, torch.manual_seed calls torch.cuda.manual_seed_all internally.

Additional evidence other than @iacob's answer can be found in the PyTorch source code

def manual_seed(seed) -> torch._C.Generator:
    ...

    if not torch.cuda._is_in_bad_fork():
        torch.cuda.manual_seed_all(seed)

    ...

3 Answers3