Python: How to fix random seed inside process?

Question

I want to make each process produce random numbers but I want them be reproducible across runs.

Here is example code:

import random
from multiprocessing import Process

random.seed(2019)

def f2():
    print('from f2:')
    v = random.randint(0, 10)
    a = random.randint(0, 10)
    print('v:', v)
    return v, a


def python_process_test():
    n_workers = 2
    workers = []
    for i in range(n_workers):
        p = Process(target=f2, args=())
        p.start()
        workers.append(p)

    for p in workers:
        p.join()


if __name__ == '__main__':
    python_process_test()

This is not desired behaviour, because values are not the same across runs:

run 1:

from f2:
v: 6
from f2:
v: 4

run 2:

from f2:
v: 0
from f2:
v: 5

UPDATE:

import time
import random
from multiprocessing import Process

random.seed(2019)

def get_random_sleep_time_v1():
    return random.randint(0,3)

def get_random_sleep_time_v2():
    from datetime import datetime
    random.seed(datetime.now())
    return random.randint(0,3)

def f2(rnd_seed, process_id):
    random.seed(rnd_seed)

    # To rundomize order in which process will print
    sleep_time = get_random_sleep_time_v1()
    #sleep_time = get_random_sleep_time_v2()
    time.sleep(sleep_time)

    print('process_id:', process_id)
    v = random.randint(0, 10)
    a = random.randint(0, 10)
    print('v:', v)
    return v, a


def python_process_test():
    n_workers = 4
    workers = []
    for i in range(n_workers):
        rnd_seed = random.randint(0,10)
        p = Process(target=f2, args=(rnd_seed,i))
        p.start()
        workers.append(p)

    for p in workers:
        p.join()


if __name__ == '__main__':
    python_process_test()

Using get_random_sleep_time_v1 I get desired behaviour but order of processes not changing from run to run:

run 1:

    process_id: 0
    v: 1
    process_id: 3
    v: 1
    process_id: 1
    v: 9
    process_id: 2
    v: 2

run 2:

    process_id: 0
    v: 1
    process_id: 3
    v: 1
    process_id: 1
    v: 9
    process_id: 2
    v: 2

Using get_random_sleep_time_v2 order of processes is random but generated values are not consistent across runs:

run 1:
process_id: 3
v: 10
process_id: 1
v: 8
process_id: 2
v: 7
process_id: 0
v: 6

run 2:
process_id: 0
v: 8
process_id: 3
v: 10
process_id: 2
v: 10
process_id: 1
v: 8

So do you still have a question, or did you edit your solution into the question (which should be an answer)? — Fred Larson, Aug 20 '19 at 15:57
@FredLarson at the top "I want to make each process produce random numbers but I want them be reproducible across runs." — mrgloom, Aug 20 '19 at 15:59
In your last version you're reseeding the RNG when you call `get_random_sleep_v2()`, and that affects the random number sequence. — Barmar, Aug 20 '19 at 20:59
If you want to make the sleep time random but still use `rnd_seed`, put `random.seed(rnd_seed)` after the call to `get_random_sleep_v2()`. — Barmar, Aug 20 '19 at 21:00
Just a heads up: if you're using functions from "numpy" to generate your random numbers, use `numpy.random.seed()` instead. — Waldir Leoncio, Jan 11 '21 at 09:45

Julien Kervizic · Answer 1 · 2019-08-20T20:58:13.457

3

You need to set the seed inside your processes:

import random
from multiprocessing import Process, Manager

def f2(i):
    random.seed(2019+i)
    print('from f2:')
    v = random.randint(0, 10)
    a = random.randint(0, 10)
    print('v:', v, 'a:', a)
    return v, a


def python_process_test():
    n_workers = 2
    workers = []

    for i in range(n_workers):
        p = Process(target=f2, args=([i]))
        p.start()
        workers.append(p)

    for p in workers:
        p.join()


if __name__ == '__main__':
    python_process_test()

This yield the following output everytime:

from f2:
v: 2 a: 3
from f2:
v: 9 a: 9

EDIT: updated to show the passthrough of reseeding each process by worker with different seeds

edited Aug 20 '19 at 20:58

answered Aug 20 '19 at 15:19

Julien Kervizic

91
5

Why is this necessary? There are other questions asking how to get *different* random sequences when using multiprocessing, they say that the processes all inherit the same RNG state. – Barmar Aug 20 '19 at 15:23
This was changed as part of python 3.7, see https://github.com/python/cpython/commit/346cbd351ee0dd3ab9cb9f0e4cb625556707877e#diff-3f5f2db4a1121be5be6b07b5a52b0b09 and more specifically https://github.com/python/cpython/commit/346cbd351ee0dd3ab9cb9f0e4cb625556707877e#diff-3214ef7af90d7b293fa0c247390ec5fe The child process get a new random seed by default – Julien Kervizic Aug 20 '19 at 20:30
I reproduced the OP's result using Python 3.6.3. – Barmar Aug 20 '19 at 20:53
https://github.com/numpy/numpy/issues/9650#issuecomment-327150506 "Python 3.6's default behavior of reseeding in the child processes was hard-coded in the _launch method of multiprocessing.popen_fork.Popen and is specific to the built-in random module" - going through the thread as well: https://github.com/numpy/numpy/issues/9650#issuecomment-327154876 "In 3.5 random is not reseeding but is using a shared state across workers." – Julien Kervizic Aug 20 '19 at 21:23

score 0 · Answer 2 · answered Aug 21 '19 at 09:19

import time
import random
from multiprocessing import Process
from datetime import datetime

random.seed(2222)

def get_random_sleep_time():
    random.seed(datetime.now().microsecond)
    return random.randint(0,3)

def f2(rnd_seed, process_id):
    # To randomize order in which process will print
    sleep_time = get_random_sleep_time()
    time.sleep(sleep_time)

    random.seed(rnd_seed)

    v = random.randint(0, 10)
    a = random.randint(0, 10)
    print('process_id:', process_id, 'v:', v)
    return v, a

def python_process_test():
    n_workers = 4
    workers = []
    for i in range(n_workers):
        rnd_seed = random.randint(0,10)
        p = Process(target=f2, args=(rnd_seed,i))
        p.start()
        workers.append(p)

    for p in workers:
        p.join()

if __name__ == '__main__':
    python_process_test()

score -1 · Answer 3 · answered Aug 20 '19 at 15:19

-1

A simple fix, so no worries:

If you want to generate the same number every time you need to pass the same seed value every time. 1

So all you need to do is call random.seed(2019) before every randint().

Essentially, every time you use the random module to obtain a value, it pulls the .seed() value from the current system time. So you'll have to manually set the seed each time, or else it'll default back to that behavior.

answered Aug 20 '19 at 15:19

Seymour Guado

246
2
13

If you keep resetting the seed, you just keep getting the same number. You only need to seed before the first call. – Barmar Aug 20 '19 at 15:27

Python: How to fix random seed inside process?

3 Answers3