I know that stable_baselines3 is able to train in parallel with make_vec_env. However, I can't find any info whether evaluation is parallelizable. So I tried to do it on customized env which has ids indicating different episodes because I'd like the evaluation to enumerate all episodes I have rather than picking random ones. Python/pytorch's multiprocessing is the natural option to parallelize the evaluation but so far I haven't got any success.
from stable_baselines3 import PPO
import torch.multiprocessing as mp
mp.set_start_method(method='forkserver', force=True)
from functools import partial
#mp.set_start_method('spawn')
import warnings
warnings.filterwarnings('ignore')
def eval_fun_one(eval_env,model, id,verbose=True):
# Helper function to facilitate parallelization
model = PPO.load(model)
obs = eval_env.reset(id=id)
steps = 0
done = False
while not done:
steps += 1
action, _ = model.predict(obs, deterministic=True)
obs, reward, done, info = eval_env.step(action)
if verbose:
print("reward={0} on {1}:{2}".format(reward, eval_env.id, steps))
return info
import copy
def eval_fun(eval_env, model, verbose=True):
with mp.Pool(mp.cpu_count()) as pool:
result = pool.map(partial(eval_fun_one,copy.deepcopy(eval_env), model ),eval_env.ids)
return result
The code will stuck and output some generic error messages:
File "/home/ec2-user/SageMaker/envs/py38/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
My question is what would be an easy way to implement it? How should I interpret the error msg.
