Running multiple Python scripts in different directories in a sequence

Question

I am trying to run multiple experiments, which are in different folder. I want to save the results in the main folder. Something like this:

Main folder

Main_run_file.py
- Results.txt
- Experiment_1
  - Run_file.py
- Experiment_2
  - Run_file.py
- Experiment_3
  - Run_file.py

I already tried with the following code:

import os

mydir = os.getcwd() # would be the MAIN folder
mydir_tmp = mydir + "\Experiment_1 - 40kmh" # add the first experiment folder name
mydir_new = os.chdir(mydir_tmp) # change the current working directory
mydir = os.getcwd() # set the main directory again
import Run_file

mydir = os.getcwd() # would be the MAIN folder
mydir_tmp = mydir + "/Experiment_1 - 60kmh" # add the second experiment folder name
mydir_new = os.chdir(mydir_tmp) # change the current working directory
mydir = os.getcwd() # set the main directory again
import Run_file

However, this only runs the first Run_file and not the second one. Can someone help me with this?

hey, have a look at https://stackoverflow.com/questions/21963270/how-to-execute-a-python-script-and-write-output-to-txt-file/21963346 , he had the same problem — fire frost, Jun 21 '19 at 10:20

olinox14 · Answer 1 · 2019-06-21T10:24:01.963

The second import Run_file is ignored, since python considers this module as already imported.

Either you replace those import statements by statements like this: import Experiment_1.Run_file, without forgetting to add the __init__.py files in your subdirectories,

Or you call your python scripts with subprocess as you would do from a command-line;

from subprocess import call

call(["python", "./path/to/script.py"])

You are also missing the point about current directories:

In the second mydir = os.getcwd() # would be the MAIN folder, mydir is still the previous subfolder
Python importing system does not care about you to change the working directory: imports are managed using the python path which depends on your python installation and the directory where you run your main script.

More here: https://docs.python.org/3/reference/import.html

score 0 · Answer 2 · answered Jun 21 '19 at 10:24

try out subprocess :

import subprocess

with open("Results.txt", "w+") as output:
    subprocess.call(["python", "./Experiment_1/Run_file.py"], stdout=output);
    subprocess.call(["python", "./Experiment_2/Run_file.py"], stdout=output);
    ...

if you need to pass arguments to your Run_file.py, just add it as so :

subprocess.call(["python", "./Experiment_2/Run_file.py arg1"], stdout=output);

Running multiple Python scripts in different directories in a sequence

2 Answers2