How do I run multiple python scripts sequentially?

Question

I have a workflow where I run multiple Python scripts sequentially. I run them manually for now.

These scripts reside in different folders, with a few in a single folder. e.g.

 1. C:\Users\harsh\My Drive\Folder\Code\3.py
 2. C:\Users\harsh\My Drive\Folder\Code\4.py
 3. C:\Users\harsh\My Drive\Folder\Code\5.py
 4. C:\Users\harsh\My Drive\Folder\Code\Helper codes\6.py
 5. C:\Users\harsh\My Drive\Folder\Code\Process\7.py

How do I setup a script that can run these programs sequentially?

I am using the PyCharm IDE on Windows 10.

BrokenBenchmark · Answer 1 · 2022-01-11T05:51:25.150

A quick solution would be to use subprocess.call():

import subprocess

filepaths = ["C:\\Users\\harsh\\My Drive\\Folder\\Code\\3.py",
    "C:\\Users\\harsh\\My Drive\\Folder\\Code\\4.py",
    "C:\\Users\\harsh\\My Drive\\Folder\\Code\\5.py",
    "C:\\Users\\harsh\\My Drive\\Folder\\Code\\Helper codes\\6.py",
    "C:\\Users\\harsh\\My Drive\\Folder\\Code\\Process\\7.py"
]

for filepath in filepaths:
    subprocess.call(["python", filepath])

However, you can also use imports, if each script's main method is laid out as such:

def main():
   ...

if __name__ == '__main__':
   main()

Then, your script could look like the following:

import module1
import module2
... # more imports

module1.main()
module2.main()
... # more main methods

to run the main methods sequentially. This is preferable (more concise, no need to spawn new processes), but may require some refactoring.

I found the first option easier to implement using `import subprocess` However I do get the `ModuleNotFoundError: No module named 'pandas'` and many other libraries. How do I implement the second option however I am unclear. — leonardo, Jan 20 '22 at 23:29

How do I run multiple python scripts sequentially?

1 Answers1