Run Python Script from another script and redirect script output to text file

Question

I would like to run a second Python script and to redirect the output of this second script to a text file. The scripts and the text file are in the same folder. I tried:

import sys
import os
    
path = 'my_path'  # working directory
os.chdir(path)
print("Current working directory: {0}".format(os.getcwd()))
        
sys.stdout = open("output.txt", "w")
execfile("script_I_want_to_run.py")
sys.stdout.close()

The program runs through once completely, after that the following error messages are shown

[SpyderKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 367, in dispatch_shell
    await result
  File "C:\ProgramData\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 665, in execute_request
    sys.stdout.flush()
ValueError: I/O operation on closed file.
[SpyderKernelApp] ERROR | Error in message handler
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 471, in dispatch_queue
    await self.process_one()
  File "C:\ProgramData\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 460, in process_one
    await dispatch(*args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 379, in dispatch_shell
    sys.stdout.flush()
ValueError: I/O operation on closed file.

However, my script is still running. In the txt file there is only the current working directory displayed. At which point do I have to start? Any Help will be appreciated!

Is it exec(open("./filename").read())? This approach leads to the same problems — , Aug 26 '22 at 08:26
maybe you should keep original `sys.stdout` in other variable and later assign it back to `sys.stdout` because other functions may still need it. — furas, Aug 26 '22 at 10:06

score 2 · Accepted Answer · answered Aug 26 '22 at 08:31

2

You can use the subprocess module. Using this submodule also makes the current working directory irrelevant as long as the path to your script is correct.

import subprocess

with open("output.txt", "wb") as output:
    with subprocess.Popen(args=["python", "path/to/script_I_want_to_run.py"], 
                          stdout=subprocess.PIPE) as script:
        output.write(script.stdout.read())

answered Aug 26 '22 at 08:31

Alexander

16,091
5
13
29

1

I like this answer, but for the purpose of the question, would it not be simpler to just `import` the script? – D.L Aug 26 '22 at 08:40
I guess not, since I want to save the output of the called script at the same time – Aug 26 '22 at 08:42
1

@sbrc Yes I know. the code saves the output of the called script and writes it to `output.txt` – Alexander Aug 26 '22 at 08:43
2

you know you could just pass `stdout=output`. – Ahmed AEK Aug 26 '22 at 08:44
2

@AhmedAEK Good point... I always pass PIPE because I usuall print the output to the terminal as well. – Alexander Aug 26 '22 at 08:45
1

for many reasons i wouldn't recommend calling python through subprocess, as the python interpreter may be embedded inside another application or an application is running a custom interpreter, and calling it through subprocess will use the system's default interpreter, which may not be available or doesn't have the required dependencies, so i would recommend running the script inside `exec` with a custom `globals`, and just redirect stdout, inside the same resource manager context. – Ahmed AEK Aug 26 '22 at 08:53
2

@AhmedAEK I have never come across any such issues. Do you run custom python interpreters often? Also the first argument in the list provided to popen can be changed to invoke whatever interpreter the user desires – Alexander Aug 26 '22 at 08:58
if you are working on an app that will be containerized or shipped to customers then using an embedded or custom interpreter becomes a necessity. if you are just developing an open source library or an internal tool then you can just use the system's python interpreter, it depends on the application at hand. – Ahmed AEK Aug 26 '22 at 09:02

score 0 · Answer 2 · answered Aug 26 '22 at 08:32

If you want to run a script from another script, then the process is quite simple. Especially if the second script is in the same folder as the first.

Just import the second script in exactly the same way as you do other imports.

So in the case of the question, this would be sufficient:

import script_I_want_to_run

Run Python Script from another script and redirect script output to text file

2 Answers2