3

I am using a third-party program designed to be run as a command line program, which outputs files that I later need to use in my code. I am working in Jupyter Lab and want to integrate the function calls into my code. The typical way to run this is:

python create_files.py -a input_a -b input_b -c -d

I then want to call this within my Jupyter notebook. I have been able to get it to work by using !, i.e.:

! python create_files.py -a input_a -b input_b -c -d

The problem with this is that when I want to specify input_a or input_b using variables, this doesn't work because it seems that ! expects a literal string, so to speak.

Is there a more clean way of doing this without having to alter the source code of this program (I have tried looking into that, and the code is written such that there is no simple way to call its main function.)

teepee
  • 2,620
  • 2
  • 22
  • 47
  • Could you try echoing the files in via something like bash? –  Dec 20 '18 at 00:58
  • 3
    How about `subprocess.Popen('python create_files.py -a %s -b %s -c -d' % (input_a, input_b))`? – Chris Dec 20 '18 at 02:14

2 Answers2

1

On Jupyter notebook, the use of subprocess to run command line script goes like this:

Simple command line version:

 dir *.txt /s /b

On Jupyter notebook:

import subprocess
sp = subprocess.Popen(['dir', '*.txt', '/s', '/b'], \
    stderr=subprocess.PIPE, \
    stdout=subprocess.PIPE, \
    shell=True)

(std_out, std_err) = sp.communicate()   # returns (stdout, stderr)

Printing out the error message, just in case:

print('std_err: ', std_err)

Printing out the echoing message:

print('std_out: ', std_out)

I think the example is clear enough that you can adapt it to your need. Hope it helps.

swatchai
  • 17,400
  • 3
  • 39
  • 58
  • This is an excellent solution; thank you. There is one more feature I want to include for the output: The program I run outputs progress text which I would like to show in the Jupyter notebook in realtime as the process runs. Your code will only print out the text as a chunk after the process competes. Thanks! – teepee Dec 20 '18 at 18:26
0

Your question is similar to the one:

How to execute a * .PY file from a * .IPYNB file on the Jupyter notebook?

You may use the following command, which is a little hacky:

%run -i 'create_files.py'

A "correct" way is to use the autoreload method. An example is as follows:

%load_ext autoreload
%autoreload 2
from create_files import some_function
output=some_function(input)

The reference of autoreload is as follows: https://ipython.org/ipython-doc/3/config/extensions/autoreload.html

Hope it helps.

William Wu
  • 1
  • 2