How to run a shell script multiple times with different arguments in a Jupyter Notebook

Question

I'm trying to run a Bash script in a jupyter notebook. I have a list of arguments that I have computed in the notebook and stored in a python list, and I want to run the Bash script with each of the parameters once. What I am looking for is essentially the ability to use a for loop to run the script with each parameter.

I am familiar with the %%sh magic, but that does not allow me to use the python for loop inside the shell.

To summarize, looking for something like this:

parameter_list = [arg1, arg2,...argn]
for arg in parameter_list:
    sh shell_script arg

Lots of good answers [here](https://stackoverflow.com/questions/89228/how-do-i-execute-a-program-or-call-a-system-command) on how to execute shell commands in python. — JNevill, Mar 13 '23 at 17:07

Wayne · Answer 1 · 2023-03-13T18:30:35.130

Based on your example pseudocode, you just need to incorporate the use of the exclamation point (covered here & here in Jnevill's resource) to send to a shell and the use of brackets to provide the Python reference to pass along into the shell, like so in a cell:

parameter_list = ["say this ", "say that "]
for arg in parameter_list:
    !echo {arg} >> output.txt

The Jupyter project evolved out of the IPython Notebook project, which evolved out of IPython, and so a lot of the magic commands and conveniences work in Jupyter when using Python there.

Something to keep in mind is though if you need speed, this is inefficient. While great for prototyping, because of speed and more features in some cases you'll want to seek the pure Pythonic routes that show up higher rated among the long list that JNevill points out sooner, rather than later.

How to run a shell script multiple times with different arguments in a Jupyter Notebook

1 Answers1