2

Is is possible to add papermill parameters to a jupyter notebook manually, e.g. in an editor? Is it possible to add papermill parameters to a .py file and have them persist when converted to a .pynb file?

Context:

I am working on running jupyter notebooks in an automated way via Papermill. I would like to add parameters to a notebook manually rather than using jupyter or jupyter lab interfaces. Ideally these parameters could be added to a python script .py file first. Upon converting the .py file to a .ipynb file the parameters would persist.

My desired workflow looks like this:

  1. Store generic notebook as a < notebook >.py file with parameters in version control repository
  2. Convert python script < notebook >.py to jupyter notebook < notebook >.ipynb
  3. Run < notebook >.ipynb via papermill and pass parameters to it
  4. Use nbconvert to produce output pdf with no code (using exclude_input argument)

Steps 1-3 will be run via a script that can be auotmated. I want to use jupytext to avoid storing the notebooks and all their associated metadata. Currently, the only way I can find to add parameters to a notebook is to add them via jupyter/(lab) interfaces. Then I can run the notebook with papermill. However, this doesn't work with the jupytext conversion.

*Note I would have added the "jupytext" tag to this but it doesn't exist yet and I don't have enough rep to create

EDIT

gooseberry's answer appears to be the correct one.

However, it doesn't actually appear to be necessary to add a parameters tag to your notebook in order to inject parameters via papermill. While papermill will give a no cell with tag parameters found in notebook warning it will still inject the parameters. Additionally, your output notebook from papermill will have a new cell:

    # Parameters
    parm1 = <val passed to papermill>
    parm2 = <val passed to papermill>
    etc.

versions:

  • papermill = 2.2.3
  • jupytext = 1.12.0
Bryce
  • 197
  • 1
  • 1
  • 13

1 Answers1

1

It depends which of the formats you chose for your .py files, but assuming you've chosen the "percent" format, where each new cell is marked with #%%, to add tags you write:

#%% tags=["parameters"]
my_text = ""
print(my_text)

Now you'll be able to inject a different value of my_text using Papermill.

You can find more info about formats here: https://jupytext.readthedocs.io/en/latest/formats.html

Dharman
  • 30,962
  • 25
  • 85
  • 135
gooseberry
  • 26
  • 5
  • Interesting. Adding the % ## (in my case) does get rid of the warning papermill was issuing. However, if I add a print statement like yours within the same cell as the "parameters" tag it doesn't print anything. If I move the print statement to a different cell it does print the correctly input values. In fact, my notebook actually seems to work just fine (with the exception of the warning papermill prints) if I don't add any parameters tag at all. – Bryce Oct 27 '21 at 15:54