Stack Overflow
Stack Overflow

Questions tagged [papermill]

Papermill is a tool for parameterizing and executing Jupyter Notebooks.

Help / Support / How do I?

For help and support or technical, please submit your questions to Stack Overflow: https://stackoverflow.com/questions/tagged/papermill.

Also check out the papermill FAQ before asking a question.

When looking for info, especially build info, one needs to check in the GitHub site.

Report Issues

Please report bugs, feature requests and installation / compatibility issues on the papermill issues tracker on GitHub. If you need help with using papermill, please do not use the issue tracker for that. Instead, direct your questions to Stack Overflow.

Documentation

Take a look at the Papermill documentation on ReadTheDocs.

72 questions
1
vote
1 answer

Apache Airflow, Papermill and custom kernels

I currently am attempting to execute jupyter notebooks with apache airflow and papermill. In order to use my custom enviorments and custom user kernels I am using the BashOperator and running a command in this format runuser {user} -c 'papermill…
Payton Weatherspoon
  • 11
  • 2
1
vote
1 answer

Running Jupyter Notebook with Different Configurations from CLI

I started to use Jupyter Notebook for my project and I use nbconvert to run it from the command line. I want to know that is there any way to have a .ipynb file and run it with some command-line arguments or reading from stdin somewhere? I read…
Mohammad Moridi
  • 367
  • 3
  • 10
1
vote
1 answer

Running Multiple functions in the same Thread in a specific order

I have three functions that execute 3 different jupyter notebooks using papermill and I want the first (job1) and second (job2) functions to run concurrently and the last function (job3) to run only once the first function (job1) has finished…
TH14
  • 622
  • 10
  • 24
1
vote
2 answers

Jupyter notebook execution based on user entered automation parameters

I am trying to build a service that would allow users using notebook to set automation parameters in a cell like the starting time as to when the notebook should start executing. The service would then take this input time and execute the notebook…
Deesha
  • 538
  • 8
  • 27
1
vote
1 answer

Can I use Papermill and Scrapbook with AWS EMR Notebooks?

I have several notebooks which are ran by a "driver" notebook using papermill. These notebooks use the scrapbook library to communicate information to the driver. The driver then passes this information as parameters to other notebooks. I want to…
Ryan Walden
  • 167
  • 10
1
vote
1 answer

Handling python error in bash script with no error codes

I am running a python cli tool, Papermill. I am running jupyter notebooks and want to be able to detect assertionErrors when the notebook are ran. When I run papermill from a bash script I always get a 0 exit code, even when there is a python error…
OracleOfDelphi4
  • 11
  • 1
1
vote
2 answers

Hide code cells in Jupyter Notebook, execute with Papermill, transform to PDF with nbconvert

I want to run a python program (not command line) using papermill to execute a jupyter notebook and then transform it into a PDF. The idea is working, but I'm unable to hide the input cells. For papermill report_mode = True is supposed to hide input…
PKL
  • 95
  • 7
1
vote
2 answers

GCFS write exceeding quota and throws HTTP error 429

When running GCFS application via papermill[gcs] papermill gs://my-bucket/test.ipynb gs://my-bucket/output/test.ipynb I'm getting Error: HTTP 429 exceeds the rate limit. Works if output notebook is written locally: papermill…
gogasca
  • 9,283
  • 6
  • 80
  • 125
0
votes
0 answers

How can I disable IPython startup scripts when executing a notebook via Papermill?

I have a Juypyter notebook that is executed by various users using Papermill. I want the outputs to be reproducible, independent of the user that is executing the notebook. I have found that some users have IPython startup scripts and these scripts…
Will Holtz
  • 677
  • 5
  • 17
0
votes
1 answer

Papermill Azure Function App: "No such kernel named python3" when executing a notebook

I am trying to develop an Azure function app that executes jupyter notebooks stored in blob storage through papermill. It's all working up until the point of papermill.execute_notebook(...) where I then get the following error: File…
Chad Goldsworthy
  • 69
  • 1
  • 7
0
votes
1 answer

AWS Lambda & Papermill: Unable to import module 'lambda_function': No module named 'rpds.rpds'

I'm trying to import papermill inside an AWS python lambda function. To to this, I first ran pip3 install papermill -t Desktop/python locally, zipped the python file and uploaded this as a lambda layer. I am now getting this error: { …
Chad Goldsworthy
  • 69
  • 1
  • 7
0
votes
0 answers

Papermill + re-running specific cell

I'm using papermill to run my input jupyter notebook for different parameters and then export them all to htmls. In my input notebook, I have the following cell, because I need to rerun a cell earlier in the notebook. from IPython.display import…
Sarah
  • 1
0
votes
0 answers

Executing an R notebook via papermill in a separate Python notebook

I have a notebook using a custom_python kernel that calls papermill to execute an another notebook which uses a conda_r_custom_r kernel. When I run the following import papermill as pm pm.execute_notebook( 'my-r-notebook.ipynb', …
jtorca
  • 1,531
  • 2
  • 17
  • 31
0
votes
1 answer

How to set the Jupyterlab notebooks default kernel with Papermill in prepare_only mode

Is there a way to set the default kernel in JupyterLab in Azure ML notebooks, in prepare_only mode? I know it's only a few clicks away to change it on the top right - but I need to do this many times a day, and I often forget. Having the right…
Tickon
  • 1,058
  • 1
  • 16
  • 25
0
votes
0 answers

can´t disable ploomber log after activating it in jupyter notebooks

I tested the plomber logging feature using notebooks for train/test some models by adding this to the pipeline.yaml file (papermill parameter) papermill_params: log_output: True and called my ploomber notebook execution with this…
Forge
  • 1,587
  • 1
  • 15
  • 36
1 2
3
4 5