2

I want to run a papermill Python script to execute a .ipynb file. As of now, it runs on local but I want to run it on another server, say another Docker container.

I tried reading the documentation(https://buildmedia.readthedocs.org/media/pdf/test-papermill/latest/test-papermill.pdf) but it states how to create a new engine, nothing about how to make remote connections and define their kernels.

Aviral Srivastava
  • 4,058
  • 8
  • 29
  • 81
  • What kernel are you wanting to use in the notebook? Python? – cwalvoort May 14 '19 at 22:40
  • I ask because if you are using Spark or Pyspark, there is some additional kernel information you need to provide if you are doing this through the spark magic. Also, posting your `kernel.json` files would help give us a better idea on how to help you. – cwalvoort May 15 '19 at 01:55
  • 1
    normal Python3 kernel – Aviral Srivastava May 15 '19 at 06:19
  • As long as jupyter and papermill are installed in your other environment, or built in your docker image, a standard python kernel should work "out of the box" with papermill. You will need to post more information about the code you're executing and the errors you're getting. – cwalvoort May 15 '19 at 11:18
  • Here you have an script to run papermill in a remote machine: https://github.com/GoogleCloudPlatform/ml-on-gcp/blob/master/dlvm/tools/scripts/notebook_executor.sh Instructions here: https://blog.kovalevskyi.com/gcp-notebook-executor-v0-1-2-8e37abd6fae1 – gogasca Sep 02 '19 at 18:39
  • 1
    @AviralSrivastava I am facing the same issue, were you able to accomplish this? thanks! – cyn0 Jan 30 '20 at 01:04

0 Answers0