Jupyter Notebook nbconvert without magic commands/ w/o markdown

Question

I have a Jupyter notebook and I'd like to convert it into a Python script using the nbconvert command from within the Jupyter notebook.

I have included the following line at the end of the notebook:

!jupyter nbconvert --to script <filename>.ipynb

This creates a Python script. However, I'd like the resulting .py file to have the following properties:

1. No input statements, such as:

   # In[27]:

2. No markdown, including statements such as:

   # coding: utf-8

3. Ignore %magic commands such as:

   1. %matplotlib inline
   2. !jupyter nbconvert --to script <filename>.ipynb, i.e. the command within the notebook that executes the Python conversion

   Currently, the %magic commands get translated to the form: get_ipython().magic(...), but these are not necessarily recognized in Python.

Answer 1

One way to get control of what appears in the output is to tag the cells that you don't want in the output and then use the TagRemovePreprocessor to remove the cells.

The code below also uses the exclude_markdown function in the TemplateExporter to remove markdown.

!jupyter nbconvert \
    --TagRemovePreprocessor.enabled=True \
    --TagRemovePreprocessor.remove_cell_tags="['parameters']" \
    --TemplateExporter.exclude_markdown=True \
    --to python "notebook_with_parameters_removed.ipynb"

To remove the commented lines and the input statement markets (like # [1]), I believe you'll need to post-process the Python file with something like the following in the cell after the cell you call !jupyter nbconvert from (note that this is Python 3 code):

import re
from pathlib import Path
filename = Path.cwd() / 'notebook_with_parameters_removed.py'
code_text = filename.read_text().split('\n')
lines = [line for line in code_text if len(line) == 0 or 
        (line[0] != '#' and 'get_ipython()' not in line)]
clean_code = '\n'.join(lines)
clean_code = re.sub(r'\n{2,}', '\n\n', clean_code)
filename.write_text(clean_code.strip())

Answer 2

The most obvious solution seems to work for me:

 jupyter nbconvert --to python a_notebook.ipynb --stdout | grep -v -e "^get_ipython" | python

Of course, you can't use something like dirs = !ls in your notebooks for this to work.

Answer 3

Hopefully this will spare people from wasting 2 hours trying to use nbconvert template structure. This is what did the job for me:

nbconvert --to python --RegexRemovePreprocessor.patterns="^%" analysis.ipynb

Answer 4

Jupyter nbconvert has made this a little bit easier with a new template structure.

Templates should be placed in the template path. This can be found by running jupyter --paths

Each template should be placed in its own directory within the template directory and must contain a conf.json and index.py.j2 file.

This solution covers all the details for adding a template.

This template will remove all the of the markdown, magic and cell numbers leaving a "runnable" .py file. Run this template from within a notebook with !jupyter nbconvert --to python --template my_clean_python_template my_notebook.ipynb

index.py.j2

{%- extends 'null.j2' -%}

## set to python3
{%- block header -%}
#!/usr/bin/env python3
# coding: utf-8
{% endblock header %}

## remove cell counts entirely
{% block in_prompt %}
{% if resources.global_content_filter.include_input_prompt -%}
{% endif %}
{% endblock in_prompt %}

## remove markdown cells entirely
{% block markdowncell %}
{% endblock markdowncell %}

{% block input %}
{{ cell.source | ipython2python }}
{% endblock input %}


## remove magic statement completely
{% block codecell %}
{{'' if "get_ipython" in super() else super() }}
{% endblock codecell%}