Papermill prints everything on the console

Question

I am working on adding new features to project. Within the project I am logging stdout to a file because some components are printing information useful for debugging. I recently added a new feature to the project which uses papermill to run jupyter notebook. The problem I am having is that papermill is printing everything to the console even if I redirect stdout to a temporary variable.

Below you can see a sample code,

    with io.StringIO() as buf, redirect_stdout(buf):
        pm.execute_notebook(
            path,
            path,
            progress_bar=False,
            stdout_file=buf,
            stderr_file=buf,
            parameters=dict(**params)
        )
        print("!!! redirected !!!")
    print("!!! redirected !!!")

The first print statement successfully gets redirected to the buf while everything pm.execute_notebook prints goes to the console. The last print statement prints on the console as expected.

score 0 · Accepted Answer · answered Nov 21 '22 at 16:08

To solve the problem I had to change the handler and logging level of the logger.

To get the logger:

logger = logging.getLogger('papermill')

To change the logging level:

logger.setLevel('WARNING')

To remove the stream handler:

logger.removeHandler(logging.StreamHandler())

Removing the stream handler and setting the right level solved my problem. Here is a link to Python logging documentation.

Papermill prints everything on the console

1 Answers1