5

I would like to see the custom logs that I create inside an AWS Sagemaker JupyterLab notebook (that uses a Glue development endpoint). I want to see them as the output of a notebook cell.

I tried with:

import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.info("I want to see it as output in my notebook cell.")

and also with:

sc = SparkContext.getOrCreate()
glueContext = GlueContext(sc)
logger = glueContext.get_logger()
logger.info("I want to see it as output in my notebook cell.")

but both give no output, in comparison to a print("Hi") which correctly gives the "Hi" output on the notebook.

Although I want the logs on the notebook itself, I've also checked if they are available as CloudWatch Logs, but there the logs of my notebook look like [W 10:42:34.540 NotebookApp] zmq message arrived on closed channel (in any case, I want them on the notebook, not in CloudWatch).

Many thanks

queise
  • 2,286
  • 21
  • 28
  • 1
    Did you ever figure this out? I am running into the same issue. I would like to code glue jobs in notebook and then when ready just copy and paste into the Glue console. Not having a consistent way to log is so annoying. – pitchblack408 Mar 17 '20 at 17:15
  • See https://stackoverflow.com/questions/18786912/get-output-from-the-logging-module-in-ipython-notebook for solution. – Hetal Patel Apr 16 '20 at 23:41

1 Answers1

1
import sys
import logging

logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler(sys.stdout))
logger.info('that should work :-)')
Paul
  • 1,939
  • 1
  • 18
  • 27
  • could you please @pitchblack408 try the proposed solution and let us know if it works? Unfortunately I don't use Sagemaker anymore so I cannot test it. – queise Aug 02 '21 at 10:55