Connecting databricks notebook to Appinsights in python code

Question

Our databricks notebook is trigerred via an adf pipeline. I would like to add logging in my python notebook and would like to connect that logging information to be viewed in appinsights. How do I achieve that in python? Any pointers code examples would help

Answer 1

To send logging information from your Databricks notebook to Azure Application Insights, you can use the OpenCensus Python library and logger.

Below is a sample code snippet to configure the logger in your notebook:

   import logging
   from opencensus.ext.azure.log_exporter import AzureLogHandler

   # Replace with your Application Insights instrumentation key
   instrumentation_key = "<your_instrumentation_key>"

   logger = logging.getLogger(__name__)
   logger.setLevel(logging.INFO)
   logger.addHandler(AzureLogHandler(connection_string=f"InstrumentationKey={instrumentation_key}"))

Please check this documentation for details and examples.