Track python simpleHttp server logging information in azure application insights application map

Question

We have different microservices(function apps, vm servers, etc) logging to application insights. A simple python http server is hosted on a linux VM, I want this server to receive a traceparent http header (W3C tracing) log the information to application insights. This python server should create a separate node in the Application map.

I am able to extract the span context from traceparent http header and use it to log the information. But i am not able to view it as a separate node in Application map.

There are middlewares for flask,django for tracing the requests. But there is no ready made solution available for python simple http server.

The goal is to have this python server on vm be represented as a separate node in Application map.

Attaching my python script for reference. (this code was written using the code from flask-middleware)

import six

import logging
import sys

from opencensus.ext.azure.log_exporter import AzureLogHandler
from google.rpc import code_pb2
from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.common import configuration
from opencensus.trace import (
    attributes_helper,
    execution_context,
    print_exporter,
    samplers,
)
from opencensus.trace import span as span_module
from opencensus.trace import stack_trace, status
from opencensus.trace import tracer as tracer_module
from opencensus.trace import utils
from opencensus.trace.propagation import trace_context_http_header_format
from opencensus.trace import config_integration

HTTP_HOST = attributes_helper.COMMON_ATTRIBUTES['HTTP_HOST']
HTTP_METHOD = attributes_helper.COMMON_ATTRIBUTES['HTTP_METHOD']
HTTP_PATH = attributes_helper.COMMON_ATTRIBUTES['HTTP_PATH']
HTTP_ROUTE = attributes_helper.COMMON_ATTRIBUTES['HTTP_ROUTE']
HTTP_URL = attributes_helper.COMMON_ATTRIBUTES['HTTP_URL']
HTTP_STATUS_CODE = attributes_helper.COMMON_ATTRIBUTES['HTTP_STATUS_CODE']

EXCLUDELIST_PATHS = 'EXCLUDELIST_PATHS'
EXCLUDELIST_HOSTNAMES = 'EXCLUDELIST_HOSTNAMES'

config_integration.trace_integrations(['logging'])

trace_parent_header= "00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01"
APP_INSIGHTS_KEY = "KEY HERE"

logging.basicConfig(
    format='%(asctime)s traceId=%(traceId)s spanId=%(spanId)s %(message)s')
log = logging.getLogger(__name__)

def callback_function(envelope):
    envelope.tags['ai.cloud.role'] = 'Pixm Agent'


handler = AzureLogHandler(
    connection_string='InstrumentationKey=APP_INSIGHTS_KEY')
handler.setFormatter(logging.Formatter('%(traceId)s %(spanId)s %(message)s'))
handler.add_telemetry_processor(callback_function)
log.addHandler(handler)


propogator = trace_context_http_header_format.TraceContextPropagator()
sampler = samplers.ProbabilitySampler(rate=1.0)
exporter = AzureExporter(
    connection_string="InstrumentationKey=APP_INSIGHTS_KEY")

exporter.add_telemetry_processor(callback_function)
try:
    span_context = propogator.from_headers(
        {"traceparent": trace_parent_header})
    log.info("he...")
    tracer = tracer_module.Tracer(
        span_context=span_context,
        sampler=sampler,
        exporter=exporter,
        propagator=propogator)

    span = tracer.start_span()
    span.span_kind = span_module.SpanKind.SERVER
    # Set the span name as the name of the current module name
    span.name = '[{}]{}'.format(
        'get',
        'testurl')
    tracer.add_attribute_to_current_span(
        HTTP_HOST, 'testurlhost'
    )
    tracer.add_attribute_to_current_span(
        HTTP_METHOD, 'get'
    )
    tracer.add_attribute_to_current_span(
        HTTP_PATH, 'testurlpath'
    )
    tracer.add_attribute_to_current_span(
        HTTP_URL, str('testurl')
    )
    # execution_context.set_opencensus_attr(
    #     'excludelist_hostnames',
    #     self.excludelist_hostnames
    # )

    with tracer.span(name="main-ashish"):
        for i in range(0, 10):
            log.warning("identity logs..."+str(i))

except Exception:  # pragma: NO COVER
    log.error('Failed to trace request', exc_info=True)

score 1 · Answer 1 · answered Aug 05 '21 at 11:27

The Application Map finds components by following HTTP dependency calls made between servers with the Application Insights SDK installed.

OpenCensus Python telemetry processors

You can modify cloud_RoleName by changing the ai.cloud.role attribute in the tags field.

def callback_function(envelope):
   envelope.tags['ai.cloud.role'] = 'new_role_name'
   
# AzureLogHandler
handler.add_telemetry_processor(callback_function)

# AzureExporter
exporter.add_telemetry_processor(callback_function)

Correlation headers using W3C TraceContext to log the information to Application Insights

Application Insights is transitioning to W3C Trace-Context, which defines:

traceparent: Carries the globally unique operation ID and unique identifier of the call.

tracestate: Carries system-specific tracing context. The latest version of the Application Insights SDK supports the Trace-Context protocol.

The correlation HTTP protocol, also called Request-Id, is being deprecated. This protocol defines two headers:

Request-Id: Carries the globally unique ID of the call.

Correlation-Context: Carries the name-value pairs collection of the distributed trace properties.

import logging

from opencensus.trace import config_integration
from opencensus.trace.samplers import AlwaysOnSampler
from opencensus.trace.tracer import Tracer

config_integration.trace_integrations(['logging'])
logging.basicConfig(format='%(asctime)s traceId=%(traceId)s spanId=%(spanId)s %(message)s')
tracer = Tracer(sampler=AlwaysOnSampler())

logger = logging.getLogger(__name__)
logger.warning('Before the span')
with tracer.span(name='hello'):
    logger.warning('In the span')
logger.warning('After the span')

You can refer to Application Map: Triage Distributed Applications, Telemetry correlation in Application Insights, and Track incoming requests with OpenCensus Python

Track python simpleHttp server logging information in azure application insights application map

1 Answers1