3

I want to log a percentage of the success calls in app insights. I came across this post https://learn.microsoft.com/en-us/azure/azure-monitor/app/sampling and I think Fixed-rate sampling is appropriate here. But does this affect all logging equally? Will some errors/failures no longer be logged?

I am looking for a solution that logs a percentage of the success calls, but keeps all failed requests/errors.

Ivan Glasenberg
  • 29,865
  • 2
  • 44
  • 60
Wouter
  • 154
  • 17

2 Answers2

1

I don't think this is supported out of the box, but you can write your own ITelemetryProcessor.

See: https://learn.microsoft.com/en-us/azure/azure-monitor/app/api-filtering-sampling#filtering-itelemetryprocessor

Application Insights in .NET uses a chain of telemetry processors that you can use to filter telemetry, so you can write your own that checks the resultCode (I think that's what Application Insights calls the HTTP status code, but you'll have to double check) of a request telemetry object, and approves it if it's 500 (or 5xx) but only has a 10% chance of sending it if it's 2xx or 3xx. You can override the OKToSend() method to perform the above check on the ITelemetry input, and return true / false accordingly.

Maybe something like (I wrote this in the browser, it won't necessarily work flawlessly as-is):

// Approves 500 errors and 10% of other telemetry objects
private bool OKtoSend (ITelemetry telemetry)
{
    if (telemetry.ResponseCode == 500) {
        return true;
    } else {
        Random rnd = new Random();
        int filter = rnd.Next(1, 11);
        return filter == 1;
    }
}
UpQuark
  • 791
  • 1
  • 11
  • 35
  • I cannot call item["resultCode"] as I get "Cannot apply indexing with [] to an expression of type ITelemetry". item as DependencyTelemetry() has a ResultCode property (string). https://learn.microsoft.com/en-us/dotnet/api/microsoft.applicationinsights.datacontracts.dependencytelemetry.resultcode?view=azure-dotnet – Wouter Apr 18 '19 at 08:22
  • @Wouter I'd guess you want the `RequestTelemetry` (https://learn.microsoft.com/en-us/dotnet/api/microsoft.applicationinsights.datacontracts.requesttelemetry?view=azure-dotnet) class instead of `DependencyTelemetry`, since you are logging requests, for which the correct property for HTTP response code would be `ResponseCode`. – UpQuark Apr 18 '19 at 15:03
  • Seems like everything passes the ITelemetryProcessor. I'll have to treat them separately. – Wouter Apr 19 '19 at 08:08
  • I have another question. Everything needs to be treated separately. I don't want to throw away dependencies that are related to an exception (otherwise there may be incomplete data to debug). Is there a way to keep everything together or figure out which dependencies are related to exceptions/errors? – Wouter Apr 23 '19 at 11:41
  • @Wouter in one of the Rails apps where I use a lot of Application Insights telemetry, there's a custom logger which sends custom traces to app insights and converts arbitrary objects to JSON, which I've used to supplement the default exception-catching with custom data. I don't love that solution and strongly suspect there are better ways you could do it with a custom telemetry processor, but it's an option. – UpQuark Apr 23 '19 at 14:32
  • Thanks for your comment. Any good links that I can take a look at? – Wouter Apr 24 '19 at 11:29
  • @Wouterday-to-day I mostly deal with their API via the [ruby](https://github.com/Microsoft/ApplicationInsights-Ruby) and [node](https://github.com/Microsoft/ApplicationInsights-node.js) libraries so for .NET I'm not positive, but this doc looks pretty close: https://learn.microsoft.com/en-us/azure/azure-monitor/app/asp-net-trace-logs, you can use something like `System.Diagnostics.Trace.TraceWarning("Slow response - database01");` or `telemetry.TrackTrace("Slow response - database01");` to send custom telemetry to app insights that supplement your automatic logging. – UpQuark Apr 24 '19 at 14:09
1

To exclude failed events from being subject to sampling, (while doing sampling for everything else) write a TelemetryInitializer with this logic.

public class PreventSamplingForFailedTelemetryInitializer: ITelemetryInitializer
{
  public void Initialize(ITelemetry telemetry)
  {
        if(failed)
        {
            // Set to 100, so that actual SamplingProcessors ignore this from sampling considerations.
            ((ISupportSampling)telemetry).SamplingPercentage = 100;
        }
   }
}

(Make sure to add this TelemetryInitializer to the TelemetryConfiguration)

Failed or not can be determined from RequestTelemetry and DependencyTelemetry from their `Success` field.

(the last one in FAQ sections has hints to answer your question https://learn.microsoft.com/en-us/azure/azure-monitor/app/sampling#frequently-asked-questions)
cijothomas
  • 2,818
  • 1
  • 13
  • 25
  • btw - you can use the above with AdaptiveSampling. In most cases AdaptiveSampling works best, and is the default sampling enabled in asp.net and asp.net core apps. – cijothomas Apr 18 '19 at 05:27