How to configure Azure Application Insights for Asp.Net Core 5.0 to send only 4% of Requests and 100% of Exceptions?

Question

When I use the Data sampling settings on Azure portal — it really works: reduces the amount of data ingestion.

It logs only 4% of the successful requests and this is good.

However it logs only 4% of the all exceptions. And this is bad. Because I miss the lonely and rare exceptions.

How to configure Azure Application Insights for Asp.Net Core 5.0 to send only 4% of Requests and 100% of Exceptions?

Can it be done in Asp.Net Core Startup.ConfigureServices method?

Use AppMetrics to not "log" but collect statistics about different errors. Surely, you don't want gazilion of data, so we, developers, take this approach in high throughput application to collect errors/success/speed/etc. The benefit - it can also be visualized in systems like Grafana and easy to monitor if something bad happens (like leaking your year budget in split-second or unique errors appearing). — eocron, May 24 '21 at 20:43
@eocron, metrics instead of logs is good approach, when an exception is standard and well known. What if each exception is unique/unexpected and you would like to know its type and error message? — Vadim Loboda, May 24 '21 at 20:52
Make unique metric appear. Just like that. Or deal with storage/traffic problems of sending gazilion of errors somewhere. — eocron, May 24 '21 at 20:55
You can use adaptive sampling and provide different sections for requests and exceptions. It will not be 4% but max N requests / sec. Alternative approach you can implement your TelemetryProcessor and put whatever logic you want there. For instance, log all exceptions, failed requests but only 4% successful requests. — ZakiMa, May 25 '21 at 05:39
@ZakiMa, good advice, but — how? Do you have a code sample to support your suggestion? How to configure the adaptive sampling and provide different sections in Asp.Net Core? — Vadim Loboda, May 25 '21 at 06:49
@VadimLoboda, you can start with this for adaptive sampling: https://learn.microsoft.com/en-us/azure/azure-monitor/app/sampling — ZakiMa, May 25 '21 at 07:04
Here a link to how to write TelemetryProcessor: https://learn.microsoft.com/en-us/azure/azure-monitor/app/api-filtering-sampling — ZakiMa, May 25 '21 at 07:04
@ZakiMa, thank you for pointing me out to the ms docs, very valuable resource for code samples (that was sarcasm). Before I dared to put a question in here, I've read the pages you mentioned. In particular, the first one says: "There is no ApplicationInsights.config for ASP.NET Core applications, so all configuration is done via code." I'm hungry for the code, do you have any? — Vadim Loboda, May 25 '21 at 07:47
The first one has an example how to configure it through code (extremely basic one). I was able to find a few questions on stackoverflow using some of class name from that snippet. Not clear whether it is fully extensible to your scenario. My guess is that you should be able to chain UseAdaptiveSampling but I'm not super positive. Right now it is not clear whether your last question about how to even configure adaptive sampling in .NET Core or how to configure for your particular ask. — ZakiMa, May 25 '21 at 08:03
You also can ask a question here: https://github.com/microsoft/ApplicationInsights-dotnet/blob/develop/BASE/src/ServerTelemetryChannel/TelemetryProcessorChainBuilderExtensions.cs Or more exactly in corresponding github repository — ZakiMa, May 25 '21 at 08:04

Tiny Wang · Accepted Answer · 2021-05-27T07:42:47.820

I think you can refer to this chapter

It mentioned that in asp.net core, we can disable the default adaptive sampling and set fixed-rate sampling by builder.UseSampling(fixedSamplingPercentage);, and according to the sdk, we can also set include and exclude options

public static TelemetryProcessorChainBuilder UseSampling(this TelemetryProcessorChainBuilder builder, double samplingPercentage, string excludedTypes = null, string includedTypes = null);

my code here:

var configuration = app.ApplicationServices.GetService<TelemetryConfiguration>();
            var builder = configuration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
            // For older versions of the Application Insights SDK, use the following line instead:
            // var builder = configuration.TelemetryProcessorChainBuilder;
            // Using fixed rate sampling
            double fixedSamplingPercentage = 10;
            string excludedTypes = "Exception";
            string includedTypes = "Request";
            builder.UseSampling(fixedSamplingPercentage, excludedTypes, includedTypes);
            builder.Build();

Each request here will lead to an exception in my test, and here's the application insights detail:

How to configure Azure Application Insights for Asp.Net Core 5.0 to send only 4% of Requests and 100% of Exceptions?

1 Answers1