0

I have a asp.net core web app (mvc) project that writes logs to elasticsearch and read logs from elasticsearch when certain things happen in the application.

Is there a way you could correlate logs that occur within the same browser transaction? (i.e. opening up a browser, user clicks button on each page redirecting them to another page until they reach the end of the page where they are finished then they close the browser) I saw something about APM but is there alternative to group these logs together as a single transaction if this makes any sense.


UPDATE:

I followed exactly the instructions from apm installation tutorial guide for Windows. When I got to step 3 which was to start the APM server it said to run the cmd Start-Service apm-server but nothing happened. So when I attempted to start the server with the cmd ./apm-server -e instead, it did some stuff that did not look like the format it should be displaying in powershell as admin.

enter image description here

I then checked the apm server status and it showed as it was correctly set up. I did not do step 2 because I am running it localhost.

enter image description here

Moving on to the last step which is the APM Agents. I went under the tab .NET and followed the instructions... installing Elastic.APM.NetCoreAll nuget package and did the following...

Startup.cs file I added the app.UseElasticApm(Configuration, new HttpDiagnosticsSubscriber(), new EfCoreDiagnosticsSubscriber());

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.UseElasticApm(Configuration, 
               new HttpDiagnosticsSubscriber(), // Enable tracing of outgoing HTTP requests
               new EfCoreDiagnosticsSubscriber()); // Enable tracing of database calls through EF Core

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseStaticFiles();

            app.UseRouting();

            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Home}/{action=Index}/{id?}");
            });
        }

in my appsettings.json file:

{
  "ApplicationName": "customer-simulation-es-app",
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Information",
        "System": "Warning"
      }
    }
  },
  "ElasticConfiguration": {
    "Uri": "http://localhost:9200"
  },
  "ElasticApm": {
    "ServerUrl": "http://localhost:8200",
    "ServiceName": "CustomerApp",
    "Environment": "production",
    "CloudProvider":  "none",
    "LogLevel": "Trace"
  },
  "AllowedHosts": "*"
}

This is what I see on the .exe file application when program starts running:

enter image description here

I run the program and perform some operations on the browser. I then go to check the agent status and it says no data has been received from agents yet. But I did it exactly how they wanted it.

enter image description here

Am I missing something that is not in the guide?

NoviceCoder
  • 449
  • 1
  • 9
  • 26

1 Answers1

1

Using the APM would be the easiest way to achieve this as it is as simple as pie to install and get running.

The .Net APM Agent understands the concept of transactions and can help a great deal with log correlation (with both Serilog or NLog)

Val
  • 207,596
  • 13
  • 358
  • 360
  • Thank you, I'll go ahead and attempt to use APM and install the .net agents for apm! – NoviceCoder May 27 '21 at 15:57
  • Awesome, let us know if you encounter any issues, but it should be straightforward. – Val May 27 '21 at 17:01
  • I am running across an issue I cannot seem to figure out why the data has not been received from agents when I implemented it in the project and ran it. Please see the section where it says Update in the original post. – NoviceCoder May 27 '21 at 17:41
  • Greta job so far. Can you check [this page](https://www.elastic.co/guide/en/apm/agent/dotnet/current/setup-asp-net-core.html) and add the last code snippet as well so that you can get some tracing information on what's happening under the hood? Also make sure to check the [troubleshooting page](https://www.elastic.co/guide/en/apm/agent/dotnet/current/troubleshooting.html#no-data-sent) for hints on what coud go wrong. – Val May 28 '21 at 04:05
  • Thank you! I went ahead and removed the line inside of `setup.cs` that had `app.UseAllElasticApm();` to `app.UseElasticApm(Configuration, new HttpDiagnosticsSubscriber(), new EfCoreDiagnosticsSubscriber());` then when I ran the project and performed operations on the application and went to check the agent status it still keeps telling me no data is passed. I then checked the troubleshooting page but didn't find what it was showing me on the .exe file. Please see the updated section for my findings. – NoviceCoder May 28 '21 at 13:29
  • It seems there's an issue with `AwsCloudMetadataProvider` not being able to reach some host, but I'm not sure this is related to APM... – Val May 28 '21 at 14:22
  • How would I be able to fix this `AwsCloudMetadataProvider` issue? I am not even using AWS... – NoviceCoder May 28 '21 at 14:43
  • It is actually related to the the APM and has been reported by someone and the solution is [here](https://github.com/elastic/apm-agent-dotnet/issues/1283). – Val May 28 '21 at 15:06
  • I went ahead and added `"CloudProvider": "none"` to the appsettings.json file and when I run the program it says User profile is available using directory as key repository and windows DPAPI to encrypt keys at rest. But Im not using any keys just confused why it is saying that. Also when I check the agent status it still shows No data has been received from agents yet. Please see updated post. – NoviceCoder May 28 '21 at 15:53
  • We're progressing as there are no more errors in the logs. Not yet sure why no data is coming up, though. – Val May 28 '21 at 15:56
  • We definitely are, I thank you. Neither do I, am I missing a step? – NoviceCoder May 28 '21 at 15:58
  • Can you [change the `LogLevel` to `Trace`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/config-supportability.html#config-log-level) (by default it's `Error`) in the configuration? – Val May 28 '21 at 16:04
  • I added it to the appsettings.json file as `"LogLevel" : "Trace"` and ran the application and it still came up as the screenshot of the exe file, but I proceeded with performing some operations and it still came back with the same message when I tried to check the agent status... do you think I should switch back to `.UseAllElasticApm();` instead of how it is in the post? Also would I need to include the `UseElasticApm()` how I have it in the `startup.cs` to the `Program.cs` file under `CreateHostBuilder()`? – NoviceCoder May 28 '21 at 16:14