How to configure a Serilog sink for logging to CloudWatch

Question

I would appreciate some guidance in setting up a Serilog sink for AWS CloudWatch with .NET Core. I'm using appsettings.json for configuration but I am unable to put the settings in the logger. When trying to write log information to CloudWatch this error appears:

An error occurred while starting the application.
AmazonClientException: No RegionEndpoint or ServiceURL configured

Amazon.Runtime.ClientConfig.Validate() in ClientConfig.cs, line 446

AmazonClientException: No RegionEndpoint or ServiceURL configured

Amazon.Runtime.ClientConfig.Validate() in ClientConfig.cs
Amazon.Runtime.AmazonServiceClient..ctor(AWSCredentials credentials, ClientConfig config) in AmazonServiceClient.cs
AWS.Logger.Core.AWSLoggerCore..ctor(AWSLoggerConfig config, string logType) in AWSLoggerCore.cs
AWS.Logger.SeriLog.AWSSink..ctor(AWSLoggerConfig loggerConfiguration, IFormatProvider iFormatProvider, ITextFormatter textFormatter) in AWSSink.cs
AWS.Logger.SeriLog.AWSLoggerSeriLogExtension.AWSSeriLog(LoggerSinkConfiguration loggerConfiguration, IConfiguration configuration, IFormatProvider iFormatProvider, ITextFormatter textFormatter) in AWSLoggerSeriLogExtension.cs
...
Microsoft.AspNetCore.Hosting.ConventionBasedStartup.Configure(IApplicationBuilder app)
Microsoft.Extensions.DependencyInjection.IServiceCollectionExtensions+InjectApiVersionRoutePolicy+<>c__DisplayClass2_0.<Configure>b__0(IApplicationBuilder app)
Microsoft.AspNetCore.Server.IISIntegration.IISSetupFilter+<>c__DisplayClass4_0.<Configure>b__0(IApplicationBuilder app)
Microsoft.AspNetCore.HostFilteringStartupFilter+<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
Microsoft.AspNetCore.Hosting.Internal.AutoRequestServicesStartupFilter+<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder builder)
Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()

My code is as follows:

In Startup.cs:

public Startup(IHostingEnvironment env)
{
    Configuration = new ConfigurationBuilder()
        .SetBasePath(env.ContentRootPath)
        .AddJsonFile("appsettings.json")
        .AddEnvironmentVariables()
        .Build();
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    loggerFactory.AddConsole(Configuration.GetSection("Logging"));
    Log.Logger = new LoggerConfiguration()
        .ReadFrom.Configuration(Configuration.GetSection("AWS.Logging"))
        .WriteTo.AWSSeriLog(Configuration)
        .CreateLogger();

    app.UseMvc();
}

My appsettings.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    }
  },
  "AWS.Logging": {
    "Region": "eu-west-1",
    "LogGroup": "MyLogGroup",
    "LogLevel": {
      "Default": "Information",
      "System": "Information",
      "Microsoft": "Information"
    }
  },
  "AllowedHosts": "*"
}

Credentials for AWS from my developer machine are set by using environment variables:

AWS_ACCESS_KEY_ID=xxxxxxxxxx
AWS_DEFAULT_REGION=eu-west-1
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxx

AWS might block your access if you use secret keys in a config file. Env vars might be ok though as long as they do not come from a file in your deployment package. — Rez.Net, Aug 05 '19 at 07:35
I'm not getting that error however no logs show up in CloudWatch... is there any extra config needed? Would it complain if it couldn't find the local credentials or other reason or fail silently? — Baron, Aug 24 '21 at 14:36

score 7 · Answer 1 · answered Nov 16 '18 at 08:38

7

Just in case anyone has the same problem, this appsettings.json solved it:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    }
  },
  "Serilog": {
    "Using": [ "AWS.Logger.SeriLog" ]
    "Region": "eu-west-1",
    "LogGroup": "MyLogGroup",
    "LogLevel": {
      "Default": "Information",
      "System": "Information",
      "Microsoft": "Information"
    }
  },
  "AllowedHosts": "*"
}

answered Nov 16 '18 at 08:38

JPG

545
3
19

5

Would you please explain how and where did you store the aws credentials and how it was picked by logger. – Vinay Jan 10 '20 at 07:02
You can create a credentials file locally to configure aws credentials for your app. You can refer to this page how to do that - https://docs.aws.amazon.com/sdk-for-net/v2/developer-guide/net-dg-config-creds.html. – Pushpak Jun 28 '21 at 22:27

Vočko · Answer 2 · 2023-05-31T22:59:02.030

I don't think the previous solution is complete, it misses the WriteTo part, the minimal configuration should look like this:

"Serilog": {
    "Using": [
      "AWS.Logger.SeriLog"
    ],
    "LogGroup": "MyLogGroup",
    "Region": "eu-west-1",
    "WriteTo": [
      {
        "Name": "AWSSeriLog",
        "Args": {
          "textFormatter": "Serilog.Formatting.Compact.RenderedCompactJsonFormatter, Serilog.Formatting.Compact"
        }
      }
    ]
  }

This requires Serilog.Formatting.Compact library to be installed (I recommend using it, much neater log records), you can also use the default:

"textFormatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"

This setup is utilising the AWS supplied library AWS.Logger.Serilog, do not forget to add it to your NuGet packages.

Also, if you are not running the app inside the AWS, the AWS credentials are to be provided.

I have just noticed that @JPG configures the `WriteTo` section in the runtime so it will actually work, anyway, this is the full appsettings approach — Vočko, Nov 16 '22 at 08:21

How to configure a Serilog sink for logging to CloudWatch

2 Answers2

Linked