Serilog doesn't write logs into AWS Elasticsearch Service

Question

I am using Serilog to write logs into AWS Elasticsearch Service in my .NET Core application but when logging into Kibana I don't see any logs written.

public Startup(IConfiguration configuration, IHostingEnvironment hostingEnvironment)
{
    const string esUrl = "https://aws-es-thinger.us-west-1.es.amazonaws.com";
    Log.Logger = new LoggerConfiguration()
        .Enrich.FromLogContext()
        .Enrich.WithExceptionDetails()
        .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(esUrl))
        {
            ModifyConnectionSettings = conn =>
            {
                var httpConnection = new AwsHttpConnection("us-east-1");
                var pool = new SingleNodeConnectionPool(new Uri(esUrl));
                var conf = new ConnectionConfiguration(pool, httpConnection);
                return conf;
            },
            AutoRegisterTemplate = true
        }).CreateLogger();
}

I am able to use HttpClient to get response successfully.

Also, I am able to load the Kibana and ElasticSearch urls from my browser. Please help me with what am I missing here.

EDIT Getting below error when connecting in Startup:

System.Net.Http.WinHttpException: A connection with the server could not be established

Did you try enabling [self log](https://github.com/serilog/serilog/wiki/Debugging-and-Diagnostics) to the console to see if there was some sort of logging error? — mason, Jul 18 '18 at 17:13
@mason Thanks using it helped me to get the error. Am getting exception "UnexpectedElasticsearchClientException: A connection with the server could not be established" — User3250, Jul 18 '18 at 17:32
Did you ever figure out what the problem was? I'm having the same issue right now. — Mario Tacke, Aug 17 '18 at 21:41

score 7 · Accepted Answer · answered Aug 18 '18 at 03:51

I needed to provide AWS Access Key and Secret Key in Connection Settings to get it work as below:

Log.Logger = new LoggerConfiguration()
.Enrich.FromLogContext()
.Enrich.WithExceptionDetails()
  .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(awsSettings.ElasticSearchUrl))
  {
      ModifyConnectionSettings = conn =>
      {
          var httpConnection = new AwsHttpConnection(awsSettings.Region
              , new StaticCredentialsProvider(new AwsCredentials
              {
                  AccessKey = awsSettings.AccessKey,
                  SecretKey = awsSettings.SecretKey,
              }));
          var pool = new SingleNodeConnectionPool(new Uri(awsSettings.ElasticSearchUrl));
          var conf = new ConnectionConfiguration(pool, httpConnection);
          return conf;
      },
      ConnectionTimeout = new TimeSpan(0, 10, 0),
      IndexFormat = "xxxxx-{0:yyyy.MM}",
      FailureCallback = e => Console.WriteLine("Unable to submit event " + e.MessageTemplate),
      EmitEventFailure = EmitEventFailureHandling.WriteToSelfLog |
           EmitEventFailureHandling.WriteToFailureSink |
           EmitEventFailureHandling.RaiseCallback,
      FailureSink = new LoggerConfiguration().WriteTo
      .RollingFile(new JsonFormatter(), "XXXXX-{Date}.txt").CreateLogger()
  })
.CreateLogger();

Hope, it helps somebody in future.

Serilog doesn't write logs into AWS Elasticsearch Service

1 Answers1