Query existing index from elasticsearch using NEST

Question

We have an elasticsearch installation with kibana and I was wondering if I can write a query using NEST to display the log files a .Net program?

I have tried creating a simple LogMessage POCO class to extract the messages but without success.

[ElasticsearchType(IdProperty = "Id")]
public class LogMessage
{
    public Guid? Id { get; set; }

    public Source Source { get; set; }
}

public class Source
{
    public String Message { get; set; }
}

The search code is very simple.

var local = new Uri("http://servername:9200");
var settings = new ConnectionSettings(local);
var elastic = new ElasticClient(settings);
var request = new SearchRequest
            {
                From = 0,
                Size = 10,
            };

var r = elastic.Search<LogMessage>(request);

What should my LogMessage class look like?

The event in kibana looks like as follows. We use serilog to log messages to elasticsearch server

{
  "_index": "oxyb-01-2016.08",
  "_type": "logevent",
  "_id": "AVbfrnje902hsaMqv0p2",
  "_score": 1,
  "_source": {
    "@timestamp": "2016-08-31T18:19:26.9228089+10:00",
    "level": "Debug",
    "messageTemplate": "Simple message",
    "message": "Simple message",
    "fields": {
      "Session": "AP2016831/08/2016 6:10:19 PM",
      "TX": "TX123-001 None",
      "ExecutionTime": 523792,
      "MethodTime": 109,
      "TransactionId": "6058862c-3f45-4956-8992-eb34eba0fa9b",
      "Workorder": "WoAP70906YY0831031604526",
    },
    "renderings": {
      "0": [
        {
          "Format": "0.00",
          "Rendering": "0.00"
        }
      ]
    }
  },
  "fields": {
    "@timestamp": [
      1472631566922
    ]
  }
}

score 2 · Accepted Answer · answered Sep 01 '16 at 02:31

The source is everything within the _source property in the response

  "_source": {
    "@timestamp": "2016-08-31T18:19:26.9228089+10:00",
    "level": "Debug",
    "messageTemplate": "Simple message",
    "message": "Simple message",
    "fields": {
      "Session": "AP2016831/08/2016 6:10:19 PM",
      "TX": "TX123-001 None",
      "ExecutionTime": 523792,
      "MethodTime": 109,
      "TransactionId": "6058862c-3f45-4956-8992-eb34eba0fa9b",
      "Workorder": "WoAP70906YY0831031604526",
    },
    "renderings": {
      "0": [
        {
          "Format": "0.00",
          "Rendering": "0.00"
        }
      ]
    }
  },

so your LogMessage type should have properties for each of these. It looks like fields can contain arbitrary keys? If that's the case, you may want to map it as a Dictionary<string, object>; if that's not the case, then map it also as a specific POCO type. In the simplest case, a mapping such as this will work

[ElasticsearchType(Name = "logevent")]
public class LogMessage
{
    [JsonProperty("@timestamp")]
    public DateTimeOffset Timestamp {get; set; }

    public string Level {get; set; }

    public string MessageTemplate {get; set; }

    public string Message {get; set; }

    public Dictionary<string, object> Fields {get; set; }

    public Dictionary<string, object[]> Renderings {get; set; }
}

We can test this works as expected with the following

void Main()
{
    var client = new ElasticClient();

    var json = @"{
    ""@timestamp"": ""2016-08-31T18:19:26.9228089+10:00"",
    ""level"": ""Debug"",
    ""messageTemplate"": ""Simple message"",
    ""message"": ""Simple message"",
    ""fields"": {
      ""Session"": ""AP2016831/08/2016 6:10:19 PM"",
      ""TX"": ""TX123-001 None"",
      ""ExecutionTime"": 523792,
      ""MethodTime"": 109,
      ""TransactionId"": ""6058862c-3f45-4956-8992-eb34eba0fa9b"",
      ""Workorder"": ""WoAP70906YY0831031604526"",
    },
    ""renderings"": {
        ""0"": [
          {
          ""Format"": ""0.00"",
          ""Rendering"": ""0.00""
        }
      ]
    }
  }";

  LogMessage log = null;

  using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(json)))
    log = client.Serializer.Deserialize<LogMessage>(stream);

  // do something with log
}

Query existing index from elasticsearch using NEST

1 Answers1