2

If in my log I print the latitude and longitude of a given point, how can I capture this information so that it is processed as a geospatial data in elastic search?

Below I show an example of a document in Elasticsearch corresponding to a log line:

{
  "_index": "memo-logstash-2018.05",
  "_type": "doc",
  "_id": "DDCARGMBfvaBflicTW4-",
  "_version": 1,
  "_score": null,
  "_source": {
    "type": "elktest",
    "message": "LON: 12.5, LAT: 42",
    "@timestamp": "2018-05-09T10:44:09.046Z",
    "host": "f6f9fd66cd6c",
    "path": "/usr/share/logstash/logs/docker-elk-master.log",
    "@version": "1"
  },
  "fields": {
    "@timestamp": [
      "2018-05-09T10:44:09.046Z"
    ]
  },
  "highlight": {
    "type": [
      "@kibana-highlighted-field@elktest@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1525862649046
  ]
}
Sufiyan Ghori
  • 18,164
  • 14
  • 82
  • 110
Alessio Frabotta
  • 117
  • 11

1 Answers1

3

You can first separate LON and LAT into their own fields as follows,

grok {
  match => {"message" => "LON: %{NUMBER:LON}, LAT: %{NUMBER:LAT}"}
}

once they are separated you can use mutate filter to create a parent field around them, like this,

filter {
  mutate {
    rename => { "LON" => "[location][LON]" }
    rename => { "LAT" => "[location][LAT]" }
  }
}

let me know if this helps.

Sufiyan Ghori
  • 18,164
  • 14
  • 82
  • 110
  • @AlessioFrabotta, updated the answer :) you can create LON and LAT as separate fields and add a parent field `location`. please read updated answer – Sufiyan Ghori May 10 '18 at 09:22
  • Thanks! One last question: in the logstash log I see that the location field is this: "location" => {"LAT" => "42", "LON" => "13.5"} 42 and 13.5 are strings, mind I expect numbers otherwise the visualization in Kibana does not work. It's correct ? Alternatively, I should construct the "location" field in this way: "location": "42, 13.5" – Alessio Frabotta May 10 '18 at 09:59
  • they are not `STRING` since I used type `NUMBER`. – Sufiyan Ghori May 10 '18 at 10:05
  • you can also force type to float like this if its string :) `%{NUMBER:LON:float}` – Sufiyan Ghori May 10 '18 at 10:08
  • Thanks! however I can not see anything on Kibana by setting the Coordinates Map view :( – Alessio Frabotta May 10 '18 at 10:30
  • in the index I see this: "location": { "properties": { "lat": { "type": "float" }, "lon": { "type": "float" } } }, maybe location should be a geo_point? – Alessio Frabotta May 10 '18 at 10:56
  • Hi, if you want it to be `geo_point` you can simply change `[location]` to `[geo_point]` in mutate filter. – Sufiyan Ghori May 10 '18 at 11:34
  • no, it's the same thing ... I wanted to say that the data type must be geo_point for the location object – Alessio Frabotta May 10 '18 at 12:21