logstash geoip.location mapping to geo_point not working

Question

I can see in my default mappings geoip.location is mapped to geo_point type:

GET myserver:9200/_template
    {
      "logstash": {
        "order": 0,
        "version": 50001,
        "template": "logstash-*",
        "settings": {
          "index": {
            "refresh_interval": "5s"
          }
        },
        "mappings": {
          "_default_": {
            "dynamic_templates": [
              {
                "message_field": {
                  "path_match": "message",
                  "mapping": {
                    "norms": false,
                    "type": "text"
                  },
                  "match_mapping_type": "string"
                }
              },
              {
                "string_fields": {
                  "mapping": {
                    "norms": false,
                    "type": "text",
                    "fields": {
                      "keyword": {
                        "type": "keyword"
                      }
                    }
                  },
                  "match_mapping_type": "string",
                  "match": "*"
                }
              }
            ],
            "_all": {
              "norms": false,
              "enabled": true
            },
            "properties": {
              "@timestamp": {
                "include_in_all": false,
                "type": "date"
              },
              "geoip": {
                "dynamic": true,
                "properties": {
                  "ip": {
                    "type": "ip"
                  },
                  "latitude": {
                    "type": "half_float"
                  },
                  "location": {
                    "type": "geo_point"
                  },
                  "longitude": {
                    "type": "half_float"
                  }
                }
              },
              "@version": {
                "include_in_all": false,
                "type": "keyword"
              }
            }
          }
        },
        "aliases": {}
      }
    }

I have this in a logstash filter to get the geoip data from one of my fields:

geoip {
  source => "myField"
  add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
  add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
}
mutate {
  convert => [ "[geoip][coordinates]", "float"]
}

But, when it gets into ES the location field is a "number"

Also if I try to use the kibana map visualization it says "No Compatible Fields"

Why isn't the default mapping working?

Edit: I also tried just geoip { source => "myfield"} because my default mapping is using location and not coordinates, but this did not work.

I also got rid of the mutate and tried this, but it doesn't work either:

geoip {
  source => "myfield"
  add_field => [ "[geoip][location]", "%{[geoip][longitude]}" ]
  add_field => [ "[geoip][location]", "%{[geoip][latitude]}"  ]
}

score 5 · Answer 1 · answered Mar 13 '17 at 20:03

5

You probably just have a typo (point 1), but including several other things to note.

You are using geoip.coordinates in logstash and geoip.location in your _mapping
You need to remove the convert => [ "[geoip][coordinates]", "float"] - that's wrong.
Once there is data in a field in ES, you can't convert it to a geo_point without re-indexing your data with a new mapping (if you are in a development stage, that usually means delete any indexes and re-insert your data).
After any mapping change, there is a button in kibana settings to reload your mapping.

answered Mar 13 '17 at 20:03

Alcanzar

16,985
6
42
59

where is the typo? – red888 Mar 13 '17 at 20:13
You have coordinates in one place and location in another. Also point 2 also applies – Alcanzar Mar 13 '17 at 20:23
cool I'm testing with all of that commented out, if the mapping is already setup right by default, maybe I just need geoip {source => "myField" } and nothing else – red888 Mar 13 '17 at 20:28
yes -- `geoip` by default creates a `[geoip][location]` field which is what your mapping is set to turn into a `geo_point` type. All of that should work as long as you wipe out your index and recreate it. (so remove your `mutate` and `add_field`s) – Alcanzar Mar 13 '17 at 21:03
1

Point number 4 worked for me like a charm. I was doing every step except that! Thanks a lot. – Suyash Apr 26 '18 at 04:51
Unfortunately, I am unable to find the button for step 4. Can someone please guide ? – Varun Chandak Feb 12 '19 at 12:47
Its under Management/Index Patterns/select pattern name on left, then in the main window on the same line it shows the indexname-* in bold text, there's a star, a reload button, and a red trashcan. It's the reload button between the * and trashcan. – Alcanzar Feb 12 '19 at 19:44

score 5 · Accepted Answer · answered Mar 15 '17 at 06:23

5

https://discuss.elastic.co/t/geoip-location-not-getting-mapped-to-a-geo-point-type/78625

Issue was my ignorance of how index mapping works. This default mapping is being applied to indices matching the name "logstash-*" and my index name did not match that.

Changing my index name to logstash-myindex worked.

answered Mar 15 '17 at 06:23

red888

27,709
55
204
392

Just to elaborate a bit more on this solution: A default **"legacy index template"** with the name **"logstash"** is doing this trick (at least nowadays). It is used for the index pattern **"logstash-*"**. That's the reason why the renaming helped. You can find the Templates in _"Stack Management / Index Management / Index Templates"_ (usually `www.your-es-server.com/app/management/data/index_management/templates`) – SeparateReality Feb 18 '21 at 08:31

logstash geoip.location mapping to geo_point not working

2 Answers2