Lucene Syntax on ELK 5.3

Question

I'm trying to find a way to parse UserAgent on kibana 5.3 so I could get the following response:

"aws-sdk-java/1.11.76 Mac_OS_X/10.12.5"
"aws-sdk-java/1.11.78 Mac_OS_X/10.11.2"
"aws-sdk-java/1.11.80 Mac_OS_X/10.10.1"
"aws-sdk-java/1.11.83 Mac_OS_X/10.12.2"
"aws-sdk-java/1.10.33 Mac_OS_X/10.12.3"

etc..

I tried with ? to replace numbers

{
  "query": {
    "match": {
      "userAgent": {
        "query": "aws-sdk-java/?.??.?? Mac_OS_X/??.??.??",
        "type": "phrase"
      }
    }
  }
}

or <0-100> for number ranges

{
  "query": {
    "match": {
      "userAgent": {
        "query": "aws-sdk-java/<0-100>.<0-100>.<0-100> Mac_OS_X/<0-100>.<0-100>.<0-100>",
        "type": "phrase"
      }
    }
  }
}

or even with *

{
  "query": {
    "match": {
      "userAgent": {
        "query": "aws-sdk-java/*.*.* Mac_OS_X/*.*.*",
        "type": "phrase"
      }
    }
  }
}

Nothing, Kibana never find anything, I need to do :

{
  "query": {
    "match": {
      "userAgent": {
        "query": "aws-sdk-java/*",
        "type": "phrase"
      }
    }
  }
}

and then it works, but i don't want to have to exclude the rest of the string.

the field userAgent is of type string, he is Searchable and Analyzed. Can anyone could give me a hint on what i'm missing? Thanks

score 1 · Accepted Answer · answered Jun 16 '17 at 22:54

You need a keyword type of sub-field for your userAgent field. Or, if you relied on Elasticsearch to map your string field it should have it by default.

After you add this keyword field, for example like this:

{
  "userAgent": {
    "type" "text",
    "fields": {
      "keyword": {
        "type": "keyword",
        "ignore_above": 256
      }
    }
  }
}

You can use this query:

{
  "query": {
    "regexp": {
      "userAgent.keyword": "aws-sdk-java/[0-9]+\\.[0-9]{2}\\.[0-9]{2} Mac_OS_X/[0-9]{2}\\.[0-9]{2}\\.[0-9]+"
    }
  }
}

Lucene Syntax on ELK 5.3

1 Answers1