5

Is there a way to query for a value of a certain length in Kibana?

For example, given the following two KV pairs:

key: "some"
key: "something"

I would like to search for key.length > 5 and retrieve "something" only.

The other option I see is to add a tag from logstash, but then I'll have to reload a couple hundred GB.

Tony Laidig
  • 1,048
  • 2
  • 11
  • 33

4 Answers4

10

You can use script query to do that in Kibana. Script Query in Kibana, There is an example for script query with key's length more than 5:

{
    "query": {
        "filtered": {
            "filter": {
                "script": {
                    "script": "doc['key'].getValue().length() > 5"
                }
            }
        }
    }
}

And also you need to enable script search in elasticsearch, you need to add the below config into elasticsearch.yml:

 script.engine.groovy.inline.search: on
chengpohi
  • 14,064
  • 1
  • 24
  • 42
  • Thanks! Docs give the reason for enabling groovy search: UPDATE: As a security precaution, starting with version 4.0.0-RC1, Kibana scripted fields default to Lucene Expressions, not Groovy... Lucene Expressions only support operations on numerical fields... – Tony Laidig Mar 24 '16 at 16:08
  • On my data date field was stored as a string can you tell me how to get substring and convert it into a date or explain to me how to query that data. Field name: date Stored as: Sat Jul 17 2021 19:03:30 GMT+0000 (Coordinated Universal Time) I want to filter by date. I want SQL Query if possible – Komal Cr Aug 30 '21 at 15:19
6

You can accomplish this with a regular expression query via the Lucene query syntax. For example key:/.{6,}/ would only match records whose key field had an entry of 6 characters or more.

HT to a similar question about Solr (which also uses Lucene's query engine).

jgibson
  • 1,013
  • 11
  • 9
4

If you can reindex your index or you are just creating it, you can create a custom tokenizer as in the following:

PUT test_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "character_analyzer": {
          "type": "custom",
          "tokenizer": "character_tokenizer"
        }
      },
      "tokenizer": {
        "character_tokenizer": {
          "type": "nGram",
          "min_gram": 1,
          "max_gram": 1
        }
      }
    }
  }, 
  "mappings": {
    "person": {
      "properties": {
        "name": { 
          "type": "text",
          "fields": {
            "keyword": { 
              "type": "keyword"
            },
            "words_count": { 
              "type": "token_count",
              "analyzer": "standard"
            },
            "length": { 
              "type": "token_count",
              "analyzer": "character_analyzer"
            }
          }
        }
      }
    }
  }
}

PUT test_index/person/1
{
  "name": "John Smith"
}

PUT test_index/person/2
{
  "name": "Rachel Alice Williams"
}

GET test_index/person/_search
{
  "query": {
    "term": {
      "name.length": 10
    }
  }
}
Mousa
  • 2,926
  • 1
  • 27
  • 35
3

You can do this by creating a scripted field directly in Kibana.

  • In Kibana, click on Settings tab and then click on your index pattern

  • You should see 2 tabs "Fields" and "Scripted fields".

  • Click on the "Scripted fields" tab. Then "Add scripted field".

  • Enter a "Name" in the Script field and enter the following:-

    doc['key'].value.length > 5

  • Click "Create Field" at the bottom. Now your scripted field will be added & can be viewed from Discover page.

Community
  • 1
  • 1
Yuvraj Gupta
  • 2,475
  • 16
  • 26
  • I don't know back then, but in 2019 it is: `doc['key'].value.length() > 5`. (`length` is a function and needs parenthesis) – HerrIvan Nov 27 '19 at 14:32
  • Should I replace the `key` by the value I added in `New Scripted Field` ? or just keep `doc['key'].value.length > 5` – Mehraj Malik Jan 20 '21 at 13:37