0

Say I have a field, data.url. Some our logs contain this field, some do not. I want to return only results where data.url is more than, say, 50 characters long. Really I just need a list of URLs.

I'm trying:

GET _search
{
  "query": {
    "bool": {
      "filter": {
        "script": {
          "script": {
            "source": "doc['data.url'].value.length() > 50",
            "lang": "painless"
          }
        }
      }
    }
  }
}

But get mixed errors:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "script_exception",
        "reason" : "runtime error",
        "script_stack" : [
          "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:90)",
          "org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:41)",
          "doc['data.url'].value.length() > 50",
          "    ^---- HERE"
        ],
        "script" : "doc['data.url'].value.length() > 50",
        "lang" : "painless",
        "position" : {
          "offset" : 4,
          "start" : 0,
          "end" : 35
        }
      },

or

        "type" : "script_exception",
        "reason" : "runtime error",
        "script_stack" : [
          "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496)",
          "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503)",
          "doc['data.url'].value.length() > 50",
          "               ^---- HERE"
        ],
        "script" : "doc['data.url'].value.length() > 50",
        "lang" : "painless",
        "position" : {
          "offset" : 15,
          "start" : 0,
          "end" : 35
        }

With

          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "No field found for [data.url] in mapping with types []"
          }

and sometimes

          "caused_by" : {
            "type" : "illegal_state_exception",
            "reason" : "A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"
          }

This field definitely exists; I can see it in the logs, search in the search field, and using term works:

GET _search
{
  "query": {
    "bool": {
      "filter": {
        "term": {
          "data.url": "www.google.com"
        }
      }
    }
  }
}

What am I missing?

I'm using Elasticsearch 7.8.

jamesdeluk
  • 186
  • 2
  • 16

1 Answers1

0

Since you are using version 7.*, you need to use this below script query

{
  "query": {
    "bool": {
      "filter": {
        "script": {
          "script": {
            "source": "doc['data.url.keyword'].length > 50",
            "lang": "painless"
          }
        }
      }
    }
  }
}

If data.url field is of keyword type, then ignore the ".keyword" at the end of the field

ESCoder
  • 15,431
  • 2
  • 19
  • 42
  • Thanks! Trying this gives me only this error: `"caused_by" : { "type" : "illegal_argument_exception", "reason" : "No field found for [data.url.keyword] in mapping with types []" }` with or without `.keyword` – jamesdeluk May 07 '21 at 17:14
  • @jamesdeluk can you please share your index mapping ? – ESCoder May 07 '21 at 17:29
  • Apologies for the delay. Is this what you mean? `Name: data.url | Type: string | Format | Searchable | Aggregatable | Excluded ` – jamesdeluk May 11 '21 at 14:11
  • @jamesdeluk I am asking for the index mapping. You can get your index mapping using GET mapping API --> https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html And please share the format in which you are indexing your data ? – ESCoder May 12 '21 at 05:39
  • @jamesdeluk can you please reply back to my previous comment ? – ESCoder May 24 '21 at 10:49
  • Sorry, I'm still waiting for my team to get back to me with some more information. – jamesdeluk May 28 '21 at 15:35