Change field values using Elasticsearch Painless

Question

I want to check the length of each field under the object alert.

If it's longer than X, change it to value was truncated since it was too long

"_source" : {
          "random" : 123455,
          },
          "@timestamp" : "2021-10-15T21:55:12.938+0000",
          "application" : "app",
          "alert" : {
            "app_key" : "XYZ",
            "host" : "Hostname",
            "check" : "CPU overloaded",
            "status" : "ok"
          },

For some reason, I get this error when trying to apply my script:

    "type" : "script_exception",
    "reason" : "runtime error",
    "script_stack" : [
      "java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1584)",
      "java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1617)",
      "java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1615)",
      "for (alert in ctx._source.alert.entrySet())\n      {\n      ",
      "                               ^---- HERE"
    ],
    "script" : " ...",
    "lang" : "painless",
    "position" : {
      "offset" : 38,
      "start" : 7,
      "end" : 65
    },
    "caused_by" : {
      "type" : "concurrent_modification_exception",
      "reason" : null

Script:

{
  "script": {
    "lang": "painless",
    "source": """
      for (alert in ctx._source.alert.entrySet())
      {
      if (alert.getValue().length() > params.number_of_characters) {
        ctx._source.alert[alert] = "value was truncated since it was too long"
      }
      }
    """,
    "params": {
      "number_of_characters": 5
    }
  }
}

score 2 · Accepted Answer · answered Oct 16 '21 at 04:14

You cannot modify the map directly while iterating over the set returned by entrySet, only through the setValue method of the entry.

Per the doc for entrySet():

Returns a Set view of the mappings contained in this map. The set is backed by the map, so changes to the map are reflected in the set, and vice-versa. If the map is modified while an iteration over the set is in progress (except through the iterator's own remove operation, or through the setValue operation on a map entry returned by the iterator) the results of the iteration are undefined. The set supports element removal, which removes the corresponding mapping from the map, via the Iterator.remove, Set.remove, removeAll, retainAll and clear operations. It does not support the add or addAll operations.

Try with the following script:

for (alert in ctx._source.alert.entrySet()){
    if (alert.getValue().length() > params.number_of_characters) {
        alert.setValue("value was truncated")      
    }      
}

Great ! It worked ! Thank you, thank you, thank you ! <3 – Dennis Oct 16 '21 at 17:34 — Dennis, Oct 16 '21 at 17:34

score 1 · Answer 2 · answered Oct 17 '21 at 08:01

Eventually, with the help of Nicolas, I used this:

PUT _ingest/pipeline/rawpayload-ingest/
{
  "processors": [
    {
      "script": {
        "description": "Loop through all fields in alert object and shorten them if needed",
        "lang": "painless",
        "source": """
          for (field in ctx.alert.entrySet()) {
            if (field.getValue() instanceof String) {
              if (field.getValue().length() > params.number_of_characters) {
              field.setValue(field.getValue().substring(0, params.number_of_characters) + "...truncated")
              }   
            }
          }
        """,
        "params": {
          "number_of_characters": 512
        }
      }
    }
  ]
}

Change field values using Elasticsearch Painless

2 Answers2