Elasticsearch - set updated_at to fields updated via _bulk script

Question

I want to put created_at and updated_at to my document and have below pipeline set to index. This works fine with usual _update API but when I do same update via _bulk API, it doesn't change updated_at field. How can I update updated_at field?

pipeline

{
  "description" : "Pipeline populates created_at and updated_at fields when doc is created or updated.",
  "processors" : [
    {
      "set" : {
        "if": "ctx.created_at == null",
        "field": "created_at",
        "value": "{{_ingest.timestamp}}"
      }
    },
    {
        "set" : {
          "field": "updated_at",
          "value": "{{_ingest.timestamp}}"
        }
    }
  ]
}

index setting

    "settings": {
      "index": {
        "default_pipeline": "add_timestamps"
      }
    },

_bulk query:

{ "update" : {"_id" : "test", "_type" : "_doc", "_index" : "myindex"} }
{ "script": { "source": "ctx._source.counter += params.count", "lang": "painless",  "params": { "count": 4 } }, "upsert": { "counter": 1 } }

^ this bulk query keep updating counter whenever it's called but updated_at is not being updated.

Answer 1

The main reason is because the ingest pipeline doesn't kick in when using the update operation. This is by design and the link I gave above provides more insights into why this is so.

There was another thread with a problem similar to yours which I answered a while back, maybe it helps.