View Completed Elasticsearch Tasks

Question

I am trying to run daily tasks using Elasticsearch's update by query API. I can find currently running tasks but need a way to view all tasks, including completed.

I've reviewed the ES docs for the Update By Query API:

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html#docs-update-by-query

And the Task API: https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html#tasks

Task API shows how to get the status of a currently running task with GET _tasks/[taskId], or all running tasks - GET _tasks. But I need to see a history of all tasks ran.

How do I see a list of all completed tasks?

Answer 1

A bit late to the party, but you can check the tasks in the .tasks system index.

You can query this index, as any other regular index. For the updateByQuery task you can do:

curl -XPOST -sS "elastic:9200/.tasks/_search" -H 'Content-Type: application/json' -d '{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "completed": true
          }
        },
        {
          "term": {
            "task.action": "indices:data/write/update/byquery"
          }
        }
      ]
    }
  }
}'

Answer 2

From documentation,

The Task Management API is new and should still be considered a beta feature.It allows to retrieve information about the tasks currently executing on one or more nodes in the cluster.

It still in beta version, so using this currently you'll only able to do these-

1. Possible to retrieve information for a particular task using GET /_tasks/<task_id>, where you can also use the detailed request parameter to get more information about the running tasks(also you can use other params as per the support)

2. Tasks can be also listed using GET _cat/tasks version of the list tasks command, which accepts the same arguments as the standard list tasks command.

3. If a long-running task supports cancellation, it can be cancelled with the cancel tasks API, with POST _tasks/oTUltX4IQMOUUVeiohTt8A:12345/_cancel

4. Task can be grouped with GET _tasks?group_by=parents