0

What can cause an index to drop its linked indices? We have several Index Lifecycle Policies in place for our cluster, but one of them (filebeat ILM policy) descended to 0 (from more then 10) linked indices, thus causing data loss, logs are no longer visible in Kibana.

enter image description here

Re-indexing does not help, since the indices in question are not present when running a listing operation:

GET /_cat/indices/%2A?v=&s=index:desc

Is there a way to retrieve the indices? How can this be prevented in the future? What is the root cause for this situation? We are running Elasticsearch on k8s cluster.

This is the policy in question:

{
  "filebeat" : {
    "version" : 2787,
    "modified_date" : "2021-06-30T13:19:52.560Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_size" : "50gb",
              "max_age" : "1d"
            }
          }
        },
        "delete" : {
          "min_age" : "30d",
          "actions" : {
            "delete" : { }
          }
        }
      }
    }
  }
}
Emilija Trpcevska
  • 53
  • 1
  • 7
  • Can you share the details of the policy? Is there a delete phase configured? – Val Jul 01 '21 at 09:29
  • This is the policy: `{ "filebeat" : { "version" : 2787, "modified_date" : "2021-06-30T13:19:52.560Z", "policy" : { "phases" : { "hot" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_size" : "50gb", "max_age" : "1d" } } }, "delete" : { "min_age" : "30d", "actions" : { "delete" : { } } } } } } }` – Emilija Trpcevska Jul 01 '21 at 09:39
  • The policy is supposed to delete indices older then 30 days only. It deleted everything till a certain point this morning. – Emilija Trpcevska Jul 01 '21 at 09:48

0 Answers0