2

Trying to have curator clean up an index I have named "syslog"

I have this action yml:

actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than hour
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: syslog
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: hours
      unit_count: 1
      exclude:

I tried both "hours" and "days" but I keeping getting this message from curator:

 Skipping action "delete_indices" due to empty list: <class 'curator.exceptions.NoIndices'>
John Y
  • 14,123
  • 2
  • 48
  • 72
red888
  • 27,709
  • 55
  • 204
  • 392

1 Answers1

4

You are telling Curator that you want it to identify indices that have a prefix value of syslog, and also have a Year.Month.Day timestring in the index name. This would match, for example, if the index name were syslog-2017.03.14. But if the name of the index is only syslog, this filter set will not match.

Perhaps you want to catch the index age relative to when the index was created. In this case you'd set

actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than hour
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: regex
      value: '^syslog$'
    - filtertype: age
      source: creation_date
      direction: older
      unit: hours
      unit_count: 1

This will match only an index named syslog whose creation_date is at least one hour older than execution time.

untergeek
  • 863
  • 4
  • 13
  • Is there an advantage of matching on name instead of creation_date? Is creation_date local to the server or is that property attached to the index? Like if the index was moved to another server would that property remane? – red888 Mar 17 '17 at 03:14
  • 1
    "Is there an advantage of matching on name instead of `creation_date`?" Not really, but if you are trying to ingest older data, name is a better way to guarantee that you have the right time frame. "Is `creation_date` local to the server?" No, it's epoch time, plus milliseconds, ergo it is always in UTC. – untergeek Mar 18 '17 at 22:06
  • what i mean is, is the creation_date property attached to the event object or is that value calculated by the server. – red888 Mar 20 '17 at 14:46
  • 1
    An elasticsearch cluster keeps a fully propagated set of cluster metadata. All index information is stored there. When a new index is created, a time stamp is stored as creation_date. It is not calculated. API calls to the cluster access this metadata. Because the metadata is cluster-wide, it doesn't matter where an index or its shards reside, the creation_date is the same. – untergeek Mar 21 '17 at 15:09