How to use greedy regexes on Curator filter?

Question

I have set up Curator to delete old Elasticsearch indexes via this filter:

(...)
filters:
- filtertype: pattern
  kind: regex
  value: '^xyz-us-(prod|preprod)-(.*)-'
  exclude:
- filtertype: age
  source: name
  direction: older
  timestring: '%Y.%m.%d'
  unit: days
  unit_count: 7
  exclude:
(...)

However, I realized that Curator uses non-greedy regexes, because this filter catches the index xyz-us-prod-foo-2018.10.11 but not xyz-us-prod-foo-bar-2018.10.11.

How can I modify the filter to catch both indexes?

Consider showing us the real input data and showing us rejected and non-rejected strings — Corion, Oct 26 '18 at 13:01

score 2 · Accepted Answer · answered Oct 30 '18 at 01:14

The answer I gave at https://discuss.elastic.co/t/use-greedy-regexes-in-curator-filter/154200 is still good, though you somehow weren't able to get the results I posted there. Anchoring the end and specifying the date regex worked for me: '^xyz-us-(prod|preprod)-.*-\d{4}\.\d{2}\.\d{2}$'

I created these indices:

PUT xyz-us-prod-foo-2018.10.11
PUT xyz-us-prod-foo-bar-2018.10.11
PUT xyz-us-preprod-foo-2018.10.12
PUT xyz-us-preprod-foo-bar-2018.10.12

And ran with this config:

---
actions:
  1:
    action: delete_indices
    filters:
    - filtertype: pattern
      kind: regex
      value: '^xyz-us-(prod|preprod)-.*-\d{4}\.\d{2}\.\d{2}$'
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 7

The results are fully matched:

2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-preprod-foo-2018.10.12 with arguments: {}
2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-preprod-foo-bar-2018.10.12 with arguments: {}
2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-prod-foo-2018.10.11 with arguments: {}
2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-prod-foo-bar-2018.10.11 with arguments: {}

I'll run more tests and let you know. Thanks for taking the time of posting here too! +1 — dr_, Oct 30 '18 at 08:06

TZubiri · Answer 2 · 2020-04-27T22:33:58.973

1

Curator's implementation of the Regex engine is using the U (Ungreedy) flag.

Ungreedy regexes make star quantifiers lazy by default, adding a "?" modifier under the Ungreedy option would turn it back to Greedy.

Try adding a '?' after the '.*' in your regex

'^xyz-us-(prod|preprod)-(.*?)-'

edited Apr 27 '20 at 22:33

answered Oct 27 '18 at 04:10

TZubiri

886
11
30

How to use greedy regexes on Curator filter?

2 Answers2