Stack Overflow
Stack Overflow

Questions tagged [elasticsearch-curator]

Tool for curating and managing Elasticsearch time-series indices

Have time-series indices in Elasticsearch? This is the tool for you!

Like a museum curator manages the exhibits and collections on display, Elasticsearch Curator helps you curate, or manage your time-series indices.

This tool is written in Python and it started from the need of automatically cleaning up Elasticsearch indices older than a certain period of time. The usual usage scenario is with Elasticsearch and Logstash, but it can be used for any Elasticsearch indices that are time-based. Not only it's able to clean up (delete) indices older than a certain period of time, or larger than a specified amount, but can also perform management operations like closing indices, aliases changes, optimize indices, replica count changes, snapshots operations.

More about this tool you can find here. Project is hosted on github.

99 questions
28
votes
2 answers

Python pip package RequestsDependencyWarning when installing elastic-search-curator

I installed elastic search curator via the following command. sudo pip install -U elasticsearch-curator It all installed OK. However now when I do the following curator_cli --version I get the following dependency warning. …
Robbo_UK
  • 11,351
  • 25
  • 81
  • 117
7
votes
1 answer

Elasticsearch cluster leaving shards unassigned

We're running an elasticsearch cluster for logging, indexing logs from multiple locations using logstash. We recently added two additional nodes for additional capacity whilst we await further hardware for the cluster's expansion. Ultimately we…
Steve Elliott
  • 659
  • 5
  • 13
5
votes
3 answers

How to delete data from a specific index in elasticsearch after a certain period?

I have an index in elasticsearch with is occupied by some json files with respected to timestamp. I want to delete data from that index. curl -XDELETE http://localhost:9200/index_name Above code deletes the whole index. My requirement is to delete…
ADARSH K
  • 606
  • 1
  • 8
  • 21
4
votes
2 answers

How to use greedy regexes on Curator filter?

I have set up Curator to delete old Elasticsearch indexes via this filter: (...) filters: - filtertype: pattern kind: regex value: '^xyz-us-(prod|preprod)-(.*)-' exclude: - filtertype: age source: name direction: older timestring:…
dr_
  • 2,400
  • 1
  • 25
  • 39
3
votes
1 answer

Snapshot of elasticsearch using Curator failing

I am trying to take a snapshot of ES Index having name of the type reincarnation.2015.07.21and store it in Amazon S3 with the below curator command:- curator --host locahost --port 9200 snapshot --repository walle_elk_archive indices --older-than…
tuk
  • 5,941
  • 14
  • 79
  • 162
2
votes
1 answer

PIP install failing with Syntax error for library

I am trying to install curator library for Lambda packaging using PIP3. For that i am running the command pip3 install curator -t ./ I am getting the following error : Collecting curator Downloading…
We are Borg
  • 5,117
  • 17
  • 102
  • 225
2
votes
2 answers

How does ELK Curator manage monthly/weekly indexes when the "time-unit" = "days"

We have a custom wrapper on top of the standard elasticsearch and curator implementation at our company. I would like to know what would be the behavior of curator dealing with "Monthly/Weekly" indexes when the default "time-unit" is set to…
kriket
  • 135
  • 7
2
votes
1 answer

Curator not finding my index

Trying to have curator clean up an index I have named "syslog" I have this action yml: actions: 1: action: delete_indices description: >- Delete indices older than hour options: ignore_empty_list: True …
red888
  • 27,709
  • 55
  • 204
  • 392
2
votes
1 answer

Error while running curator for deleting older indices

I want to delete the elasticsearch indices older than 7 days. So I have installed curator 4.2 as my elasticsearch version is 5.0.0 (curator version before 4.x are not compatible with elasticsearch v5) we need to create configuration file and action…
Sachchit Bansal
  • 486
  • 5
  • 11
2
votes
1 answer

Elasticsearch Marvel set .marvel-2015-* indices to max number

Hey I am using marvel alongside elasticsearch and I am trying to avoid using curator to clean indices that look like ".marvel-2015-*" is there a specific config or set of configs that I can use to accomplish this. Note: I am using chef to provision…
alb
  • 29
  • 4
2
votes
1 answer

Is rolling indices possible with an 'atomic alias move' for the curator from ElasticSearch?

In the old times I have written and used my own plugin 'rollindex' to handle the 'rolling' of e.g. log data. Then I've seen curator and thought that this is a good replacement as it can add and remove aliases. But how can I move aliases atomically?…
Karussell
  • 17,085
  • 16
  • 97
  • 197
1
vote
0 answers

Reindex / Shrink indices to original names - ElasticSearch Curator

I am using Elasticsearch Curator to manage my indices. I want to shrink / reindex a given index and then rename it back to the original index name. I can do this manually because I know the indices name, but it's hard to do this using Curator. This…
alina
  • 291
  • 2
  • 9
1
vote
1 answer

Unable to interpolate environment variable inside POD via the helm chart for stable/elasticsearch-curator

I am trying to make an helm chart for my custom elk stack. I use the stable/elasticsearch-curator chart as a dependency. In my values.yaml file, I use some env variables to pass the elasticsearch host: esClusterName: &esClusterName…
OLIVIER
  • 858
  • 9
  • 18
1
vote
2 answers

Facing error while rollover based on alias and template in elasticsearch 5.6.0

I am facing a strange problem while rollover of a particular alias. Error: { "error": { "root_cause": [ { "type": "illegal_argument_exception", "reason": "index name [twitter] does not match pattern '^.*-\\d+$'" } …
Moksh
  • 185
  • 1
  • 6
  • 16
1
vote
0 answers

Unable to register a snapshot repository in elasticsearch using repository-s3 plugin

I'm running an Elasticsearch V6.3.2 cluster on Kubernetes, and I've installed the repository-s3 plugin. I've added my AWS Access Key and Secret Key to the elasticsearch.keystore. I'm trying to register my snapshot repository with: curl -X PUT -H…
3minus1
  • 149
  • 6
1
2 3 4 5 6 7