0

I'm taking ES snapshots for every 15min, snapshot name will be in the format es-snapshot-YYYY-MM-DD-HH-MM-SS

Two questions:

  1. As per documentation it will be incremental, and I never attempted to delete any. But after 3months, now I couldn't able to query the list of snapshots (giving timeouts). Am I doing anything wrong in maintaining my ES snapshots?

  2. If I'm OK to delete few older snapshots, Is there a way to delete by using a prefix like "es-snapshot-2019-06*"?

Venkat Papana
  • 4,757
  • 13
  • 52
  • 74

1 Answers1

0

Am I doing anything wrong in maintaining my ES snapshots?

Probably not. From the Elasticsearch snapshot API docs:

Getting all snapshots in the repository can be costly on cloud-based repositories, both from a cost and performance perspective. If the only information required is the snapshot names/uuids in the repository and the indices in each snapshot, then the optional boolean parameter verbose can be set to false to execute a more performant and cost-effective retrieval of the snapshots in the repository. Note that setting verbose to false will omit all other information about the snapshot such as status information, the number of snapshotted shards, etc. The default value of the verbose parameter is true.

The documentation does suggest that you may experience timeouts as your snapshot repository grows. Running a GET /_snapshot/<repo-name>/_all?verbose=false should give you a successful response.

Is there a way to delete by using a prefix like "es-snapshot-2019-06*"?

Yes, that syntax is supported. In terms of how that works with incremental snapshots, I don't have a reference for this in the docs but my understanding is that the incremental files from your 2019-06-* snapshots would all get merged into your first 2019-07-01 snapshot, and this snapshot would effectively just be a bigger increment if that makes sense.

avik
  • 2,708
  • 17
  • 20
  • thanks for the response. But I couldn't able to delete the snapshots using prefix and wildcard '*' at end. now we decided to use a different s3 bucket for snapshots. – Venkat Papana Aug 29 '19 at 13:00