0

Delete all Elasticsearch snapshots at once.

I have a couple of file system snapshot repositories like the following.

PUT /_snapshot/test
{
  "type": "fs",
  "settings": {
    "location": "/mnt/nfs/test",
    "max_restore_bytes_per_sec": "1024mb",
    "compress": "true",
    "max_snapshot_bytes_per_sec": "1024mb"
  }
}

And I have snapshots from 2021 in that repositories. The snapshots look like the following.

  • index_name-2021-02-03
  • index_name-2021-02-04
  • index_name-2021-02-05
  • ...
  • index_name-2023-05-25

The only method I can find is removing snapshots one by one with some shell script, each snapshot deletion process takes 20 minutes. the index_name index size of about 10TB (1 pri + 1 rep). the Elasticsearch version is 7.7.

Questions:

  1. Is there any way to remove multiple snapshots at once? (with regex)
  2. Will it be faster if I remove multiple snapshots at once like DELETE _snapshot/repo_name/index_name-2021-02-03,index_name-2021-02-04?
  3. How can I remove all snapshots in the repository? (removing a repository won't remove snapshots.)

Similar question. How to delete a snapshot data in Elasticsearch?

Musab Dogan
  • 1,811
  • 1
  • 6
  • 8
  • Hey Musab, have those snapshots been taken manually or are they created via a snapshot lifecycle management policy? – Val May 29 '23 at 11:33
  • Hey Val, all snapshots triggered by some shell script, the ES not eligible for to run SLM policy because they are using OSS version of elasticsearch. – Musab Dogan May 29 '23 at 11:35
  • Thanks, though, Snapshot lifecycle management is available in the Free and Open version of ES: https://www.elastic.co/subscriptions – Val May 29 '23 at 11:36
  • SLM is available on free and basic but not in OSS as far as I researched. – Musab Dogan May 29 '23 at 11:42
  • What version are you using? – Val May 29 '23 at 11:45
  • Elasticsearch-OSS 7.7.0 – Musab Dogan May 29 '23 at 11:47
  • You're right, unfortunately, SLM [was not supported](https://web.archive.org/web/20200513183210/https://www.elastic.co/subscriptions) on ES-OSS 7.7.0 (released on 13.05.2020) – Val May 29 '23 at 11:50
  • 1
    Even more troublesome that 7.7.0 didn't even have API endpoints for deleting snapshots, that only came in 7.8.0 – Val May 29 '23 at 11:54
  • Are those indexes present in snapshots also opened in your cluster or only present in snapshots? – Val May 29 '23 at 11:56
  • Some of them still exist some of them not. But I don't need the old once any more – Musab Dogan May 29 '23 at 12:02
  • Ok, then if you close or delete the older indexes you don't need, you should be able to leverage the [`_cleanup` repository endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/snapshots-register-repository.html#snapshots-repository-cleanup) in order to delete all the unreferenced data in the snapshots, and eventually the snapshots themselves – Val May 29 '23 at 12:13

0 Answers0