8

I'm using a small elasticsearch cluster from the elastic cloud.
I need to:

  1. download one of the indices from elasticsearch to my local machine for analysis
  2. set up elasticsearch node locally and restore this index into it.

In Kibana UI in section Snapshot and Restore I can see my snapshots and this hint:

Use repositories to store and recover backups of your Elasticsearch indices and clusters.

But how do I download the actual data from elasticsearch index to my machine (as a bunch of jsons) and import it into elasticsearch running locally?

Viacheslav Shalamov
  • 4,149
  • 6
  • 44
  • 66
  • 1
    Hi, I don't know if that can help you. When I need do something like that I used this useful library. https://github.com/elasticsearch-dump/elasticsearch-dump. – Adrian Jul 13 '20 at 08:41

1 Answers1

4

With a small cluster, and with just a few indices, I'd use the reindex api and just let your local instance index the data directly from your remote.

POST _reindex
{
  "source": {
    "remote": {
      "host": "https://...cloud.es.io:9243",
      "username": "user",
      "password": "pass"
    },
    "index": "source"
  },
  "dest": {
    "index": "dest"
  }
}

Have a look here for the official documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html

Simon Stenström
  • 51
  • 4
  • 1
    Interesting approach, and probably preferred in most cases. How about, if the source and destination have no connectivity, and I actually need to dump the data to file? – Jolta Oct 06 '21 at 08:26
  • @Jolta I've only done this to remote storage, but the snapshot and restore api supports this. A little messier though: https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html – Simon Stenström Oct 11 '21 at 14:59
  • I can't actually see how to "download" the snapshot once created like the OP wanted. Only how to create a snapshot (which I presume is stored on the disk of the ES cluster). – Jolta Oct 13 '21 at 14:19
  • Creating a remote snapshot and restoring it on a local storage would mean the data is on local storage. Analyse that data is however hard. If the OP needs the data as csv for example, an external library like @Adrian mentioned in another comment would be needed. – Simon Stenström Oct 27 '21 at 06:46