0

I have an ES cluster for which we started to stream data using an Azure Function (on python) into Azure Blobs. Basically, I am getting the last 5 min data in JSON format, compressing the list of JSON's and storing them on the Blob for live data. We have an S3 bucket where snapshots have gone into. Is there a way to read the JSON's in the files with python without having to restore the indexes on a separate ES to get historical data? It's a lot of data and having to run a separate instance just to restore it so then I can extract the JSON's is just not cost-effective.

Andrei Budaes
  • 591
  • 7
  • 22

1 Answers1

1

No, it's not possible, to access the data in the snapshots you will need to restore them.

But you do not need a separate instance/cluster, you can restore using the same instance/cluster you have and renaming the restored index as the example in the documentation.

leandrojmp
  • 7,082
  • 2
  • 19
  • 24
  • The reason I need a separate cluster is that this is historical data that if we were to restore on our current cluster (production) will interfere with the live data being sent to it and affect its performance – Andrei Budaes Aug 01 '21 at 18:25