5

I have an index named "products" and another one named "products_temp". I want to fill the "products_temp" with new products then delete the "products" index and rename the "products_temp" index to "products". Is this possible in Elasticsearch? If it is, what's the recommended approach?

I have to repeat this "products resync" process every day once.

tedi
  • 6,350
  • 5
  • 52
  • 67
  • 1
    Depending on the version of ES you are using: [ES Reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) – Sohaib Farooqi Dec 21 '17 at 07:54
  • I'm currently using version 5.6.2. But I can update to 6.1.1 as the project is new. If I'm not mistaken then the Reindex API is supported since version 2.3.0? I read it here: https://www.elastic.co/blog/reindex-is-coming so it should work. Thanks! Will try it. – tedi Dec 21 '17 at 08:04
  • 1
    Yes Reindex api is supported for version 5.6 too. – Sohaib Farooqi Dec 21 '17 at 08:07

2 Answers2

3

Renaming indices is not possible. Recommended way is to use aliasand point that to whatever index you want.

Refer: How to rename indices

Praneeth
  • 731
  • 3
  • 10
  • Ok. So how would I do this with aliases? If I have the index "products" and then I create a new one "products_temp". I fill the temp with new products and then delete "products" and add the alias "products" to the "products_temp" index (this is fine but i have to repeat these steps programmatically). What happens the next time that I want to "resync" the products? Then I can't repeat the same process as I can't create a new index named "products_temp" because it already exists. Then I would have to save a flag or something somewhere. Is this the way to do it? – tedi Dec 21 '17 at 08:11
  • This is probably the faster way (performance wise) to do it. As with reindex i would have to delete all old documents first. – tedi Dec 21 '17 at 08:44
  • Every time you're creating a new index, append a timestamp to it(`product_1513845889`) and once the index is created successfully, point the alias to this newly created index, all of this can be done programatically. – Praneeth Dec 21 '17 at 08:47
  • But then i need to save the timestamps somewhere so that I can delete them after the "resync" correct? I guess that the easiest way to do this would be to have 2 temporary names and always check if the first one exists and if it does, then you create a temp index with the second temp name else with the first temp name. – tedi Dec 21 '17 at 08:50
  • 1
    And delete a temp index, accordingly while creating another temp index. – Praneeth Dec 21 '17 at 09:51
1

it is fully convertible, in your beat.yml configuration on Elastic output you add:

setup.ilm.enabled: false
output.elasticsearch.index: "customname -% {[agent.version]} -% {+ yyyy.MM.dd}"
setup.template.name: "customname"
setup.template.pattern: "customname- *"

Screenshot

janw
  • 8,758
  • 11
  • 40
  • 62
Khai Pham
  • 9
  • 2
  • Their is no mention of Beats in the first post. Why do you answer with beat when the issue is scoped only into Elasticsearch ? – Jaycreation Oct 29 '20 at 08:47