How to create rolling index with date as the index name?

Question

My elastic search index will ingest thousands of documents per second. The service which puts documents in the index doesn't creates a new index, instead it just gets current data in nodejs and indexes docs in "log-YYYY.MM.DD". So, we know index is created automatically if not present.

Now, my question is can this practice of creating index and putting docs at the same time cause performance issues or failures given that index wil be ingesting thousands of docs per second ?

If the answer to above question is yes, how can I create a rolling index whitch date as the index name? Say today is 5 May, 2021, so I want automatic creation of index for 6 May, 2021 in the format log-2021.05.06.

If target index for indexing request doesn't exist then index will be created automatically https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation — Ajinkya, May 05 '21 at 23:52
@Ajinkya I'm aware of this but I wanna ensure that does this automatic creation of index comes with performance drops given it'll be ingesting thousands of docs simultanesously? — Broken Phone, May 06 '21 at 05:29

score 0 · Answer 1 · answered May 06 '21 at 13:04

For your first question, may be this can help how many indices?

For the second question I think you can use index-alias like

PUT /_index_template/logdate_template
{
   "index_patterns": [
        "log*"
    ],
    "priority": 1,
    "template": {
        "aliases": {
            "log":{}
        },
    "mappings": {
    //your mappings
    }
  }
  }
}

As here index_pattern is "log*", in your application code you can have a job, which creates index everyday by generating date in required format and calling

PUT log-YYYY.MM.DD

The advantage of index-alias is: You can access all those indices with "log" only.

How to create rolling index with date as the index name?

1 Answers1