0

I'm trying to figure out possible solution to the following problem. In my current implementation I'm relying on so called filtered aliases where I can create a view of the same index for each user e.g.:

POST /aliases { "actions": "add": { "index": "events", "alias": "events_123", "filter": { "match": { "user_id": "123" } } } }

All my write/read api calls are using aliases instead of the index name. Unfortunately, I cannot use filtered aliases anymore due to the recent changes related to introduction of index per day pattern managed by an external component. Is there a simple (or not so simple) way to provide my user_id filter as a parameter to a rest api call. For example, if I had a POST /events*/_search {...} request, would it be possible to add a filter without the need to modify the search query itself?

rilush
  • 1
  • 1

1 Answers1

2

You can keep using filtered aliases — the index can also include a wildcard. Not sure what your index structure looks like, but something like this will work:

POST /aliases
{
  "actions": {
    "add": {
      "index": "events*",
      "alias": "events_123",
      "filter": {
        "match": {
          "user_id": "123"
        }
      }
    }
  }
}

If you keep adding indices over time, the alias needs to be applied through an index template (documentation) — this is for example how the Beats do it:

PUT _template/events
{
  "index_patterns": ["events_*"],
  "aliases": {
    "events_123": {
      "filter": {
        "match": {
          "user_id": "123"
        }
      }
    }
  }
}

PS: You can have a list of aliases, but I only added a single one to keep it simple.

xeraa
  • 10,456
  • 3
  • 33
  • 66
  • The approach you describe is good for situations where the list of indexes is fixed. The glob pattern would ensure all indexes are aliased. In my scenario new index is added every day (rollover done by ISM job in aws). This requires updating the existing aliases with the newly created index. The problem here is that I don't control the rollover mechanism. That's why I was looking for an alternative way to provide user id filter by using other means than alias. – rilush Sep 14 '20 at 13:50
  • The elasticsearch documentation says the opposite (also verified by a simple test in Kibana) - https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-aliases.html "POST /_aliases { "actions" : [ { "add" : { "index" : "test*", "alias" : "all_test_indices" } } ] } In this case, the alias is a point-in-time alias that will group all current indices that match, it will not automatically update as new indices that match this pattern are added/removed." – rilush Sep 15 '20 at 19:53
  • 1
    Sorry, that was confusing / misleading: If you keep adding indices, you need to apply the alias through a template (that's what the Beats are doing for example). I've updated the answer with that and removed the comment. – xeraa Sep 18 '20 at 03:02
  • Thank you, xeraa:-) That gave me a very helpful lead on how to implement it for my scenario. – rilush Sep 19 '20 at 20:53