Elastic TSDS uses wrong backing index after downsample

Question

I have created a TSDS in elastic with the following settings:

{
  "index": {
    "mode": "time_series",
    "number_of_shards": "2",
    "number_of_replicas": "1",
    "routing_path": [
      "name",
      "id"
    ]
  }
}

It uses a component template with the following settings:

{
  "index": {
    "lifecycle": {
      "name": "<>_time_series"
    },
    "look_ahead_time": "2d",
    "codec": "best_compression"
  }
}

This lifecycle policy stays in hot phase for 30 days and in the warm phase immediatly down samples the data to 7days interval.

After 30 days (yesterday), the TSDS downsampled into two indices:

downsample-6lvo-.ds-<>-2023.06.06-000001

and

.ds-metrics-<>-2023.07.06-000002

However, I can't write to the TSDS anymore. I get the following error if I post to the TSDS as I did before:

{
  "error": {
    "root_cause": [
      {
        "type": "cluster_block_exception",
        "reason": "index [downsample-6lvo-.ds-<>-2023.06.06-000001] blocked by: [FORBIDDEN/8/index write (api)];"
      }
    ],
    "type": "cluster_block_exception",
    "reason": "index [downsample-6lvo-.ds-<>-2023.06.06-000001] blocked by: [FORBIDDEN/8/index write (api)];"
  },
  "status": 403
}

If I write directly to the newly created index (.ds-metrics-<>-2023.07.06-000002), I get the following error:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "index request with op_type=create targeting backing indices is disallowed, target corresponding data stream [<>] instead"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "index request with op_type=create targeting backing indices is disallowed, target corresponding data stream [<>] instead"
  },
  "status": 400
}

It looks like the backing index is wrong somehow?

Here are the index settings for downsample-6lvo-.ds-<>-2023.06.06-000001:

{
  "index.query.default_field": [
    "*"
  ],
  "index.write.wait_for_active_shards": "1",
  "index.refresh_interval": "-1",
  "index.hidden": "true",
  "index.time_series.end_time": "2023-07-08T12:49:52.000Z",
  "index.time_series.start_time": "2023-06-04T12:44:55.000Z",
  "index.blocks.read_only_allow_delete": "false",
  "index.blocks.write": "true",
  "index.priority": "100",
  "index.number_of_replicas": "1",
  "index.routing_path": [
    "searchterm_name",
    "ean"
  ],
  "index.lifecycle.name": "<>_time_series",
  "index.lifecycle.indexing_complete": "true",
  "index.mode": "time_series",
  "index.downsample.source.name": ".ds-<>-2023.06.06-000001",
  "index.downsample.source.uuid": "<>",
  "index.downsample.status": "success",
  "index.routing.allocation.include._tier_preference": "data_warm,data_hot",
  "index.look_ahead_time": "2d"
}

And here are the index settings for .ds-metrics-<>-2023.07.06-000002:

{
  "index.query.default_field": [
    "*"
  ],
  "index.refresh_interval": "1s",
  "index.write.wait_for_active_shards": "1",
  "index.hidden": "true",
  "index.time_series.end_time": "2023-07-10T12:49:52.000Z",
  "index.time_series.start_time": "2023-07-08T12:49:52.000Z",
  "index.blocks.read_only_allow_delete": "false",
  "index.priority": "100",
  "index.number_of_replicas": "1",
  "index.routing_path": [
    "searchterm_name",
    "ean"
  ],
  "index.lifecycle.name": "<>_time_series",
  "index.mode": "time_series",
  "index.routing.allocation.include._tier_preference": "data_hot",
  "index.look_ahead_time": "2d"
}

Elastic TSDS uses wrong backing index after downsample

0 Answers0