"Message":"Your request: '/_cluster/allocation/reroute' is not allowed."}

Question

Am working on AWS cluster re-assigning the replica shard in an AWS ES cluster of node 1. Where i want both primary and replica shards to be reside. But when a run the below command am getting an error

curl  -XPOST 'http://AWS-CLuster-name.com/_cluster/allocation/reroute' -d '{ "commands": [{"allocate_replica":{"index": ".kibana", "shard": 0, node: "nodename" "primary": false }'

When run the above command am getting below error.

{"Message":"Your request: '/_cluster/allocation/reroute' is not allowed."}

Does this mean manual shard allocation is not allowed in AWS? Could anyone please clarify.

Thanks

I really wish AWS stops messing with developers/admins. one would hope there's an override for specific scenarios where invoking such APIs is the only option left to try — asgs, Jul 26 '22 at 16:22

Rodney Gitzel · Answer 1 · 2017-03-17T00:30:49.947

Correct, AWS does not allow you to control shard allocation directly, at least not in the usual way.

I'm hoping it can be done in some other AWS-specific way, as I've got a yellow cluster that I'd really rather be green. :(

[Followup: it seems the Elasticsearch domain eventually realizes it can fix this itself. In our case, because we were getting short on free space, replicas were being left unassigned. We freed up space, and maybe 30 minutes later the shards started replicating all on their own.]

score 0 · Accepted Answer · answered Mar 07 '17 at 18:27

0

This is not allowed in elasticsearch, since it would violate the concept as replicas for failover or distributed at all. https://www.elastic.co/guide/en/elasticsearch/guide/master/replica-shards.html

Having the same primary and replica shard on the same node is useless and simpy not allowed. Has nothing to do with AWS.

answered Mar 07 '17 at 18:27

cinhtau

1,002
8
16

11

This answer is only partially correct, and partially wrong. You can't re-route two shards onto the same node, true, but that's not the error the OP is seeing. AWS simply doesn't allow re-routing, valid or otherwise. – Rodney Gitzel Mar 16 '17 at 22:41

score 0 · Answer 3 · answered May 17 '21 at 11:39

Elasticsearch cluster status becomes yellow or red due to multiple reasons. Sometimes it could so happen that, even after a considerable amount of time, the Elasticsearch domain doesn't fix the issue with the cluster status. This happens particularly when the maximum number of retries for shard allocation exceed the limit (5 retries).

In this scenario, as per the documentation, a manual shard allocation can be triggered by disabling and re-enabling the replica shard on the index.

You can figure out which index's shard allocation had failed by running the following

curl -XGET 'MyES_Endpoint/_cat/shards?h=index,shard,prirep,state,unassigned.reason' | grep UNASSIGNED

Once you figure out which index's shard is unassigned, you can disable the replica shards for the index with the below command

curl -XPUT 'MyES_Endpoint/<index>/_settings' -H 'Content-Type: application/json' -d'
{
  "index" : {
    "number_of_replicas" : 0
  }
}'

You can re-enable the replica shards by setting it back to the appropriate number.

Caveat

Handling shard allocation manually on clusters that run heavy workloads is not suggested because, when we remove the replica shards for an index, we are left with just the primary shard and all the incoming load is then to be handled by just that shard alone. Any node failure then will result in the status of the cluster going to Red.

Take a look at the documentation to understand further on the different reasons for this and how they could be dealt with.

"Message":"Your request: '/_cluster/allocation/reroute' is not allowed."}

3 Answers3

Caveat