NoShardAvailableException after starting the Elasticsearch.bat

Question

I have started elasticsearch.bat and I completed first indexing using Nest

ElasticClient.Index query.

Then I made my first query using

var results = ElasticClient.Search<Product>(body =>
     body.Query(query =>
     query.QueryString(qs => qs.Query(key))));

This is all I have done. Later I restarted elasticsearch console using elasticsearch.bat and now it keeps giving me error message NoShardAvailableException. I deleted and redownloaded a new elasticsearch.bat and i keep getting same error. How can I resolve it?

I am using 1.7.1 version and btw I installed Marvel plugin also.

Do you have the elasticsearch log? `log/{your_cluster_name}.log`. What happens if you call `curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'` (or use the browser instead of `curl`? — Danielson, Aug 14 '15 at 13:50
@Danielson it will show me following information. { "cluster_name" : "elasticsearch", "status" : "red", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, "active_primary_shards" : 8, "active_shards" : 16, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 8, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0 } why is that status red, any idea? — Emil, Aug 19 '15 at 15:11
@Danielson but this is now after I restarted the elasticsearch.bat after some days. so it looks like now everything is restarted and running. but I would like to know if there is a way to restart all sharks while bat is running without closing and reopening. — Emil, Aug 19 '15 at 15:14
Your status is red, meaning some shards are unassigned or lack data, due probably to incorrect shutdown,,,,fix that first... it needs to say status is green... also explains why you got your exception, your shards are broken, 8 unassigned shards — Danielson, Aug 19 '15 at 20:07
@Danielson i will search how to fix this but in mean time if you know, can you give me a clue how to get this solved. thanks — Emil, Aug 20 '15 at 07:29

score 1 · Accepted Answer · answered Aug 20 '15 at 07:59

1

Your problem is not related with a version, so updating will not resolve the issue. The issue is that shards cannot be assigned to nodes. As shown by your call, see "status": "red" and "unassigned_shards": 8:

{
    "cluster_name": "elasticsearch",
    "status": "red",
    "timed_out": false,
    "number_of_nodes": 2,
    "number_of_data_nodes": 2,
    "active_primary_shards": 8,
    "active_shards": 16,
    "relocating_shards": 0,
    "initializing_shards": 0,
    "unassigned_shards": 8,
    "delayed_unassigned_shards": 0,
    "number_of_pending_tasks": 0,
    "number_of_in_flight_fetch": 0
}

First off, you can try reassigning the unassigned_shards, using (see es for more on this):

curl -XPOST 'localhost:9200/_cluster/reroute' -d '{"commands": [
    {"allocate": {
        "index": "{your_index_name}", 
        "shard": 3, 
        "node": "{your_assigning_node_ide}", 
        "allow_primary": true }
    }]
}'

Which shards are unassigned? To see this, use:

curl -XGET http://localhost:9200/_cat/shards | grep UNASSIGNED | awk '{print $0}'

When you know which shards create the problem, you can start by trying to recover the indices, using (indices recovery:

curl -XGET http://localhost:9200/index1,index2/_recovery

I find the grep UNASSIGNED statement particularly useful if, a couple out of a lot, are unassigned. Sometimes it is just easier (of course depending on the ease of refilling you indices), to delete and refill you index, in that case (delete indices) :

curl -XDELETE 'http://localhost:9200/concept_cv,concept_pl,concept_pt/'

Then reinsert your data.

This issue most probably was due to incorrect shutdown from your cluster, possibly also OOM exceptions. For more information on status : red:

https://t37.net/how-to-fix-your-elasticsearch-cluster-stuck-in-initializing-shards-mode.html
http://elasticsearch-users.115913.n3.nabble.com/how-to-resolve-elasticsearch-status-red-td4020369.html

answered Aug 20 '15 at 07:59

Danielson

2,605
2
28
51

Sorry if it is a silly question but how to run this curl command on windows. Cmd is recognizing it i believe. I am not on Linux. – Emil Aug 20 '15 at 08:57
1

When you type `curl` then enter, do you get `curl: try 'curl --help' or 'curl --manual' for more information`? Or an unrecognized command? For the former, you should use `"` instead of `'` (double quote instead of single). For the latter, download http://curl.haxx.se/download.html... Note, for `get` requests, you can also just call it in the web browser – Danielson Aug 20 '15 at 09:05
ok I figured out using curl command but I have no index anymore. I deleted created indexes before. But it still shows red. are those shards related to indixes. If not, how can I reset everything? I want to start over like a factory settings. is it possible? – Emil Aug 20 '15 at 09:18
I have deleted "data" folder and restarted the ES.bat but it still shows me red. This supposed to restart everything, not? in the path data\elasticsearch\nodes\0\indices I have 3 folders called kibana, .marvel-2015.08.14, ..marvel-kibana. I guess those comes from the plugin which I installed. Could this be the problem? – Emil Aug 20 '15 at 09:29
1

I don't have plugins installed, but, to start over, you should delete `data\*`, I'm guessing those plugins also have state information on the indices. Those you've already deleted. I think I read, somewhere, the libraries for the plugins are `installed` in the `lib/` dir. So, I think deleting the other directories in the `data` directory will fix your problem. If the plugins stop working (highly unlikely), use https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-plugins.html to reinstall the plugins – Danielson Aug 20 '15 at 09:38
it was the problem indeed. I dont know why this plugin added 3 indexes on my data folder but I removed each with curl command. Simply deleting data doesnt help because when I start ES again, they come back. Now finally status is green and "unassigned_shards" : 0 – Emil Aug 20 '15 at 09:41
I read somewhere that if a coworker of mine runs ES on same port as default http://localhost:9200/ in the same LAN, it will be considered as 2nd cluster and this might affect the status and unassigned_shards even my one doesnt have any unassigned_shards but if he has. Is that correct? How does this work? – Emil Aug 20 '15 at 09:44
1

Note, I'm just a user. But I'd guess not. But that would be dependent on the settings in the `elasticsearch.yml`, especially in the `Discovery` section. But , if, what your coworker is saying, has happened, you must see a trace of it in your `log`, like this: `[INFO ][cluster.service ] [YOUR_NODE] added {[HIS_NODE_NAME][{some_hash}][{HIS_COMPUTER_NAME}][inet[/{HIS_IP_ADDRESS}:9301]]{max_local_storage_nodes=2, mas..........` – Danielson Aug 20 '15 at 09:50
1

indeed it was. it was showing me 2 nodes although i had only 1 instance running but we had another instance on the server for testing. when I stopped it, instances were reduced to 1. just FYI :) – Emil Aug 20 '15 at 15:36

NoShardAvailableException after starting the Elasticsearch.bat

1 Answers1