I'm a SolrCloud newbie, my setup is 3 shards, 3 replicas, external Zookeeper
Today I found shard3 down, replica3 had taken over as leader, so indexing was occurring to replica3 not shard3. I stopped Tomcat/SOLR in reverse order (R3,R2,R1,S3,S2,S1) and restarted in forward order (S1,S2,S3,R1,R2,R3). I did not delete any tlog or replication.properties files. The cloud graph shows all hosts with their correct assignments. As I understand it these assignments are set in Zookeeper on the first startup.
My question is how does the data that was indexed to replica3 get back to the revived shard3?
And surprisingly shard3 = 87G while replica3 = 80G.
Confused!