If there are three nodes, is it reasonable to run ten indexes with three shards? Or is it reasonable to have 10 indexes with two shards? (Replication 1)
Because it is a date-based index, i plan to create a snapshot when the capacity is exceeded.
Asked
Active
Viewed 1,505 times
0
-
1That all depends on the load and uptime you need to achieve. Less shards you have - less redundancy you have. so if you have 3 nodes, I would say 2 shards is reasonable number. But you should set up 1 node to be read only for example. Number of indexes - is not enough to make any decision. How big are indexes? 1Gb? 10Gb? 10Gb? 1Tb? that is the question. How powerful are your nodes? How many clients knock to elastic to get data? etc... – Alex Nov 13 '18 at 14:54
-
1Alex is right about the number of indices, But there is probaly a misunderstanding of sharding. Please be aware that sharding has nothing to do with realibility. Shrading is just about distributing your data into several lucene indices (shards). If done right, it can improve your performance. But loosing a node means loosing your shards on this node if there a no replicas available. So if you want realibility, you should have at least one replica. In this way, if a primary shard is lost, the cluster will be able to route the requests to the replica shard and still serve further requests. – ibexit Nov 13 '18 at 16:41
-
1please have also a look on this question I answered few days ago. This should help you to choose a good number of shards per index: https://stackoverflow.com/questions/53214628/elasticsearch-how-does-sharding-affect-indexing-performance/53216210#53216210 – ibexit Nov 13 '18 at 16:45
-
Thank you. @Alex Each shard is for logging for 40 Gb in a week. Data is input from a number of logstash similar to elasticsearch. – 김태우 Nov 14 '18 at 00:55
-
If you use a high-performance server computer, i will have fewer nodes, and i can use many low-performance servers. I think a high performance server is more efficient. – 김태우 Nov 14 '18 at 02:19