1

I have an ELK Stack set up and accepting log data from 2 of my applications and everything is working ok. Its been running for 25 days and I have nearly 4GB of Data/Documents on a 25GB server.

enter image description here

My question

I have 8 applications in total that I would like to hook up to my ELK Stack.

Is the one cluster OK for this, or do I need to add more clusters? say a cluster for each applications data? If so how do I do that without having to re-index my data?

Why does cluster health say "yellow (244 of 488)?

Should I index each application to index on it own index rather than the default "logstash-{todays-date}". Like my-app-1-{todays-date}, my-app-2-{todays-date} etc..?

your help is greatly appreciated

G

Gman
  • 2,433
  • 3
  • 26
  • 36

2 Answers2

1

Your cluster is yellow because your logstash-* indices are configured with 1 replica and you probably have a single node. 244 of 488 means that you have 488 shards in all your indices but only 244 are assigned on your single node and 244 remain to be assigned to new nodes. This is not a problem per se, but if your current node were to fail for some reason, you'd probably lose some data, whereas if you had 2+ nodes, the data would be replicated on other nodes, your cluster would be green (and you'd see 488 of 488) and you'd have a lower risk of losing data.

As for your second question, nothing prevents you from storing all the logs from your eight applications in the same daily logstash indices. You just need to make sure that your logstash configuration accounts for every different apps and adds one field with the application name (e.g. app: app1, app: app2, etc) to the indexed log events so that you can then distinguish within Kibana from which app each log event has been issued.

Val
  • 207,596
  • 13
  • 358
  • 360
  • thanks, I have a better understanding of what needs to be done. Hope this helps others and I have mark it as the answer. – Gman Nov 18 '15 at 15:52
  • Thanks Gman, don't hesitate to ask further questions if needed. – Val Nov 18 '15 at 15:59
1

I have only used Elasticsearch and no the complete ELK stack, but I can give some ideas and guess what is going on. 488 = 2 x 244 , so I guess there are un-assigned replica shards in the single-machine cluster. You can update this setting ad-hoc and set it to zero:

curl -XPUT 'localhost:9200/my_index/_settings' -d ' {"index" : {"number_of_replicas" : 0}}'

You should update logstash index template not to use replicas when you are running just a single machine. Also your shards seem to be only about 20 MB in size so I'd recommend each index to use just one shard instead of five, each shard consumes extra resources. Having multiple shards increases indexing speed but slows down queries, you should check if one is sufficient or not.

Index / application / day would speed-up querying if dashboards are mostly application-specific, and you can create a day-specific alias to-be used by cross-application queries.

NikoNyrh
  • 3,578
  • 2
  • 18
  • 32