0

When I launch elasticsearch (with logstash), it consumes 100% of the CPU.

If I check the status of the cluster I can see 

{
    "cluster_name": "elasticsearch",
    "status": "red",
    "timed_out": false,
    "number_of_nodes": 2,
    "number_of_data_nodes": 1,
    "active_primary_shards": 301,
    "active_shards": 301,
    "relocating_shards": 0,
    "initializing_shards": 4
    "unassigned_shards": 300
}

Why do I have so many active shards? Where can I configure it?

I tried to configure the threadpool, the cahe and the memory (https://gist.github.com/reyjrar/4364063)

I tried 

curl -XPUT 'localhost:9200/_settings' -d ' {"index.routing.allocation.disable_allocation": false}'

My kibana is very slow and I cannot use it any more.

I have 30gb of indexes for 45 days of logs.

Philippe Benedetti
  • 1
  • 1

1 Answers1

0

The number of shards is normal given the amount of days you have. By default logstash will create indices with 5 primary shards.

Your ES cluster health is Red. Are all your nodes up and running? It says only 2 data nodes are up, is that expected?

Are you running the embedded ES inside logstash? If so , that's only recommended for only development/testing use and you should be using a standalone cluster

jlintz
  • 463
  • 3
  • 8
  • I am running ES on a single server. And I have two nodes with the same ip. Do you have some tips to configure ES to run on a small server. I will try to use a standalone cluster. One serveur, one node, one node data, no replication. Should i reduce the number of shards to 3? Does it save memory and CPU? (I will also reduce the number of days of history and test the limit by increasing day-by-day) – Philippe Benedetti Sep 29 '14 at 02:34