1

I am experiencing some sort of strange behavior of our production graylog system. Every time the graylog server cycles the deflector (we have a limit of 2.5 million documents per index), it stops indexing messages and raises a lot of indexing errors.

From the logs i get the info that it could not create the deflector index, but nothing more, no details. Graylog then continues and tries to write the incoming messages into the old index hence the indexing errors.

We are running graylog 1.0.2 and elasticsearch 1.4.5 under CentOS 6.6 x86_64 using Kernel 3.10.77-1.el6.elrepo.x86_64

Any help would be greatly appreciated as i spent around two days debugging it, maybe i need to rise a bug ticket.

Thanks for your help!

Sebastian

Sebastian Jaekel
  • 101
  • 1
  • 6

1 Answers1

1

For all who might encounter the same problem, we finally solved it! First we had a retention policy of 2.500.000 documents / index and a maximum of 50 indices. That seemed too much indices so we lowered it by changing the retention to 10.000.000 documents / index with a maximum of 15 indices, but that made it worse. Finally we lowered the number of documents per index and now have a retention policy of 1.000.000 docs / index and a maximum of 150 indices. That setting works fine and we dont loose any messages any more have no problems when cycling the deflector.

Sebastian Jaekel
  • 101
  • 1
  • 6