0

How many logs/second can usually be handled by syslog servers? By syslog servers, I am referring to rsyslog, syslog-ng, splunk etc. The intent of the question is to find out at what logs/second rate the OS (Linux kernel >=3.0) becomes bottle-neck.

So far, I am able to forward about 10,000 logs/sec using UDP. If I increase the rate, logs start getting dropped. I don't know what is causing the drops, is it the application (it is a custom build log server) or is it the OS?

Can you please share your experience with regards to how many logs/second you are able to transfer without seeing any drops?

user837208
  • 105
  • 3

2 Answers2

1

This question can not be answered simply in its current form. There is no general rule for the upper limit of logs transfer rate. This can be varied based on several factors like your network configuration/reliability (between sender and recipient), machines specs, and whether they are loaded or not, etc...

UDP is not reliable protocol. You can try using TCP as it is connection-oriented protocol can be used for reliable data transmission. It imposes a little overhead more than UDP, but it may worth trying.

Khaled
  • 36,533
  • 8
  • 72
  • 99
1

You can increase the kernel UDP buffers, but this will not solve the problem. If you want to forward this amount of messages reliably, you should forget UDP syslog and use TCP instead. We have measured over 500k events per second with nxlog when transferring syslog messages over TCP.

b0ti
  • 986
  • 1
  • 6
  • 13