3

Are there any products (preferably open source) that can load balance UDP packets transparently?

We need to scale our syslog servers and require the original source IP be kept intact.

Alternatively are there any other methods for scaling syslog services?

AndyC
  • 233
  • 4
  • 14
  • 1
    Off the cuff, I would think static horizontal sharding would be the better way to scale syslog traffic because with load balancing, you are either going to have logs originating a one host landing on different destination hosts (a mess to reconstruct) or if your load balancer is "sticky" to avoid this, then you're going to have horizontal sharding anyway but it's going to be random instead of organized and systematic. How would you visualize this load balancing working? Would it be sticky? Would it do something with regard to failover? – Michael - sqlbot Oct 11 '13 at 03:34
  • A better and simpler question might be... do your systems not support syslog over TCP? – Michael - sqlbot Oct 11 '13 at 03:35
  • @Michael-sqlbot It's actually customer syslog traffic so we have less control than normal, and they have a huge estate so there would be large resistance to change. This is why I'm thinking of scaling options from our end. Saying that, they might however be open to the benefits of TCP (reliability), but they also have a lot of legacy kit which may not be so easy to change. Regarding 'sticky' streams - yes we'd ideally do that on source IP to ensure all packets from one host are together, but that may not play well with failover (which we'd also need). It's a tough combination. – AndyC Oct 11 '13 at 07:33

2 Answers2

1

You can use syslog-ng to do this. To keep the original source ip/name, you can set keep_hostname(yes) if it is being passed from one syslog-ng server to another. If not, you can compile in the option for spoof_source.

More information on relaying can be found here

Clayton Dukes
  • 444
  • 2
  • 9
  • Yes you are correct, but unfortunately that doesn't solve the load balancing issue - it just passes on the load to another syslog server. – AndyC Oct 05 '13 at 18:11
0

You could configure some form of DNS load balancing (having multiple syslog servers sat behind the same A/CNAME record) as connections don't require any form of stickyness. You'd want to tune the TTL in this scenario as a value too low will result in pauses in the logs pending DNS lookups, and a value too high will not rotate any of the load between servers.

Matthew Steeples
  • 1,303
  • 1
  • 10
  • 17
  • Stickyness would actually be preferred (although not a big issue if they don't). The issue I see here is, in this case the source packets are from customer servers and I don't believe all machines have access to DNS (very tightly locked down) – AndyC Oct 11 '13 at 07:37