7

Rsyslog on a RHEL 6 server is receiving messages locally on port 514 UDP. These messages are sometimes much larger than normal syslog message sizes. I am seeing rsyslog handle all of the messages just fine, it writes to the local files without issue. However, when I add a remote host to rsyslog config these same large messages are truncated at around 2048 characters.

I am running rsyslogd version: 5.8.10

rsyslogd 5.8.10, compiled with:
  FEATURE_REGEXP:               Yes
  FEATURE_LARGEFILE:            No
  GSSAPI Kerberos 5 support:        Yes
  FEATURE_DEBUG (debug build, slow code):   No
  32bit Atomic operations supported:    Yes
  64bit Atomic operations supported:    Yes
  Runtime Instrumentation (slow code):  No

The only changes I have made to rsyslog conf are these two things below:

This is at the very top of my rsyslog conf file:

 $MaxMessageSize 64k

This is at the very bottom of my rsyslog conf file:

$template RemoteHost,"<%%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%
*.* @my-rsyslog-central-logger:514; RemoteHost

Does anyone know why rsyslog would be truncating logs over UDP to the remote host, but is able to handle the logs when writing them to the local files?

**Note I did double check it wasn't happening at the remote rsyslog host, by using netcat to listen on a local port that rsyslog was forwarding to over UDP.

** Note I did try TCP and it did not truncate the large forwarded message, so now the question is why does UDP truncate. (I'm assuming the answer may be related to the properties of UDP, but I still want to know and maybe fix it if possible for UDP forwarded messages)

nictrix
  • 173
  • 1
  • 7

1 Answers1

4

UDP doesn't have sequence numbers, there would be no way to combine messages coherently (if they arrive out of order)

Syslog UDP Transport - https://www.rfc-editor.org/rfc/rfc5426

3.1. One Message Per Datagram

Each syslog UDP datagram MUST contain only one syslog message, which MAY be complete or truncated. The message MUST be formatted and truncated according to RFC 5424 [2]. Additional data MUST NOT be present in the datagram payload.

Community
  • 1
Jeff Warnica
  • 474
  • 2
  • 8
  • Thanks, this is what I figured it was, but wanted to see if there was a way to not fragment the packets. I think the better answer is the section here: http://tools.ietf.org/html/rfc5426#section-3.2 - this talks about actual size of the packet that will successfully be sent based on recommendations from the RFC. So in other words stop using UDP for syslog messages sent over the wire. (However, I believe there is still a case to use UDP locally on a server) – nictrix Feb 20 '14 at 07:14
  • If its the local server, just use sockets (/dev/log) – Jeff Warnica Feb 20 '14 at 14:12
  • Ahh yes good idea – nictrix Feb 20 '14 at 18:06
  • So how did you end up sending large syslog messages over-the-wire? I tried TCP and was also unsuccessful. – Chris F Mar 09 '17 at 21:27
  • Did you able to do it. I am trying this without luck too. – Ramadheer Singh Feb 08 '18 at 20:48
  • The same RFC that @JeffWarnica links to says that the maximum message size is limited by the max size of a UDP datagram - 64k minus the IP and UDP overhead. These datagrams can be fragmented at the IP layer, and in most cases, a 2K UDP will already have been fragmented - standard ethernet is 1500 bytes max. RFC also says that "the ability to receive larger messages is encouraged". So the RFC does not say this shouldn't be done. Whether it's a good idea is another question; if you lose a fragment of one of those messages the whole thing is gone. But that applies to UDP syslog in general. – Dan Pritts Mar 13 '19 at 17:29