1

My requirement is to log all messages on the remote machine. In order to achieve the goal I have two identical versions of rsyslog (rsyslogd 8.1901.0 (aka 2019.01)) on both machines (server: 192.168.122.12 and client: 192.168.122.13).

Besides, if the remote machine is down I need to buffer all messages in order to send them later when it is online. The problem is, that this have a very strange behavior. When I shut down remote server, and log something the message is never sent, even when the remote machine is online again. Sometimes when I log something again some of the old (buffered) messages are sent (never all of them). As you may see I use TCP connection.

My configuration is:

Server: /etc/rsyslog.conf

module(load="imuxsock") # provides support for local system logging
module(load="imklog")   # provides kernel logging support
module(load="imtcp")
input(type="imtcp" port="514" ruleset="remote")
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$WorkDirectory /var/spool/rsyslog

$DebugLevel 2
$DebugFile /var/log/rsyslog-debug.log

template (name="DynFile" type="string" string="/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log")

template(name="CustomFileFormat" type="list") {
    property(name="timereported" dateFormat="rfc3339")
    constant(value=" ")
    property(name="hostname")
    constant(value=" ")
    property(name="syslogtag")
    property(name="msg" spifno1stsp="on" )
    property(name="msg" droplastlf="on" )
    constant(value="\n")
}

ruleset(name="remote") {
  action(type="omfile" dynaFile="DynFile" Template="CustomFileFormat")
  stop
}
$IncludeConfig /etc/rsyslog.d/*.conf
auth,authpriv.*                 /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
daemon.*                        -/var/log/daemon.log
kern.*                          -/var/log/kern.log
lpr.*                           -/var/log/lpr.log
mail.*                          -/var/log/mail.log
user.*                          -/var/log/user.log
mail.info                       -/var/log/mail.info
mail.warn                       -/var/log/mail.warn
mail.err                        /var/log/mail.err
*.=debug;\
        auth,authpriv.none;\
        news.none;mail.none     -/var/log/debug
*.=info;*.=notice;*.=warn;\
        auth,authpriv.none;\
        cron,daemon.none;\
        mail,news.none          -/var/log/messages
*.emerg                         :omusrmsg:*

Client: /etc/rsyslog.conf

$LocalHostName my.test.machine.corp.es

module(load="imuxsock") # provides support for local system logging
module(load="imklog")   # provides kernel logging support
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$WorkDirectory /var/spool/rsyslog

$DebugLevel 2
$DebugFile /var/log/rsyslog-debug.log

*.* action(
        type="omfwd"
        target="192.168.122.12"
        port="514"
        protocol="tcp"
        queue.type="linkedlist"
        queue.size="10000"
        queue.filename="fwd_msgs"
        action.resumeRetryCount="-1"
        queue.saveOnShutdown="on"
        action.resumeinterval="30"
)

$IncludeConfig /etc/rsyslog.d/*.conf
auth,authpriv.*                 /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
daemon.*                        -/var/log/daemon.log
kern.*                          -/var/log/kern.log
lpr.*                           -/var/log/lpr.log
mail.*                          -/var/log/mail.log
user.*                          -/var/log/user.log
mail.info                       -/var/log/mail.info
mail.warn                       -/var/log/mail.warn
mail.err                        /var/log/mail.err
*.=debug;\
        auth,authpriv.none;\
        news.none;mail.none     -/var/log/debug
*.=info;*.=notice;*.=warn;\
        auth,authpriv.none;\
        cron,daemon.none;\
        mail,news.none          -/var/log/messages
*.emerg                         :omusrmsg:*
Michael
  • 61
  • 1
  • 3
  • You might consider moving to [relp](https://www.rsyslog.com/doc/v8-stable/configuration/modules/imrelp.html) which claims to be more reliable than tcp. It is often in a separate package, eg rsyslog-relp. – meuh May 06 '20 at 15:01
  • Are you able to see how many messages the queue keeps? My understanding from your config is that you store the messages in a queue of size 10000, use a filename `fwd_msgs` to store them in and then retransmit the messages. If the queue fills up while the remote server is offline, then I'd assume you can only transmit what still remains in the queue so you could encounter some data loss. – The Stupid Engineer Dec 11 '20 at 11:21

1 Answers1

0

You can try setting up this clause:

$ActionSendTCPRebindInterval nbr- [available since 4.5.1] - instructs the TCP send action to close and re-open the connection to the remote host every nbr of messages sent. Zero, the default, means that no such processing is done. This directive is useful for use with load-balancers. Note that there is some performance overhead associated with it, so it is advisable to not too often “rebind” the connection (what “too often” actually means depends on your configuration, a rule of thumb is that it should be not be much more often than once per second).

You can try something like $ActionSendTCPRebindInterval 10000. Keep in mind that this causes some extra performance overhead if the reopen connection is too often.

PS. You can read this on the documentation.

Dilson Rainov
  • 136
  • 4