rsyslog: action suspended, next retry is

Question

I'm trying to configure rsyslog to receive logs sent from other devices on port 3100 (my manager chose that port and I will get him to change it to 514 later), and save (append) those logs in local files. So I created /etc/rsyslog.d/remote.conf thus:

$umask 0000
template(name="DynFile" type="string" string="/var/log/remote/%$YEAR%-%$MONTH%-%$DAY%/%HOSTNAME%")
ruleset(name="RemoteMachine"){ action(type="omfile" dynaFile="DynFile" dirCreateMode="0755") }

# provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="3100")

# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="3100")

#Enable sending system logs over UDP to rsyslog server
*.* @rsyslog-ip-address:3100

#Enable sending system logs over TCP to rsyslog server
*.* @@rsyslog-ip-address:3100

#Set disk queue when rsyslog server will be down:
$ActionQueueFileName queue
$ActionQueueMaxDiskSpace 1g
$ActionQueueSaveOnShutdown on
$ActionQueueType LinkedList
$ActionResumeRetryCount -1

added debuggery to the systemd unit:

ExecStart=/usr/sbin/rsyslogd -n -d

then restarting rsyslog:

systemctl daemon-reload
systemctl restart rsyslog

then reviewing the port bindings:

# ss -tulpn|grep 3100
udp    UNCONN     0      0         *:3100                  *:*                   users:(("rsyslogd",pid=11899,fd=5))
udp    UNCONN     0      0        :::3100                 :::*                   users:(("rsyslogd",pid=11899,fd=6))
tcp    LISTEN     0      25        *:3100                  *:*                   users:(("rsyslogd",pid=11899,fd=7))
tcp    LISTEN     0      25       :::3100                 :::*                   users:(("rsyslogd",pid=11899,fd=8))

so it looks like rsyslog is listening as per the configuration file; however, the last few lines of journalctl -e -u rsyslog are:

Sep 05 16:10:22 office-zabbix-proxy systemd[1]: Starting System Logging Service...
Sep 05 16:10:22 office-zabbix-proxy liblogging-stdlog[11899]:  [origin software="rsyslogd" swVersion="8.24.0" x-pid="11899" x-info="http://www.rsyslog.com"] start
Sep 05 16:10:22 office-zabbix-proxy systemd[1]: Started System Logging Service.
Sep 05 16:10:22 office-zabbix-proxy liblogging-stdlog[11899]: action 'action 1' suspended, next retry is Mon Sep  5 16:10:52 2022 [v8.24.0 try http://www.rsyslog.com/e/2007 ]
Sep 05 16:10:22 office-zabbix-proxy liblogging-stdlog[11899]: action 'action 2' suspended, next retry is Mon Sep  5 16:10:52 2022 [v8.24.0 try http://www.rsyslog.com/e/2007 ]

I presume these 'actions' correspond to the two module/input declarations in the configuration file, but I have no idea why they have been 'suspended', whether or not it matters, and what I should do about it.

However, there are no files under /var/log/remote/ (which exists, with 0777 permissions) so I presume something is stopping the configuration from doing what I want. The OS is Debian 11 and is not in SELinux mode. Help please?

This is all totally obvious to those that understand rsyslog configuration, but the lines starting `*.*` are in the so-called obsolete legacy system and are rewritten as `action()` lines by rsyslog upon startup, so that's from where the _action 1_ etc. logging arises. — Michael NGV, Sep 06 '22 at 23:30

LinuxRsa · Answer 1 · 2023-04-14T10:13:33.823

any advice on the below. I keep on getting this issue ( action 'action 18' suspended )

[root@log_col ~]# systemctl status rsyslog.service
● rsyslog.service - System Logging Service
   Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2023-04-14 08:09:58 UTC; 34min ago
     Docs: man:rsyslogd(8)
           http://www.rsyslog.com/doc/
 Main PID: 6023 (rsyslogd)
   CGroup: /system.slice/rsyslog.service
           └─6023 /usr/sbin/rsyslogd -n

Apr 14 08:09:58 log_col systemd[1]: Stopped System Logging Service.
Apr 14 08:09:58 log_col systemd[1]: Starting System Logging Service...
Apr 14 08:09:58 log_col rsyslogd[6023]:  [origin software="rsyslogd" swVersion="8.24.0-57.el7_9.3" x-pid="6023" x-info="http://www.rsyslog.com"] start
Apr 14 08:09:58 log_col systemd[1]: Started System Logging Service.
Apr 14 08:18:27 log_col rsyslogd[6023]: action 'action 18' suspended, next retry is Fri Apr 14 08:18:57 2023 [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2007 ]
Apr 14 08:20:35 log_col rsyslogd[6023]: action 'action 19' suspended, next retry is Fri Apr 14 08:21:05 2023 [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2007 ]
Apr 14 08:22:42 log_col rsyslogd[6023]: action 'action 20' suspended, next retry is Fri Apr 14 08:23:12 2023 [v8.24.0-57.el7_9.3 try http://www.rsyslog.com/e/2007 ]
[root@log_col ~]#


[root@log_col ~] ss -tulpn | grep 514
udp    UNCONN     0      0         *:514                   *:*                   users:(("rsyslogd",pid=2201,fd=4))
udp    UNCONN     0      0      [::]:514                [::]:*                   users:(("rsyslogd",pid=2201,fd=5))
tcp    LISTEN     0      25        *:514                   *:*                   users:(("rsyslogd",pid=2201,fd=6))
tcp    LISTEN     0      25     [::]:514                [::]:*                   users:(("rsyslogd",pid=2201,fd=7))
[root@log_col ~]
[root@log_col ~] sestatus
SELinux status:                 disabled
[root@log_col ~]


[root@log_col syslog]# grep -v "^#\|^$" /etc/rsyslog.conf
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514
$template RemoteLogs,"/var/log/syslog/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs
$FileOwner syslog
$FileGroup syslog
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging off
$IMJournalStateFile imjournal.state
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
*.* @@IP-ONE:514
*.* @@IP-TWO:514
*.* @@IP-THREE:514
[root@log_col syslog]#

rsyslog: action suspended, next retry is

1 Answers1