4

SystemD journal limits logged messages according to RateLimitIntervalSec and RateLimitBurst global configuration options. There is a possibility to change those values per specific unit/service using LogRateLimitIntervalSec and LogRateLimitBurst directives. Unfortunately I can't make these per-service options work. I hope that you will help me find what I'm doing wrong or confirm that there is something wrong with systemd.

My environment:

# uname -srvmpio
Linux 4.18.0-193.14.2.el8_2.x86_64 #1 SMP Sun Jul 26 03:54:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/centos-release
CentOS Linux release 8.2.2004 (Core)

# systemctl --version
systemd 239
+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy

# rsyslogd -v
rsyslogd  8.1911.0-3.el8 (aka 2019.11)

Make sure global journald limits are left as default in /etc/systemd/journald.conf:

[Journal]
#RateLimitIntervalSec=30s
#RateLimitBurst=10000

and restart if any change was necessary

systemctl restart systemd-journald

Prepare the test script /root/test.sh:

#!/bin/bash
for i in {1..1000000}
do
   echo "Hello world $i"
done

Prepare the test service unit /root/test.service:

[Unit]
Description=Test limit service

[Service]
Type=simple
WorkingDirectory=/root
ExecStart=/root/test.sh
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=test
LogRateLimitIntervalSec=0
LogRateLimitBurst=0

[Install]
WantedBy=multi-user.target

Make sure rsyslog rate limit is off by changing /etc/rsyslog.conf:

module(load="imjournal"
       StateFile="imjournal.state"
       Ratelimit.Interval="0")

Configure test service log file location by creating /etc/rsyslog.d/test.conf:

if ($programname == 'test') then {
    action(
        type="omfile"
        File="/root/test.log"
    )
    stop
}

Restart rsyslog:

systemctl restart rsyslog

Install the test service:

# systemctl enable /root/test.service

Verify test service rate limit settings (LogRateLimitIntervalSec is internally stored as LogRateLimitIntervalUSec):

# systemctl show --property LogRateLimitIntervalUSec test.service --no-pager
LogRateLimitIntervalUSec=0

# systemctl show --property LogRateLimitBurst test.service --no-pager
LogRateLimitBurst=0

Start the service:

systemctl start test.service

My result:

  • /root/test.log stops at Hello world 12500
  • journalctl -u test.service --no-pager stops at Hello world 12500
  • /var/log/messages reports (sometimes this message is not logged)
systemd-journald[304398]: Suppressed 982500 messages from test.service

I've also confirmed that changing the global journald settings to (and restarting systemd-journald):

[Journal]
RateLimitIntervalSec=0
RateLimitBurst=0

results in no rate limiting as expected.

I'm clueless at this point. Even tried to analyze the systemd source code but it's over my head. If anyone would at least recreate my steps and report the results that would be helpful in reporting this on systemd Github issue tracker.

Thanks.

Furgas
  • 163
  • 1
  • 5

1 Answers1

2

The RateLimitIntervalSec=, RateLimitBurst= section in man journald.conf says:

To turn off any kind of rate limiting, set either value to 0.

However you cannot find this in the LogRateLimitIntervalSec=, LogRateLimitBurst= section in man systemd.exec. With your test.service you can easy verify that setting this to something other than 0 does work as expected.

ctx
  • 136
  • 3