"EOF occurred while idle" when using network directive in syslog-ng

Question

I want to forward logs from a local machine to a distant server. Thanks to an SSH tunnel, the distant server is reachable through localhost:5514. I wrote the following conf:

# Filters
filter f_local0 { facility(local0); };
filter f_worker { program("^worker."); };

# Destinations
destination d_log01_local { file("/home/myuser/logs/worker/$YEAR$MONTH$DAY/$PROGRAM.$YEAR$MONTH$DAY" template("$HOST $MSG\n") create_dirs(yes) dir_group("adm") group("adm") dir_perm(0770) perm(0644) ); };
destination d_log01_remote { network("localhost" transport(tcp) port(5514) persist-name("remote_worker") disk-buffer( reliable(yes) disk-buf-size(50000000) )); };

# Logs
log { source(s_src); filter(f_local0); filter(f_worker); destination(d_log01_local); };
log { source(s_src); filter(f_local0); filter(f_worker); destination(d_log01_remote); };

The local destination works perfectly but the remote one does not: I do not receive any logs on the server located at localhost:5514.

When I tail -f /var/log/syslog, syslog logs these lines:

Oct 22 11:25:05 esad-10076 syslog-ng[17319]: Syslog connection established; fd='35', server='AF_INET(127.0.0.1:5514)', local='AF_INET(0.0.0.0:0)'
Oct 22 11:25:05 esad-10076 syslog-ng[17319]: EOF occurred while idle; fd='35'
Oct 22 11:25:05 esad-10076 syslog-ng[17319]: Syslog connection broken; fd='35', server='AF_INET(127.0.0.1:5514)', time_reopen='60'

I don't understand the error nor do I know how to fix it.

I should specify that before writing this conf, I had another conf that used syslog instead of network and this conf works perfectly. I just need to use network now to use the power of reliable in order to not lose any logs in case of network outage.

# Filters
filter f_local0 { facility(local0); };
filter f_worker { program("^worker."); };

# Destinations
destination d_log01_local { file("/home/myuser/logs/worker/$YEAR$MONTH$DAY/$PROGRAM.$YEAR$MONTH$DAY" template("$HOST $MSG\n") create_dirs(yes) dir_group("adm") group("adm") dir_perm(0770) perm(0644) ); };
destination d_log01_remote { syslog("localhost" transport(tcp) port(5514) persist-name("remote_worker")); };

# Logs
log { source(s_src); filter(f_local0); filter(f_worker); destination(d_log01_local); };
log { source(s_src); filter(f_local0); filter(f_worker); destination(d_log01_remote); };

score 1 · Accepted Answer · edited Oct 07 '21 at 06:58

The problem was occuring at the other end: the receiver side. There are 2 different syslog protocols in the wild: the rfc3164 one (the old one) and the rfc5424 one (the new one). See this blog post for more info.

I was receiving logs on my server with the old protocol and the messages did not match the logs sent by the sending machine using the newer protocol (hence the "Header" problem).

My conf was the following:

source mysource {
        syslog(
               ip(0.0.0.0) 
               transport(udp) 
               port(514) 
               max-connections(80) 
               keep-hostname(yes) 
        );
        syslog(
               ip(0.0.0.0) 
               transport(tcp) 
               port(514) 
               max-connections(80) 
               log_iw_size(8000)
               keep-hostname(yes) 
        );
};

...

When I tail -f /var/log/syslog, I get the following error:

Oct 29 16:37:28 servername syslog-ng[718]: Syslog connection accepted; fd='83', client='AF_INET(127.0.0.1:40034)', local='AF_INET(0.0.0.0:514)'
Oct 29 16:37:28 servername syslog-ng[718]: Invalid frame header; header=''
Oct 29 16:37:28 servername syslog-ng[718]: Syslog connection closed; fd='83', client='AF_INET(127.0.0.1:40034)', local='AF_INET(0.0.0.0:514)'

Writing the following line in place of the two syslog directives fixes the problem:

source mysource {
    network(transport(tcp) port(514) max-connections(80));
}

"EOF occurred while idle" when using network directive in syslog-ng

1 Answers1