0

Can I use syslog-ng to mirror all log files onto a remote server without specifying every file? Both are running syslog-ng OSE version 3.5.2. The file("/var/log/*") setting seems promising, but it does not appear to recurse, and reconstituting the file names on the other side of a syslog() source seems daunting.

Should I give up on this and do some sort of scripting or use rsync or something else entirely?

The source is essentially an embedded Linux computer with limited flash storage and the others is pretty hefty. They are connected to the same fast switch at 1GB. It is not OK for the embedded Linux to crash without having already sent most of its logs over, so some sort of continuous update would seem appropriate.

tpc1095
  • 5
  • 3

2 Answers2

0

You may export simply the logs by doing the following configuration :

destination d_syslog_central { udp( "SERVER"); };
log { source(s_all); destination(d_syslog_central); };

where SERVER is the IP of the log server. All the services using the syslog will be exported. You may configure the missing one to use them (like Apache...)

Dom
  • 6,743
  • 1
  • 20
  • 24
0

it depends.

If you have applications that log directly into files under /var/log/, then you have the following possibilities

  • Reconfigure the applications to log into syslog instead of files, then syslog-ng can read the incoming messages and forward them to your logserver. If your applications support logging into syslog, then this is the recommended way to go.
  • The commercial version of syslog-ng supports wildcard file sources (/var/log/* and the like). This feature is currently not available in syslog-ng Open Source Edition
  • As a workaround, you can use the confgen plugin of syslog-ng (I'm not sure it is available in version 3.5, but it surely is in 3.6 and newer). With the confgen plugin, you can run a script that generates a section into the syslog-ng configuration file: you can use it to list the files in /var/log as file sources (note that this solution will not add new files from /var/log automatically, you'll have to periodically restart syslog-ng for that).

BTW, if possible, you might want to update your syslog-ng to a newer version, 3.5 is rather old and somewhat buggy. The recent 3.9 version supports diskbuffers and other nice features.

HTH

Robert Fekete
  • 552
  • 1
  • 3
  • 6
  • The challenge is to not simply collect the data and forward it to a port, but to find a way to keep the original file names and directory structure. If I have /var/log/anaconda and under that, I have nine (9) files then this is awkward. I found a trick where if I force a syslog "app_name" (${PROGRAM}" then I can use that in a file() destination to create a file name. This means I need a source() per log source (they seem to be all different) and generating the config won't universally succeed.. – tpc1095 Feb 09 '17 at 20:17
  • You can use rewrite rules to create custom fields and write the path/filename into this field. If you use the syslog() driver, then it is easy to use these fields on the server as well. See this blogpost about forwarding filenames with syslog-ng: https://czanik.blogs.balabit.com/2015/03/using-rfc5424-syslog-to-forward-file-names/ – Robert Fekete Feb 10 '17 at 11:33