6

I am a newbie to Linux and rsyslog. I have used the logfiles for many years, but I have never had to set one up. At this point I have some Proof of Concept devices pointing to my Debain Linux server. I have the syslog messages coming in and being written to a single file: /var/log/prd/fwlog I am only concerned about 3 device types - switches, routers and firewalls. (all cisco) My rsyslog.conf is fairly simple, I have only modified the basic config, commented out the stuff I didn't like/need?

snipped out the comment out stuff. 

$ModLoad immark  # provides --MARK-- message capability    

$ModLoad imudp
$UDPServerRun 514

$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022

$WorkDirectory /var/spool/rsyslog

$IncludeConfig /etc/rsyslog.d/*.conf

*.*          /var/log/prd/fwlog

Finally my questions!

  1. I want to rotate and separate the routers and switches in a log with a date stamp called 'rslog-YYYY-MM-DD' also the firewalls into a log with a date stamp called 'fwlog-YYYY-MM-DD'

  2. I want to compress(gzip?) the logs after 48hrs.

what do I need to add to my config?

I think I added the directory and file to my rsyslog in /etc/logrotate.d/rsyslog 

/var/log/syslog
{
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}

/var/log/prd/fwlog*
/var/log/prd/rslog*

{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}

Thanks in advance for any help!

Security_Pete
  • 99
  • 1
  • 1
  • 11

2 Answers2

5

I want to rotate and separate the routers and switches in a log with a date stamp called 'rslog-YYYY-MM-DD' also the firewalls into a log with a date stamp called 'fwlog-YYYY-MM-DD'

To start, you need to separate your firewall and switches with filtering in rsyslog. Exactly how to do this varies based on the version of rsyslog you are running. They have changed configuration syntax quite a bit over time. My notes below are based on an older release of Rsyslog v3 that ships with Red Hat. You will want to verify this against the documentation for your release.

For a property based filter, it will look something like;

:fromhost-ip,isequal,"192.168.1.1"              /var/log/prd/fwlog
&~
:fromhost-ip,isequal,"192.168.1.254"            /var/log/prd/rslog
&~

The next part is your desired filename. For that, you will combine filtering with rsyslog's templates to generate dynamic filenames for your logs.

$template Firewall,"/var/log/prd/fwlog-%$YEAR%-%$MONTH%-%$DAY%"
$template Switch,"/var/log/prd/rslog-%$YEAR%-%$MONTH%-%$DAY%"

:fromhost-ip,isequal,"192.168.1.1"              -?Firewall
&~
:fromhost-ip,isequal,"192.168.1.254"            -?Switch
&~

I want to compress(gzip?) the logs after 48hrs.

The last part, compression, would rely on a daily cron job that compresses files. (Where $date is $today - 2.) The date command already has a built in format for YYYY-MM-DD, so we'll use that. (%F)

gzip /var/log/prd/*-$(date --date='2 days ago' +%F)
Aaron Copley
  • 12,525
  • 5
  • 47
  • 68
  • with the facility modifications would I just add the template above where the rules are? – Security_Pete Dec 30 '14 at 20:14
  • 1
    Yes, and apply the template to your facility filter. `local4.* -?Firewall` – Aaron Copley Dec 30 '14 at 20:19
  • Can you edit your response to show that in code so I can just paste it? I do not want to muddle through it with my limited knowledge of the context knowing that it took me hours to get the the messages to go to the the log :( – Security_Pete Dec 30 '14 at 20:22
  • `local4.* -?Firewall` under the template definition. Make backup copies of your config as you go so it's easier to roll back a change. – Aaron Copley Dec 30 '14 at 20:26
  • that worked perfectly! now if I can figure out why my facility4 stuff is not showing up... :/ – Security_Pete Dec 30 '14 at 20:35
  • `facility4`? `local4` is *a facility*, but [there is no](http://wiki.gentoo.org/wiki/Rsyslog#Facility) `facility4`? – Aaron Copley Dec 30 '14 at 20:40
  • oh man, I misspoke - local4 is correct. I mean facility - local4 I put that cron job into the crontab? correct? – Security_Pete Dec 30 '14 at 20:43
  • 1
    No. Your cronjob is to gzip your logs. This runs daily at 0100. Make sure `gzip` and `date` is in `/bin` on your system. `0 1 * * * /bin/gzip /var/log/prd/*-$(/bin/date --date='2 days ago' +%F)` – Aaron Copley Dec 30 '14 at 20:44
  • 1
    While specifying the IP address in the config may work, consider whether this is a long term good idea. As in, what about when you add a new switch? – dmourati Dec 30 '14 at 20:49
  • those are in /bin - but where does that snippet of code go? – Security_Pete Dec 30 '14 at 20:50
  • @dmourati - yes, this is why I decided to filter by using the facility as you suggested. I already had them separated that way, but since they did not show up visibly in the log file, I thought it would not work. – Security_Pete Dec 30 '14 at 20:52
  • @dmourati I upvoted your answer for using local4/local7. I agree that it's a good suggestion. – Aaron Copley Dec 30 '14 at 20:53
  • @Security_Pete crontab. – Aaron Copley Dec 30 '14 at 20:54
  • Upvoted yours too b/c you put more thought into the templates while I just pointed to the docs. I've moved on to syslog-ng b/c I hate all the changes to Ranier script and the conf files in rsyslog. – dmourati Dec 30 '14 at 20:54
  • @AaronCopley - does it belong in any specific order or can I append it to the end of the list I see by default? – Security_Pete Dec 30 '14 at 20:57
  • 1
    Append is fine. – Aaron Copley Dec 30 '14 at 21:00
  • @AaronCopley _ i moved to Ubuntu, will this affect the template used since it is still using rsyslog? I ask because it is not functioning and I am trying to troubleshoot – Security_Pete Jan 12 '15 at 16:08
  • The version of Rsyslog could affect it. (Not so much the OS itself.) Rsyslog has gone through a lot of change over the years that has not been backwards compatible. Fire up a new question and let's see what you have. Include version numbers. – Aaron Copley Jan 12 '15 at 18:41
2

First you should understand a bit more about syslog facility and severity. Those represent the two values you've added as *.* in your conf.

http://wiki.gentoo.org/wiki/Rsyslog#Facility

http://wiki.gentoo.org/wiki/Rsyslog#Severity

If you can set your sending daemons to use a different facility and/or severity for the routers/switches from the firewall, you should be able to easily create filter rules on your central log server to separate the logs out into different files as you've specified. For example, send routers/switches as local1 and firewall as local2.

Other than those settings, you could also separate out switch logs from firewall logs by filtering on the source IP address. The rsyslog property is called fromhost-ip.

Once you have the inbound rsyslog setup, you'll need to fine tune your logrotate settings. I think both file paths should be on one single line for starters. To compress after two days worth of logs will require some additional effort. See:

https://stackoverflow.com/questions/4495476/logrotate-compression-files-modified-x-number-of-days

You can test with logrotate -f /etc/logrotate.conf /etc/logrotate.d/rsyslog

For more details see:

http://articles.slicehost.com/2010/6/30/understanding-logrotate-on-debian-part-1

Community
  • 1
dmourati
  • 25,540
  • 2
  • 42
  • 72
  • 1
    Additional comment re: date strings in filenames for rsyslog. You probably want templates: http://www.rsyslog.com/doc/master/configuration/templates.html – dmourati Dec 30 '14 at 19:32
  • I do not see the facility in the messages I am receiving on linux. I have a kiwi syslog set up and it does show the facility. My routers are set to local4, and firewalls are local7. When I did not get the messages i simply set it to *.* – Security_Pete Dec 30 '14 at 19:36
  • 1
    Your rsyslog server sees the facility and can act on it. Something like local4.* /var/log/prd/rslog local7.* /var/log/prd/fwlog – dmourati Dec 30 '14 at 19:38
  • 1
    Good idea for using one facility for switches and one for routers, etc. – Aaron Copley Dec 30 '14 at 19:50
  • here is what I added, but I dont see anything going to my rslog... local7.* /var/log/prd/fwlog local4.* /var/log/prd/rslog – Security_Pete Dec 30 '14 at 20:11
  • Double check with tcpdump on your rsyslog server like so:https://www.fir3net.com/UNIX/Linux/how-to-determine-the-syslog-facility-using-tcpdump.html – dmourati Dec 30 '14 at 20:53
  • which directory is tcpdump located? I tried to run it and it replied command not found – Security_Pete Dec 30 '14 at 21:06
  • I checke din /usr/sbin but there is no tcpdump, there is tcpd, tcpdchk, tcpdmatch – Security_Pete Dec 30 '14 at 21:12
  • sudo apt-get install tcpdump – dmourati Dec 30 '14 at 22:28