2

I have a logstash.conf file where i'm defining two distinct path for two different type of logs one is for system logs and another is for network logs. However, these logs are being collected on the same directory location as /scratch/rsyslog where its creating an individual folder for each host before dumping the logs, for example ..

1) /scratch/rsyslog/server01/messages.log /scratch/rsyslog/server02/messages.log and so on for the system logs

2) For network logs its like: /scratch/rsyslog/Sep/messages.log

Below is the input Filter and path for both type of logs. now the problem is that i'm using wildcard to match to get all the names with * here path => [ "/scratch/rsyslog/*/messages.log" ] which gets everything.

input {
  file {
    path => [ "/scratch/rsyslog/*/messages.log" ]
    type => "syslog"
  }
  file {
    path => [ "/scratch/rsyslog/Sep/messages.log" ]
    type => "apic_logs"
  }
}

So, in the First path which is system logs i need that starts with lowercase letters which may include some numbers though like server01.

Maybe i'm thinking ^[a-z0-9]

Whereas in second path which is network logs i need to get where first letter startswith uppercase letter following lowercase (these are month names usually like i mentioned Sep , it gets changed itself on the month end).

maybe ^[A-Z].* for second one

i'm looking to get a regex which can fit into this situation. any help will be much appreciated.

nitzien
  • 1,117
  • 9
  • 23
Karn Kumar
  • 8,518
  • 3
  • 27
  • 53

1 Answers1

2

Unfortunately, path in file plugin doesn't support regex. It only support wildcard.

nitzien
  • 1,117
  • 9
  • 23
  • One possibility outside logstash is to write a bash script that based on regular expression, create symbolic links to these folders at another location. Then, logstash can stash from new location. This script can be scheduled by cron and add new symbolic links foe any new folders. – nitzien Sep 28 '18 at 16:20
  • @hmm , but that's something which will not work for me. can we do something @ rsyslog.conf ? – Karn Kumar Sep 28 '18 at 17:38
  • i realized ` ^[A-Z].*` this is not regex rather its globbing which can be used i beleieve. – Karn Kumar Sep 29 '18 at 10:48