-1

I trying to use logstash 7.6.1 for logging my f5 waf/asm, when i trying to collect some logs from my f5 remote logging, i run logstash -f f5.config from my elk server.. it says this and looped:

[[main]>worker1] kv - Exception while parsing KV {:exception=>"Invalid FieldReference: `info tmm2[16492]: Rule /Common/myrule : source logreq: /mywebsitepath/"}

so what's problem here and how to fix this?

this is my f5.config :

input {
  syslog {
    port => 5144
  }
}
filter {
  kv {
    field_split => ","
  }
  mutate {
    split => { "attack_type" => "," }
    split => { "sig_ids" => "," }
    split => { "sig_names" => "," }
    split => { "sig_cves" => "," }
    split => { "staged_sig_ids" => "," }
    split => { "staged_sig_names" => "," }
    split => { "staged_sig_cves" => "," }
    split => { "threat_campaign_names" => "," }
    split => { "staged_threat_campaign_names" => "," }
    split => { "violations" => "," }
    split => { "sub_violations" => "," }
  }
  geoip {
    source => "ip_client"
  }
}
output {
  elasticsearch {
    hosts => ['myip:9200']
    index => "waf-logs-%{+YYY.MM.dd}"
  }
}

this is kibana i used for visualize logstash : Project

Thanks in advance.

s1gnific4nt
  • 53
  • 2
  • 11
  • You need to share a sample of the messages that are giving you this error so people can try to reproduce it and help you. – leandrojmp Apr 21 '20 at 16:33
  • Hi, thank you for replying me. sample of the messages like what? all i know error messages i got is there in part of "Exception while parsing KV". – s1gnific4nt Apr 23 '20 at 03:32
  • A sample of the messages that your F5 is sending, an example of the structure of your log messages, if you do not have it, start your pipeline without the filter block and the messages will be stored in your index without any parsing, then you can look in Kibana and share some messages (overriding personal information). – leandrojmp Apr 23 '20 at 03:53

1 Answers1

0

The KV Parser by default treats [] special characters for the key.

[a][0]=1 is valid and would set a.0 = 1 in Elastic
[a=1 is not

You can set

remove_key_value => "\[\]"

to circumvent the issue

Muttley
  • 21
  • 3