0

I have an elk stack used with cyberoam, and I want to parse this message with logstash can you help me please:

"<30>date=2017-02-19 time=21:59:15 timezone=\"IST\" device_name=\"CR200iNG\" device_id=C20313272882-BQ2EUG log_id=010302602002 log_type=\"Firewall\" log_component=\"Appliance Access\" log_subtype=\"Denied\" status=\"Deny\" priority=Information duration=0 fw_rule_id=0 user_name=\"\" user_gp=\"\" iap=0 ips_policy_id=0 appfilter_policy_id=0 application=\"\" application_risk=0 application_technology=\"\" application_category=\"\" in_interface=\"PortF\" out_interface=\"\" src_mac=dd:dd:dd:02:1c:e4 src_ip=192.168.200.9 src_country_code= dst_ip=255.255.255.255 dst_country_code= protocol=\"UDP\" src_port=32771 dst_port=7423 sent_pkts=0 recv_pkts=0 sent_bytes=0 recv_bytes=0 tran_src_ip= tran_src_port=0 tran_dst_ip= tran_dst_port=0 srczonetype=\"\" srczone=\"\" dstzonetype=\"\" dstzone=\"\" dir_disp=\"\" connid=\"\" vconnid=\"\"",

To be clear: 

date=2017-02-19 
time=21:59:15 
timezone=\"IST\" 
device_name=\"CR200iNG\" 
device_id=C20313272882-BQ2EUG 
log_id=010302602002 
log_type=\"Firewall\" 
log_component=\"Appliance Access\" 
log_subtype=\"Denied\" 
status=\"Deny\" priority=Information duration=0 
fw_rule_id=0 
user_name=\"\" 
user_gp=\"\" 
iap=0 
ips_policy_id=0 
appfilter_policy_id=0 
application=\"\" 
application_risk=0 
application_technology=\"\" 
application_category=\"\" 
in_interface=\"PortF\" 
out_interface=\"\" 
src_mac=c4:04:15:02:1c:e4 
src_ip=192.168.200.9 
src_country_code= 
dst_ip=255.255.255.255 
dst_country_code= 
protocol=\"UDP\" 
src_port=32771 
dst_port=7423 
sent_pkts=0  
recv_pkts=0 
sent_bytes=0 
recv_bytes=0 
tran_src_ip= 
tran_src_port=0 
tran_dst_ip= 
tran_dst_port=0 
srczonetype=\"\" 
srczone=\"\" 
dstzonetype=\"\" 
dstzone=\"\" 
dir_disp=\"\" 
connid=\"\" 
vconnid=\"\""
,

and can you kindly tell me how to parse the captured packets using logstash, because there is an ability in the cyberoam to capture the packets in the network and I sent this data to logstash but logstash is not showing data in kibana

best regards

user136591
  • 1
  • 3

1 Answers1

1

Looking at the format here, it looks like the kv filter is most appropriate here.

filter {
  kv {
    source => "message"
    add_tag => [ 'cyberoam' ]
  }
}

The kv filter will split off key1=value key2=value sets in a string and turn them into fields. This seems like a good fit for you. Keys you know you don't want to include can be specified with exclude_keys => [ 'key1', 'key2' ]

sysadmin1138
  • 1,263
  • 11
  • 11
  • I am using the following: kv { source => "syslog_message" } mutate { replace => ["type", "%{syslog_program}"] remove_field => ["syslog_message", "syslog_timestamp"] gsub => [ 'message', '= ', '="" '] } – user136591 Feb 20 '17 at 05:47