1

In logstash I am getting this output, however, I am trying to get repo: "username/logstashrepo" only from this output. Please share your thoughts how to grep only that value and assign to variable.

message: "github_audit: {"actor_ip":"192.168.1.1","from":"repositories#create","actor":"username","repo":"username/logstashrepo","user":"username","created_at":1416299104782,"action":"repo.create","user_id":1033,"repo_id":44744,"actor_id":1033,"data":{"actor_location":{"location":{"lat":null,"lon":null}}}}",
@version: "1",
@timestamp: "2014-11-18T08:25:05.427Z",
host: "15-274-145-63",
type: "syslog",
syslog5424_pri: "190",
timestamp: "Nov 18 00:25:05",
actor_ip: "192.168.1.1",
from: "repositories#create",
actor: "username",
repo: "username/logstashrepo",
user: "username",
created_at: 1416299104782,
action: "repo.create",
user_id: 1033,
repo_id: 44744,
actor_id: 1033,

I am using this in my config file:

input {
  tcp {
    port => 8088
    type => syslog
  }
  udp {
    port => 8088
    type => syslog
  }
}

filter {
  grok {
    match => [
      "message",
      "%{SYSLOG5424PRI}%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{GREEDYDATA:message}"
    ]
    overwrite => ["host", "message"]
  }
  if [message] =~ /^github_audit: / {
    grok {
      match => ["message", "^github_audit: %{GREEDYDATA:json_payload}"]
    }

    json {
      source => "json_payload"
      remove_field => "json_payload"
    }
  }
}

output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

I actually posted the question here, for some reason I can't edit and followup.

how to grep particulr field from logstash output

Community
  • 1
  • 1
bsd
  • 5,943
  • 3
  • 19
  • 16

1 Answers1

1

You can have the json filter store the expanded JSON object in a subfield. Use the mutate filter to move the "repo" field into the toplevel and delete the whole subfield. Partial example from the json filter and onwards:

json {
  source => "json_payload"
  target => "json"
  remove_field => "json_payload"
}

mutate {
  rename => ["[json][repo]", "repo"]
  remove_field => "json"
}
Magnus Bäck
  • 11,381
  • 3
  • 47
  • 59
  • sorry, one more doubt, can i save this particular value (repo: "username/logstashrepo") into any variable or file from above output, I need to use this value for execution thanks in advance – bsd Nov 18 '14 at 17:04
  • Logstash has several output plugins available. You could e.g. save the field or message to a file or run a command. – Magnus Bäck Nov 18 '14 at 20:57
  • Hi , I've used to remove other fields from my output and I need only repo: "username/logstashrepo" from output , i've used this method, but I am not getting "repo: "username/logstashrepo" only json { source => "json_payload" remove_field => "json_payload" } mutate { rename => ["[json][repo]", "repo"] remove_field => ["[json][syslog5424_pri]"] remove_field => ["[json][host]"] remove_field => ["[json][type]"] } – bsd Nov 25 '14 at 11:41