What would be grok filter

Question

This is the following json sample in which I want to filter and index on basis of Client Id and User Id which is in Message tag in json.

"message": "12 Jul 2016 15:28:14,851 http-bio-9080-exec-3 [INFO ]    corporate_access                                    - Request details - Uri: /corporate/create, Ip: x.x.x.x, User id: 12461, Client id:11048",

I want to index the user activity on basis of Client Id and User Id. My filter in logstash conf is :

filter {
  grok {

match => {
        "message" => "Uri: %{URIPATHPARAM:url}%{SPACE}Ip: %{IP:ip},%{SPACE}User id: %{WORD:Userid}, Client id:%{WORD:Clientid}"
}

 }
}

Can you post your log line and logstash config with grok filter separately? — Mrunal Pagnis, Jul 12 '16 at 07:45
The number of spaces, etc is different in your comment and in your question. Which Log line format is the correct one? — Mrunal Pagnis, Jul 12 '16 at 09:30
@MrunalPagnis The comment form might strip multiple spaces. But if you are using `%{SPACE}` in your pattern, it won't matter. — baudsp, Jul 12 '16 at 09:41

baudsp · Answer 1 · 2016-07-12T08:22:34.193

0

You can use this grok filter:

grok {
    match => {
        "message" => [
            "%{MONTHDAY} %{MONTH} %{YEAR} %{TIME} %{GREEDYDATA} \[%{DATA}\]%{SPACE}%{WORD}%{SPACE}- Request details - Uri: %{URIPATH}, Ip: %{IP}, User id: %{NUMBER:user_id}, Client id: %{NUMBER:client_id}"
        ]
    }
}

Note: I have removed the ** around User id and Client id, since it look like it was just to add emphasis on the interesting parts of the log line. But if you really have ** in your logs, the pattern must be modified with: \*\*User id:\*\* %{NUMBER:user_id}, \*\*Client id:\*\*%{NUMBER:client_id}.

edited Jul 12 '16 at 08:22

answered Jul 12 '16 at 08:16

baudsp

4,076
1
17
35

@user2481458 With the grok filter from your question? – baudsp Jul 12 '16 at 09:46
I have no idea why your filter (`filter { grok { match => { "message" => "User id: %{NUMBER:UserId}, Client id:%{NUMBER:ClientId}" } } }`) doesn't work. I tested it (LS 2.2) without problems – baudsp Jul 12 '16 at 10:15

score 0 · Answer 2 · answered Jul 19 '16 at 12:59

This worked !!

  filter {
      if [type] == "corporate-access" {
        grok {
          break_on_match => false
          match => { "message" => "Uri: %{URIPATHPARAM:url}%{SPACE}" }
           match => { "message" => "User id: %{WORD:Userid}, Client id:%{WORD:Clientid}" }
           add_tag => "%{Userid}"
           add_tag => "%{Clientid}"
           add_tag => "%{url}"

        }
      }

What would be grok filter

2 Answers2