Grok RSpec test failing

Question

I'm trying to get grok working with logstash but struggling to get off the starting block. I've tried to simplify things down to a succinct test which is here:

require "test_utils"

describe "basic grokking" do
  extend LogStash::RSpec

  config <<-CONFIG
    filter {
      grok {
        match => [ "message", "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" ]
      }
    }
  CONFIG

  sample ({'@message' => '55.3.244.1 GET /index.html 15824 0.043'}) do

    puts subject.inspect # view http://www.codeshare.io/OhDC0

    insist { subject["client"] } == "55.3.244.1"
  end
end

I get the following error:

  1) basic grokking "{"@message":"55.3.244.1 GET /index.html 15824 0.043..." when processed
 Failure/Error: Unable to find matching line from backtrace
 Insist::Failure:
   Expected "55.3.244.1", but got nil

No amount of syntax tweaking is getting a result and I also can't figure out how to inspect the subject to find out what is there.

The ultimate aim is to use grok to extract the following HttpRequestId:

[HttpRequestId = e29041b2-a4a0-4bf3-ba05-2de5e7bcf444] 2015/04/10 08:12:51:632 [DEBUG] ... log message ...

Using something like this:

grok {
    match => [ "Message", "\[HttpRequestId = %{UUID:HttpRequestId}" ]
}

NOTE I have checked my patterns against https://grokdebug.herokuapp.com/ and they work. It's something to do with the way I am testing.

What language are you writing the wrapper for this in? Looks like Ruby; might be worth tagging. — FarmerGedden, Apr 10 '15 at 13:27
Yes it's ruby. Added the tag. Reference: http://koendc.github.io/2013/10/11/logstash-test-configuration.html — Matt Canty, Apr 10 '15 at 13:28
What version of logstash are you using? I'm encountering the same issue with logstash 1.4.2. — mooreds, Apr 28 '15 at 19:28

score 2 · Accepted Answer · answered Apr 28 '15 at 20:46

2

After extensive investigation, here's the minimal rspec test for a grok expression for logstash 1.4.2:

# encoding: utf-8

require "test_utils"

describe "simple test" do
  extend LogStash::RSpec

   config <<-CONFIG
    filter {
      grok {
         match        => { "message" => "am%{GREEDYDATA:foo}" }
      }
    }
  CONFIG

  sample 'amessage' do
    insist { subject["foo"] } == "essage"
    insist { subject["message"] } == "amessage"
  end
end

Note that you must have the # encoding ... line, otherwise you'll get the Expected "essage", but got nil message.

This mailing list message helped: https://groups.google.com/d/msg/logstash-users/rs6jlAxv36M/1ylU-GJ-DVQJ

answered Apr 28 '15 at 20:46

mooreds

4,932
2
32
40

Now I see the #encoding everywhere. I'm sure I've never seen it before now! Look forward to trying this out tomorrow. – Matt Canty Apr 28 '15 at 20:56
I will do. Didn't have a chance today. I'll keep this up to date when I have something. – Matt Canty Apr 29 '15 at 22:17
2

Apologies for the delay, this work is on the back-burner now so I'm only looking at it when I get the chance. Thanks for pointing out the encoding, that was the trick. – Matt Canty May 29 '15 at 14:51

Grok RSpec test failing

1 Answers1