1

I would like to create a statistic for JUnit test durations via logstash to elasticsearch.

Currently after every exectution of a JUnit test I write all metadata of the unit test (testclass, testmethod...) in a log file that will be parsed with logstash and send to elasticsearch

17:17:45,616 INFO test-class: CommandTest ... hostname: bernina os-name: Windows 7 ...
17:17:46,216 INFO test-class: GUITest ... hostname: bernina os-name: Windows 7 ...
17:17:47,216 INFO test-class: SeleniumTest ... hostname: bernina os-name: Windows 7 ...

The current logstash config looks like this

input {
    file {
        path => "E:/jenkins/builds/**/log"
        start_position => beginning 
    }
}
filter {
    grok {
        patterns_dir => ["E:/logstash-2.2.1/pattern"]
        match => { "message" => "test test-class: %{DATA:testClass}...hostname: %{DATA:hostname} os-name: %{DATA:osName}... }
    }
    if "_grokparsefailure" in [tags] {
        drop { }
    }
}
output {
    elasticsearch {
        hosts  => ["imstestserver"]
    }
}

This works fine but the problem is that a lot of meta data is repeated and slows down our build (hostname,os-name,os-architecture-data-model, pid, jenkins-build-numme is always the same)

What I would like to do is log this repetitive information just once when the test suite has been started.

17:17:44,616 INFO Starting suite with hostname: bernina os-name: Windows 7 ...
17:17:45,616 INFO test-class: CommandTest...
17:17:46,216 INFO test-class: GUITest..
17:17:47,216 INFO test-class: SeleniumTest...

Is there a way in logstash that I can store the fields from the very first line (hostname, os-name) and reuse this for every upcoming line. So that in elasticsearch I would have the following:

"_source": {
    "testClass": "GUITest",
    "hostname": "bernina",
    "osName": "Windows 7",
  }

"_source": {
    "testClass": "SeleniumTest",
    "hostname": "bernina",
    "osName": "Windows 7",
  }

"_source": {
    "testClass": "CommandTest",
    "hostname": "bernina",
    "osName": "Windows 7",
  }
sandrozbinden
  • 1,577
  • 1
  • 17
  • 28

0 Answers0