I would like to create a statistic for JUnit test durations via logstash to elasticsearch.
Currently after every exectution of a JUnit test I write all metadata of the unit test (testclass, testmethod...) in a log file that will be parsed with logstash and send to elasticsearch
17:17:45,616 INFO test-class: CommandTest ... hostname: bernina os-name: Windows 7 ...
17:17:46,216 INFO test-class: GUITest ... hostname: bernina os-name: Windows 7 ...
17:17:47,216 INFO test-class: SeleniumTest ... hostname: bernina os-name: Windows 7 ...
The current logstash config looks like this
input {
file {
path => "E:/jenkins/builds/**/log"
start_position => beginning
}
}
filter {
grok {
patterns_dir => ["E:/logstash-2.2.1/pattern"]
match => { "message" => "test test-class: %{DATA:testClass}...hostname: %{DATA:hostname} os-name: %{DATA:osName}... }
}
if "_grokparsefailure" in [tags] {
drop { }
}
}
output {
elasticsearch {
hosts => ["imstestserver"]
}
}
This works fine but the problem is that a lot of meta data is repeated and slows down our build (hostname,os-name,os-architecture-data-model, pid, jenkins-build-numme is always the same)
What I would like to do is log this repetitive information just once when the test suite has been started.
17:17:44,616 INFO Starting suite with hostname: bernina os-name: Windows 7 ...
17:17:45,616 INFO test-class: CommandTest...
17:17:46,216 INFO test-class: GUITest..
17:17:47,216 INFO test-class: SeleniumTest...
Is there a way in logstash that I can store the fields from the very first line (hostname, os-name) and reuse this for every upcoming line. So that in elasticsearch I would have the following:
"_source": {
"testClass": "GUITest",
"hostname": "bernina",
"osName": "Windows 7",
}
"_source": {
"testClass": "SeleniumTest",
"hostname": "bernina",
"osName": "Windows 7",
}
"_source": {
"testClass": "CommandTest",
"hostname": "bernina",
"osName": "Windows 7",
}