How to compare WebdriverIO reports and generate the difference?

Question

We are using WebdriverIO for our automated tests and we generate HTML reports with Mochawesome in the end based on the result JSON files.

Now we have a lot of implemented tests and we want to fetch the difference between two testruns as fast as possible. Therefore it would be cool if we will have a possibility to compare two testrun results with each other and to generate also a HTML report only with the test result differences.

Maybe there is a still existing implemantation/package to do that? Yes, of course it is possible to compare the two different JSON result files with each other, but I prefer a still implemented solution to save effort.

How would you do the comparison in my case?

Thanks, Martin

score 0 · Answer 1 · answered Jan 14 '20 at 11:06

0

You could set up a job in a CI tool like Jenkins.

Here it always compares the latest results with the previous build and tells you if it is a new failure, regression issue or a fixed script.

Regression indicates that test passed in the previous build, but failing in the new build
Failed indicates, it is failing from the past couple of builds
Fixed indicates, that it was failing in the last build, but now passing in the latest build

answered Jan 14 '20 at 11:06

Seema Nair

155
7

what CI system you are using? and if it is Jenkins what plugin is it? – GRme Jan 14 '20 at 17:35
Hi Martin, I am using Jenkins . Results are using Junit Report plugin – Seema Nair Jan 15 '20 at 05:41
Ah ok, so with the JUnit plugin it is possible to get such history data? That would be a possible solution. Thanks! I will give it a try – GRme Jan 15 '20 at 05:43
I wish you the Best Martin! – Seema Nair Jan 15 '20 at 06:49

How to compare WebdriverIO reports and generate the difference?

1 Answers1