4

I have two runners in my automation project as follows:

  1. Main runner - Executes all the @ui-test tagged test cases and if a scenario is failed target/rerun.txt will be populated with the scenario location (e.g. features/Dummy.feature:22):

    @RunWith(Cucumber.class)
@CucumberOptions(
    features = "classpath:features",
    plugin = {"pretty", "html:target/cucumber-html-report", "json:target/cucumber.json", "rerun:target/rerun.txt"},
    tags = {"@ui-test", "~@ignore"}
)

public class RunCukesTest {
}

  2. Secondary runner - Re-executes the scenarios from target/rerun.txt:

    @RunWith(Cucumber.class)
@CucumberOptions(
    features = "@target/rerun.txt",
    plugin = {"pretty", "html:target/cucumber-html-report-rerun", "json:target/cucumber_rerun.json"}
)

public class ReRunFailedCukesTest {
}

When the execution is performed two result json files are created:

  • cucumber.json
  • cucumber_rerun.json

Jenkins will collect the results via Cucumber-JVM Reports plugin and will create a combined report.

The problem is, even if all the target/rerun.txt tests are passed in the second run, the report status will remain failed because of the cucumber.json.

Is there a way (to set up Cucumber-JVM Reports plugin or modify the upper presented runners) to overwrite cucumber.json with the results from cucumber_rerun.json and to publish only the modified cucumber.json?

Another sub-keywords: maven, java, cucumber-java8, cucumber-junit, junit

LeeWay
  • 793
  • 3
  • 16
  • 28
  • I am using ruby + cucumber for my tests and my issue is exactly the same as yours. In my case, the cucumber creates only one json file that has both re-run and the first time run status. The issue here is the tests are updated but not the reports. Any approach that solved the issue for you? – Emjey Sep 25 '17 at 08:52

2 Answers2

1

I had problem similar to yours, though, I've used single runner, handled re-runs from testNG(re-runs was one of the reasons I've switched from JUnit to TestNG) directly and as a results I had increased amount of tests in my json report. My solution was to clean json files afterwards, despite the fact that Jenkins knows about failed tests it won't mark build as failed or as unstable. In your particular case you may try to somehow match tests from rerun.json and exclude them from regular json report. For parsing jsons I may recommend using Jackson FasterXML

Mikhail
  • 665
  • 4
  • 16
  • Hi @Mikhail, Can you please expand your answer? If i have lot of testng tests and want to combile all jsons, what should I do ? – Nael Marwan May 29 '17 at 16:32
1

I use Jenkins cucumber reporting latest release with below config in Jenkins.

Image Of Config In Jenkins

1st Runner

@RunWith(Cucumber.class)
@CucumberOptions(
  features="FolderFeature",
  glue={"Gluefolder"},
  plugin={"html:target/cucumberpf-html-report",
    "json:target/cucumberpf.json"}
  )

public class RunPF {

}

2nd Runner 

@RunWith(Cucumber.class)
@CucumberOptions(
  features="Blah/Test.feature",
  glue={"mygluefolder"},
  plugin={"html:target/cucumber-html-report",
    "json:target/cucumber.json"}
  )

public class RunRA {
 
}

I had failed in both .json files and when it passed both were merged and updated correctly in one cucumber report.

Here is the error:

[CucumberReport] Preparing Cucumber Reports
[CucumberReport] JSON report directory is "C:\Users\ajacobs\workspace\com.mytest.framework\target\"
[CucumberReport] Copied 2 json files from workspace "C:\Users\admin\workspace\yourtest\target" to 
  reports directory "C:\Users\admin\.jenkins\jobs\Regression\builds\21\cucumber-html-reports\.cache"
[CucumberReport] Processing 2 json files:
[CucumberReport] C:\Users\admin\yourtest\builds\21\cucumber-html-reports\.cache\cucumber.json
[CucumberReport] C:\Users\admin\yourtest\builds\21\cucumber-html-reports\.cache\cucumberpf.json
Finished: SUCCESS
Adenaan Jacobs
  • 11
  • 4