0

My team is just getting started with X-Ray, and we are setting up our pipelines. However, while doing this I noticed that if I submit a Junit xml file to X-Ray via the REST api, it will create new tests for any test data that isn't already in the system.

Is there a way to have X-Ray ignore test results for tests that don't exist for the test execution? I don't want it constantly creating extra tests.

For example:

  1. (Jira/X-Ray Server) TestExecution MyExecution has test testA
  2. From client, I submit a Junit xml file containing results for testA and testB in the MyExecution TestExecution
  3. testB now exist on the server under MyExecution

I would like to be able to submit the Junit xml file without it creating extra tests.

caffein
  • 575
  • 8
  • 26
  • Tests of the type "generic" have the field "Test definition". When you look into the JUnit XML, you can find the attribute "name" for each test case. The name and the test definition should be identical. – Sven-Eric Krüger Sep 08 '20 at 17:26
  • I realize that, and have actually used it. However, if a developer forgets to set that correctly, or is working on building new tests, it will result in the problem of unwanted tests being created. – caffein Sep 08 '20 at 17:36

1 Answers1

0

Whenever you import automation results using the REST API, or any of the available CI plugins, Xray will autoprovision ("generic") Test entities. The flow is detailed here.

Xray tries to find a unique identifier for the automated test; in the case of JUnit, it's based on the full classname plus the name of the test method; this will become part of the Generic Definition field. The process for JUnit is described in more detail here.

How it works for a different test automation framework/report formats, is similar and is detailed on respective documentation pages. If a "generic" Test is found, then the Test is reused and a Test Run is created against it. Otherwise, the Test will be auto-provisioned.

This process isn't configurable. However, in theory, if the user that you use for the submission of automation results isn't able to create Test issues, you may have what you need.

Things like these are usually not configurable because they are normally a consequence of applying good practices usually discussed internally with the team(s).

Sérgio
  • 1,777
  • 2
  • 10
  • 12