Use Custom Test Case Designer for Better visibility of the test cases

Question

We are using Xray as a Jira Plugin and previously we have written automated test case in a separate project. We are using Testng for Test case automation. After integration of Jira with Xray Plugin, we have migrated existing test cases from Testng and created corresponding Xray Test cases. We want to take it to next level. So idea is we want to analyse the value of various attributes that are configured in Testng project.

Eg. Suppose we want to test Jira epic that has an acceptance criteria to show his account balance. And in TestNg we have written couple of Test cases, we want to view what are the values we have used for different use cases in form of table and also get possible option to execute any single test case by that table itself. So in future if I want to run specific test case I can run directly. And later export test cases to Xray Test Case

it is not quite clear about this question, what are your main goals? — Murat Ishenbaev, Jun 11 '23 at 18:55
From my understand you would like to have 2 things: 1) see the test data ok Xray side and the results for each data row. And 2) trigger the execution of one of those rows from Jira side. Is that it? — Sérgio, Jun 15 '23 at 12:42

score 0 · Accepted Answer · answered Jun 20 '23 at 16:36

To be able to track test data, if you're using TestNG, you can use the xray-testng-extensions package. It has preliminary support for it, but it requires you to:

use the "new" Xray JSON report, instead of the JUnit XML one
use "manual" Tests (i.e., tests of "Manual" test type) and report results against them

To trigger a Test from Jira/Xray side, it's not direct; you can trigger a CI/CD job, which will have the logic to execute whatever tests but it's up to you to implement that. Triggering a CI/CD job requires having Xray Enterprise, but it's also possible using some Jira customization (e.g., with ScriptRunner or Jira automation capabilities).

To be able to analyze data rows, of data-driven tests, there's no specific report for it as of June 2023. But please have in mind that it all depends exactly of what you aim to analyze (e.g., the dimensions, etc). One suggestion for it could try Allure Reports for that purpose (once again, it all depends on what you want to measure/analyze; it would be great if you can provide the exact reports/metrics to the Xray support team).

So idea is if somehow i can see the values that we have in testng.so that say i select one epic somehow it shows me all the values in table format . And in that table, even I can also edit the test case value and run it from then and there itself and also export it as Xray Test cases. — Shirsh Sinha, Jun 21 '23 at 15:20
You can see the table data on a test run (on a specific result of a specific test). The other things are not possible/not straightforward. You can create manual tests and define the Data on Xray side but editing it both in testng and Xray is not possible, if that's what you were trying to achieve. — Sérgio, Jun 21 '23 at 16:49

Use Custom Test Case Designer for Better visibility of the test cases

1 Answers1