How to send test information within Test Execution in Jira Xray?

Question

We are trying to send specific test launch information to the Test Execution in Xray with no luck. Tests are built on Jenkins and communication with Xray is via Xray Connector. We want to have test information in the description and comments of the Test Execution.

Any suggestions will be appreciate

Tech stack: WebdriverIO + Cucumber + SauceLabs + Jira Xray

Can you please clarify your question. What do you mean exactly with no luck? Besides, what flow are you following https://docs.getxray.app/pages/viewpage.action?pageId=62267221#TestinginBDDwithGherkinbasedframeworks(e.g.Cucumber)-Workflows ? Where do you edit your Cucumber scenarios? In Xray or outside of Jira (e.g. in some IDE and persist them on Git)? — Sérgio, Feb 18 '21 at 11:18
Thank you for answering. By no luck, I meant that we don’t know how to do it. Jira is our source of truth. In Jenkins, we are setting import and export tasks. When we execute tests feature files are retrieved from Xray, then tests launch with webdriverIO, then a couple of JSONs with results is created. We join all the results to one JSON and set Export task with Cucumber JSON format in Jenkins plugin. Additionally, we don't want to create a new Test Execution just update the old one. — beauvoir, Feb 19 '21 at 08:03
Ok, understood; thanks. So you want to customize the Description field of the Test Execution issue and at the same time add comments on the Test Execution issue? — Sérgio, Feb 19 '21 at 13:09
Yes indeed, of course, if it's possible. If it's some kind of tricky I could customize one of them - I just need to add somewhere in Test Execution information like a link to Sauce Labs. — beauvoir, Feb 22 '21 at 07:35

score 1 · Answer 1 · answered Feb 23 '21 at 15:18

I did some research on Jira and Xray side.

In Xray server, when using the so called "cucumber multipart" endpoint, a new Test Execution issue will always be created. You can specify a JSON content that contains a "fields" JSON object where you can set the values for some of the custom fields you have on the Test Execution (e.g. the "summary" or other). Bellow, you may find an example of the auxiliary JSON object (stored in a file for example createTestExec_with_description.json). You can define the description field and embed there a link. This follows the "simple update" syntax of Jira, as described here.

However, you cannot add a comment while the issue is being created (this is a "limitation"/decision of Jira REST API when you invoke the creation of Jira issues).

{
"fields": {
    "project": {
        "key": "BOOK"
    },
    "summary": "Results for cucumber execution",
    "description": "For more info please check [here|https://www.example.com]",
    "issuetype": {
        "id": "9"
    },
    "customfield_11805" : [
        "iOS"
    ],
"fixVersions" :
    [
     {
        "name": "1.0"
     }
    ]
}
}

Then you can submit you cucumber JSON report plus this auxiliary file using something like:

curl -u $USERNAME:$PASSWORD -F info=@createTestExec_with_description.json -F result=@data.json $JIRA_BASE_URL/rest/raven/1.0/import/execution/cucumber/multipart

If you're using Xray on Jira Cloud, Xray's API allows to specify an existing Test Execution issue to update/overwrite. I didn't check if you can use a mix of "simple" and "verb/operation" updates, as Atlassian refers to them in the docs (i.e. use the "fields" object plus and "update" object at the same time). If that was possible, you would probably do something like:

{
"fields": {
    "project": {
        "key": "BOOK"
    },
    "summary": "Results for cucumber execution",
    "description": "For more info please check [here|https://www.example.com]",
    "issuetype": {
        "id": "9"
    },
    "customfield_11805" : [
        "iOS"
    ],
"fixVersions" :
    [
     {
        "name": "1.0"
     }
    ]
},

"update": {
  "comment": [
     {
        "add": {
           "body": "latest results [here|https://www.example.com]"
        }
     }
   ]
}
}

The previous examples have some fields that you can safely remove, depending on your needs. You'll also to adapt them to your Jira configuration/environment.

score 0 · Answer 2 · answered Feb 26 '21 at 07:15

@Sergio It didn't work in my case. I've created two JSON files, 'one.json' that is combined by adding all JSON test results with jq command, and the second file, called 'extra.json' that contains JSON structure that you have shown. In extra.json key was related to ticket number (taken from one.json) and custom fields name taken from Jira API /rest/api/2/issue/ABC-194/editmeta. Using multipart upload with URL like this

curl -k -H "Content-Type: application/json" -X POST -u login:password -F info=@extra.json -F result=@one.json https://jira/rest/raven/1.0/import/execution/cucumber

Jira has accepted both JSON files but on Jira, nothing happens, no change, no added text. Next step I've combined those two files into one bigger JSON file and using normal upload (using Jira API but not multipart) and the same result, Jira has accepted JSON but nothing new has Jira shows using the web interface. Comining json, changing key - nothing gives me any small change.

extra.json:

{
"fields": {
    "project": {
        "key": "@ABC-9"
    },
    "summary": "Results for cucumber execution",
    "description": "For more info please check [here|https://www.example.com]",
    "issuetype": {
        "id": "9"
    },
    "customfield_16134" : [
        "link"
    ],
"fixVersions" :
    [
     {
        "name": "1.0"
     }
    ]
}
}
Jira's definition gathered using api:
"customfield_16134": {
  "required": false,
  "schema": {
    "type": "string",
    "custom": "com.atlassian.jira.plugin.system.customfieldtypes:textfield",
    "customId": 16134
  },
  "name": "Revision",
  "operations": [
    "set"
  ]
},

You need to use the specific endpoint for multipart which is something like https://jira/rest/raven/1.0/import/execution/cucumber/multipart It's called multipart because it accepts multipart content; besides it has a specific endpoint that you need to use. Therefore, the curl request should be something like: curl -k -H "Content-Type: application/json" -X POST -u login:password -F info=@extra.json -F result=@one.json https://jira/rest/raven/1.0/import/execution/cucumber/multipart — Sérgio, Feb 26 '21 at 11:29
Sorry, that I didn't mention that in the previous answer but I used multipart endpoint and It didn't work. — beauvoir, Feb 27 '21 at 12:37
can you explain exactly what you obtained in the API call response? And exactly what did not work? Was the api call successful? Did the summary and the description got set? If not, what did you got? As I've mentioned, adding comments at the same time you create an issue (i.e. the Test Execution issue) shoudln't be supported — Sérgio, Mar 01 '21 at 11:02

score 0 · Answer 3 · answered Dec 14 '21 at 21:51

In my case, was task to update Test Run status via API. I did next steps:

Retrieve information of test runs that are in test execution

curl -H "Content-Type: application/json" -X GET -u <jira_user_username>: https://<JIRA_PORTAL>/jira/rest/raven/1.0/api/testexec/<TEST_EXECUTION_ISSUE-KEY>/test

Sample Response

[
     {
        "id": 10444291,
        "status": "TODO",
        "key": "<TEST_RUN_ISSUE_KEY>",
        "rank": 1
      }
]

Do update of Test Run using above Test Run Id

curl -H "Content-Type: application/json" -X PUT -u <jira_user_username>: https://<JIRA_PORTAL>/jira/rest/raven/1.0/api/testrun/10368224/status?status=PASS

Response: 200 status code.

XRay API reference: https://docs.getxray.app/display/XRAY/REST+API

How to send test information within Test Execution in Jira Xray?

3 Answers3