0

I'm trying to use Import Execution Results - REST to import Behave JSON results with Multipart. I can get the import to work, but the "The execution details displays the result of the Cucumber Scenario." does not work.

Here's what I'm doing:

  • I'm creating a new Test Execution.
  • I'm using a test I already create(type: Cucumber, Scenario Type: Scenario Outline)
  • I'm using the following feature file with Behave

I'm using the following feature file with Behave:

Scenario Outline: Verify the UI content of SOME page
    Given I access to SOME in "PAGE" as "DEFAULT_USER"
    When I set "<toggle_type>" toggle "with" page refresh
    Then I can see page displayed with the title "SOME | PAGE"
    And I compare "SOME IMAGE" image with a "98" similarity with "<toggle_type>"

    @SOME-TAG
    Examples:
      | toggle_type |
      | spring      |
      | default     |

I'm running behave with:

behave --no-skipped --tags=SOME-TAG -f json.pretty -o='reports/behave/Xray_behave_result.json'

I'm importing the report with:

POST /rest/raven/1.0/import/execution/behave/multipart

The server reply is:

{"testExecIssue":{"id":"4678506","key":"SOMETHING-55744","self":"https://stage-myjira.somedomain.com/rest/api/2/issue/4678506%22%7D,%22testIssues%22:%7B%22success%22:%5B%7B%22id%22:%224678305%22,%22key%22:%22SOME-TAG%22,%22self%22:%22https://stage-myjira.somedomain.com/rest/api/2/issue/4678305%22%7D%5D%7D,%22infoMessages":\[\]}

But when I look at the Test Details for the test excution:SOMETHING-55744 of the test: SOME-TAG, it's empty: empty results screenshot

This is the resulting json: "Xray_behave_result.json"

[
  {
    "elements": [
      {
        "keyword": "Scenario Outline",
        "location": "something_features/features/something_web.feature:74",
        "name": "Verify the UI content of SOME page -- @2.1 ",
        "status": "passed",
        "steps": [
          {
            "keyword": "Given",
            "location": "something_features/features/something_web.feature:60",
            "match": {
              "arguments": [
                {
                  "name": "page",
                  "value": "some_page"
                },
                {
                  "name": "something",
                  "value": "other thing"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:70"
            },
            "name": "I access to SOME \"page\" as \"DEFAULT USER\"",
            "result": {
              "duration": 11.369250059127808,
              "status": "passed"
            },
            "step_type": "given"
          },
          {
            "keyword": "When",
            "location": "something_features/features/something_web.feature:61",
            "match": {
              "arguments": [
                {
                  "name": "toggle_type",
                  "value": "spring"
                },
                {
                  "name": "refresh_condition",
                  "value": "with"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:362"
            },
            "name": "I set \"spring\" toggle  \"with\" page refresh",
            "result": {
              "duration": 3.7522451877593994,
              "status": "passed"
            },
            "step_type": "when"
          },
          {
            "keyword": "Then",
            "location": "something_features/features/something_web.feature:62",
            "match": {
              "arguments": [
                {
                  "name": "page_title",
                  "value": "SOME | PAGE"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:261"
            },
            "name": "I can see page displayed with the title \"SOME | PAGE\"",
            "result": {
              "duration": 9.731576442718506,
              "status": "passed"
            },
            "step_type": "then"
          },
          {
            "keyword": "And",
            "location": "something_features/features/something_web.feature:63",
            "match": {
              "arguments": [
                {
                  "name": "banner",
                  "value": "SOME IMAGE"
                },
                {
                  "name": "similarity",
                  "value": "98"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:2443"
            },
            "name": "I compare \"SOME IMAGE\" image with a \"98\" similarity with \"spring\"",
            "result": {
              "duration": 0.0009975433349609375,
              "status": "passed"
            },
            "step_type": "then"
          }
        ],
        "tags": [
          "SOME-TAG"
        ],
        "type": "scenario"
      },
      {
        "keyword": "Scenario Outline",
        "location": "something_features/features/something_web.feature:75",
        "name": "Verify the UI content of SOME page -- @2.2 ",
        "status": "passed",
        "steps": [
          {
            "keyword": "Given",
            "location": "something_features/features/something_web.feature:60",
            "match": {
              "arguments": [
                {
                  "name": "page",
                  "value": "SOME"
                },
                {
                  "name": "affiliation",
                  "value": "DEFAULT_USER"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:70"
            },
            "name": "I access to SOME in \"PAGE\" as \"DEFAULT USER\"",
            "result": {
              "duration": 15.28377103805542,
              "status": "passed"
            },
            "step_type": "given"
          },
          {
            "keyword": "When",
            "location": "something_features/features/something_web.feature:61",
            "match": {
              "arguments": [
                {
                  "name": "toggle_type",
                  "value": "default"
                },
                {
                  "name": "refresh_condition",
                  "value": "with"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:362"
            },
            "name": "I set \"default\" toggle \"with\" page refresh",
            "result": {
              "duration": 2.4945812225341797,
              "status": "passed"
            },
            "step_type": "when"
          },
          {
            "keyword": "Then",
            "location": "something_features/features/something_web.feature:62",
            "match": {
              "arguments": [
                {
                  "name": "page_title",
                  "value": "SOME | PAGE"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:261"
            },
            "name": "I can see page displayed with the title \"SOME | PAGE\"",
            "result": {
              "duration": 5.220450162887573,
              "status": "passed"
            },
            "step_type": "then"
          },
          {
            "keyword": "And",
            "location": "something_features/features/something_web.feature:63",
            "match": {
              "arguments": [
                {
                  "name": "banner",
                  "value": "SOME IMAGE"
                },
                {
                  "name": "similarity",
                  "value": "98"
                },
                {
                  "name": "page",
                  "value": "SOME PAGE"
                }
              ],
              "location": "general_common_steps/common_something_steps.py:2443"
            },
            "name": "I compare \"SOME IMAGE\" image with a \"98\" similarity with \"default\"",
            "result": {
              "duration": 0.0029616355895996094,
              "status": "passed"
            },
            "step_type": "then"
          }
        ],
        "tags": [
          "SOME-TAG"
        ],
        "type": "scenario"
      }
    ],
    "keyword": "Feature",
    "location": "something_features/features/something_web.feature:1",
    "name": "SOME NAME",
    "status": "passed",
    "tags": []
  }
  ]

This is the info JSON I am passing:

{
    "fields": {
        "project": {
            "id": "the id of the project"
        },
        "summary": "Brand new Test execution",
        "issuetype": {
            "id": "10007"
        }
    }
}
petu
  • 13
  • 1
  • 3
  • Can you share the "Xray_behave_result.json"? From your description you are using the multipart endpoint, this endpoint is expecting two files, the results file and an information file where you can define some extra details. Are you passing both files? If yes please share the info file also. If you do not want to add extra details you should be using the https://docs.getxray.app/display/XRAY/Import+Execution+Results+-+REST#ImportExecutionResultsREST-BehaveJSONresults. – Cristiano Cunha Mar 24 '23 at 09:23
  • @CristianoCunha I added the info and the result JSONs to the question – petu Mar 24 '23 at 15:34

1 Answers1

0

Behave is not generating a json report compatible with Cucumber. Check the following tutorial that uses a custom reporter to generate a report that, after importation, will show details in each step: https://docs.getxray.app/display/XRAY/Testing+using+Behave+in+Python

Another possible solution is to use converters that will convert behave reports into compatible Cucumber reports.

Cristiano Cunha
  • 381
  • 1
  • 3