Reporters for Flutter integration testing

Question

I am looking for examples of integrations among Flutter Integration library and test management tools like TestRail or Xray. For Cypress and other browser automation tools I can find plenty of solutions but for Flutter integration basically nothing.

Answer 1

For Xray, there's no tutorial yet. However, it should be fairly simple.

Run your tests using flutter test and pass the --machine argument to generate a JSON report. Use the junitreport package to generate a JUnit XML report. Then you can add it to your command as such:

flutter test --machine | tojunit -o junit.xml

If you redirect this to a file, then you have the report that you can submit to Xray for example. For Xray server/datacenter, you need to call the REST API endpoint, something like:

curl -H "Content-Type: multipart/form-data" -u username:password -F "file=@junit.xml" http://yourserver/rest/raven/1.0/import/execution/junit?projectKey=JIRAPROJECTKEY

For Xray on Jira Cloud, the syntax is slightly different. You have to make an authentication request first, and then do another submiting the JUnit XML report.

token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "32A27E69B0AC4E5000000...","client_secret": "d62f81eb9ed859e1....." }'  https://xray.cloud.getxray.app/api/v2/authenticate)
curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token"  --data @"junit.xml" https://xray.cloud.getxray.app/api/v2/import/execution/junit?projectKey=XTP

Answer 2

For Report in flutter integration testing you can use 3rd party tools like TESTRAIL.

I created Package to easily integrate TESTRAIL in flutter: flutter_testrail

For E.g Check this Flutter_TestRail Example

Now once you created your TestRail Account then configure in our test setup:

    TestRail.configure(
      username: 'USERNAME',
      password: 'PASSWORD',
      /// The url that points to the test rail server => https://example.testrail.com
      serverDomain: 'https://YOUR_SERVER.testrail.com',
    )

Define your initial test status in test file:

 TestStatus testStatusResults= TestStatus(
          caseId: 1758,
          statusId: 3,
          comment: 'Verify that user is able to see the splash screen',
        ),
        

And then update your test status on TESTRAIL:

  /// Equivalent Status Code:
  /// 1: Passed
  /// 2: Blocked
  /// 3: Untested (not allowed when adding a new result)
  /// 4: Retest
  /// 5: Failed
  ///
  ///  So Pass Status Code according to your test status
  static Future<void> reportMultipleTestCaseResults(
    List<TestStatus> testStatusResults,
  ) async {
    final testRun = await TestRun.get(3); //replace 3 with your own Run Id.
    await testRun.addResultsForCases(
      testStatusResults,
    );
  }