Is there a way to run QAF BDD test on AWS device farm?

Question

I'm trying to execute QAF BDD test using AWS device farm (Appium test using TestNG) but unfortunately it wont trigger the execution on testcase even there is no error. The same working fine while execute it locally.

The testng.xml configurations:

<suite name="QAF Demo" verbose="0">
    <test name="Launch_App" enabled="true">
        <groups>
            <run>
                <include name="SMOKE"/>
            </run>
        </groups>
        <classes>
            <class name="com.qmetry.qaf.automation.step.client.text.BDDTestFactory"/>
        </classes>
    </test>
</suite>

BDD file:

SCENARIO: SampleTest
META-DATA: {"description":"Sample Test Scenario","groups":["SMOKE"]}

    Given print message

END

The log from AWS :

Start Appium TestNG test
[DeviceFarm] java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG -testjar *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output -verbose 10
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[TestRunner] Running the tests in 'Launch_App' with parallel mode:false
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@7cf10a6f priority: 10
include groups [SMOKE]
 exclude groups: [] Scanarios location:  
[TestClass] Creating TestClass for [ClassImpl class=com.qmetry.qaf.automation.step.client.text.BDDTestFactory]
[TestNG] Running:
  /tmp/scratchE1xraV.scratch/test-packagel0A01c/testng.xml

[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test Launch_App on 1  classes,  included groups:[SMOKE ] excluded groups:[]
===== Test class
com.qmetry.qaf.automation.step.client.text.BDDTestFactory
======
[TestNG] RUNNING: Suite: "Launch_App" containing "0" Tests (config: /tmp/scratchE1xraV.scratch/test-packagel0A01c/testng.xml)
===== Invoked methods
=====
Creating /tmp/customer_log_directoryz6SHdY/test-output/QAF Demo/Launch_App.html
Creating /tmp/customer_log_directoryz6SHdY/test-output/QAF Demo/Launch_App.xml

===============================================
    Launch_App
    Tests run: 0, Failures: 0, Skips: 0
===============================================

[TestNG] 
[TestNG] ===============================================
[TestNG]     Launch_App
[TestNG]     Tests run: 0, Failures: 0, Skips: 0
[TestNG] ===============================================

===============================================
QAF Demo
Total tests run: 0, Failures: 0, Skips: 0
===============================================

Is it due to custom method selector used by QAF?

it looks that it didn't found any bdd file. Did you packaged resources and scenarios directory in zip? When extracted it need to be available in execution directory. — user861594, May 07 '20 at 17:11
@user861594, I packaged in zip in same way as mentioned [here](https://docs.aws.amazon.com/devicefarm/latest/developerguide/test-types-android-appium-java-testng.html). additionally i mentioned ` ${project.basedir}/config ` in pom.xml so it can find testng.xml — NarendraR, May 07 '20 at 17:30
Is there anything else i need to do to package resources and scenarios ? — NarendraR, May 07 '20 at 17:34
Have you already tried running these tests with the [custom test environment](https://docs.aws.amazon.com/devicefarm/latest/developerguide/custom-test-environments.html)? It seems more well-suited for running tests. If you think its something specific to AWS Device Farm, you can try posting the URL of the run where you are experiencing this issue on the AWS Device Farm [forums](https://forums.aws.amazon.com/forum.jspa?forumID=193&start=0) for help from them. — Tobe E, May 08 '20 at 22:29

score 3 · Answer 1 · answered Jun 03 '20 at 06:29

To integrate QAF with AWS Device Farm you can follow below steps.

As per AWS mentioned do required configuration in your project. Refer working with Appium on AWS Device Farm Documentation.
Add below folders in your assembly zip files. You can add these folders by updating your assembly/zip.xml.

<fileSet>
    <directory>${project.basedir}</directory>
    <outputDirectory>./</outputDirectory>
    <includes>
      <include>/resources/</include>
      <include>/scenarios/</include>
      <include>/config/</include>
    </includes>
</fileSet>

Now build zip using mvn clean package -DskipTests=true command.
Follow the same reference link to upload your test package zip.
Provide path of testng xml in your configuration yml file.

java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG "config/testrun_config.xml" -d $DEVICEFARM_LOG_DIR/test-output -verbose 10

Move QAF Results to build artifacts folder to fetch QAF result.

post_test:
    commands:
      - cd $DEVICEFARM_TEST_PACKAGE_PATH
      - cp -R test-results $DEVICEFARM_LOG_DIR
      - cp -R img $DEVICEFARM_LOG_DIR
      - cp -R img $DEVICEFARM_SCREENSHOT_PATH

Is there a way to run QAF BDD test on AWS device farm?

1 Answers1