Testng - Skip dependent tests for only failed data sets

Question

I am attempting to modify my dependent tests so they are ran in a specific way and have yet find a way possible. For instance, say I have the following two tests and the defined data provider:

@Dataprovider(name = "apiResponses")
Public void queryApi(){
  return getApiResponses().entrySet().stream().map(response -> new Object[]{response.getKey(), response.getValue()}).toArray(Object[][]::new);
}

@Test(dataprovider = "apiResponses")
Public void validateApiResponse(Object apiRequest, Object apiResponse){
  if(apiResponse.statusCode != 200){
    Assert.fail("Api Response must be that of a 200 to continue testing");
  }
}

@Test(dataprovider = "apiResponses", dependsOnMethod="validateApiResponse")
Public void validateResponseContent(Object apiRequest, Object apiResponse){
  //The following method contains the necessary assertions for validating api repsonse content
  validateApiResponseData(apiResponse); 
}

Say I have 100 api requests I want to validate, with the above, if a single one of those 100 requests were to return a status code of anything other than 200, then validateResponseContent would be skipped for all 100. What I'm attempting to achieve is that the dependent tests would be skipped for only the api responses that were to return without a status code of 200 and for all tests to be ran for responses that returned WITH a status code of 200.

score 0 · Answer 1 · answered Feb 19 '18 at 03:02

0

Would'nt adding a if/else block solve this?

@Test(dataprovider = "apiResponses")
Public void validateApiResponse(Object apiRequest, Object apiResponse){
  if(apiResponse.statusCode != 200){
    Assert.fail("Api Response must be that of a 200 to continue testing");
  } else {
    validateApiResponseData(apiResponse);
  }
}

answered Feb 19 '18 at 03:02

nu1silva

149
1
9

While yes it would, but then I would be calling a set of multiple different tests from within another test. Not an ideal solution and it's what was the initial implementation. Not only that but makes test results harder to read. It is my fallback if I can't come up with a better solution though. – Maverick1872 Feb 21 '18 at 15:44

score 0 · Accepted Answer · answered Feb 19 '18 at 16:50

You should be using a TestNG Factory which creates instances with both the apiRequest and apiResponse in it for each instance. Now each instance would basically first run an assertion on the status code before it moves on to validating the actual api response.

Here's a sample that shows how this would look like:

public class TestClassSample {
    private Object apiRequest, apiResponse;

    @Factory(dataProvider = "apiResponses")
    public TestClassSample(Object apiRequest, Object apiResponse) {
        this.apiRequest = apiRequest;
        this.apiResponse = apiResponse;
    }

    @Test
    public void validateApiResponse() {
        Assert.assertEquals(apiResponse.statusCode, 200, "Api Response must be that of a 200 to continue testing");
    }

    @Test(dependsOnMethods = "validateApiResponse")
    public void validateResponseContent() {
        //The following method contains the necessary assertions for validating api repsonse content
        validateApiResponseData(apiResponse);
    }

    @DataProvider(name = "apiResponses")
    public static java.lang.Object[][] queryApi() {
        return getApiResponses().entrySet()
                .stream().map(
                        response -> new java.lang.Object[]{
                                response.getKey(), response.getValue()
                        })
                .toArray(Object[][]::new);
    }
}

Hmm this is interesting and I'm not super familiar with TestNG Factories. I'll have to dig into this and the documentation. Thanks for the suggestion! — Maverick1872, Feb 21 '18 at 15:56
Believe this to be the correct answer, have been unable to get it to work with my current inheritance tree though. Once I make some tweaks I will update again. — Maverick1872, Mar 09 '18 at 18:39
Final Update: This is the correct answer and works as expected. Instead of defining the Factory annotation on the Constructor of my class, it's just another method that instantiates the necessary test class. This was necessary just due to my inheritance tree and the order somethings were running in. Thanks Krishnan! — Maverick1872, Mar 13 '18 at 14:53

Testng - Skip dependent tests for only failed data sets

2 Answers2