1

Lets say we scripted the scenarios following way for our evolving servers

Actual server v1 response 
response = { id: "1", name: "karate" }
Mocking client v1 schema 
schema = { id: "#string", name: "#string }
* match response == schema

Actual server v2 response
response = { id: "2", name: "karate", value: "is" }
Mocking client v2 schema
schema = { id: "#string", name: "#string, value: "#string" }
* match response == schema

Actual server v3 response
response = { id: "3", name: "karate", value: "is", description: "easy" }
Mocking client v3 schema
schema = { id: "#string", name: "#string", value: "#string", description: "#string" }
* match response == schema

Similarly for backward compatibility testing of our evolving servers, we script the scenarios following way

Actual server v3 response
response = { id: "3", name: "karate", value: "is", description: "easy" }
Mocking client v1 schema 
schema = { id: "#string", name: "#string }
* match response contains schema

Actual server v2 response
response = { id: "2", name: "karate", value: "is" }
Mocking client v1 schema 
schema = { id: "#string", name: "#string }
* match response contains schema

Actual server v1 response 
response = { id: "1", name: "karate" }
Mocking client v1 schema 
schema = { id: "#string", name: "#string }
* match response contains schema

Proposal is to be able to use some kind of flag in match statement that dynamically decides the kind of match we do during testing. Lets say that the name of flag is SOMEFLAG and we provide the kind of match we want to do during testing (set in karate-config.js for global effect)

var SOMEFLAG = "contains"; 
OR
var SOMEFLAG = "==";

Now in scenario we do following

# Depending on what is set in karate-config.js, it will use either contains or == for verification.
* match response SOMEFLAG schema

Is it possible to do this in karate ?

Also note that success of this idea really depends on https://github.com/intuit/karate/issues/826 due to ability match nested object using contains match.

Dixie
  • 81
  • 1
  • 1
  • 7

1 Answers1

0

Personally, I am strongly against this idea because it will make your tests less readable. It is a slippery slope once you start this. For an example of what happens when you attempt too much re-use (yes, re-use can be a bad thing in test automation, and I really don't care if you disagree :) - see this: https://stackoverflow.com/a/54126724/143475

What I would do is something like this:

* def lookup = 
"""
{
  dev: { id: "#string", name: "#string },
  stage: { id: "#string", name: "#string, value: "#string" },
  preprod: { id: "#string", name: "#string", value: "#string", description: "#string" }
}
"""
* def expected = lookup[karate.env]
* match response == expected

EDIT - I have a feeling that the change we made after this discussion will also solve your problem - or at least give you some new ideas: https://github.com/intuit/karate/issues/810

Peter Thomas
  • 54,465
  • 21
  • 84
  • 248
  • 1
    Take backward compatibility example where we match v3 server response with v1 client schema from above example Your 1st solution does not work because * def lookup = v1ClientSchema: { id: "#string", name: "#string } * def expected = lookup[karate.env] * match response == expected Your 2nd solution Looking at karate.filterKeys(), SCHEMA needs to be a superset, however in my case my response is acting as a superset. So this will not work either. I would say if karate provides a way to dynamically decide on METHOD, RESPONSE CODE then why not match statement ? – Dixie Jul 24 '19 at 19:30
  • I marked your answer however due to my low reputation of less than 15, I guess its not displayed publicly. On a side note, I thought my original questions was clear enough. – Dixie Jul 24 '19 at 20:18
  • @Dixie no because I cannot read your comment above, and please read this also - so try again if you need to: https://stackoverflow.com/help/how-to-ask – Peter Thomas Jul 24 '19 at 20:59