Jmeter - How to handle duplicate values captured in RegExpression from the same response

Question

How to ensure/handle duplicate values are not getting captured while using regular expression?

In my scenario i need to capture multiple offers from my response. But cant use the same offers again and again within the transaction

Answer 1

There are multiple ways to do this. If you are using the default Regular Expression Extractor, the problem is it creates the variables (eg: offer_1, offer_2 etc), ready to go with the execution. It would've been easier if it returned a ArrayList of some sort where we can remove the duplicates. What I am about to suggest is to add those variables to a list in a JSR223(groovy) sampler/post processor then convert them back to usual jmeter variables to use in the usual jmeter script flow.

Snippet:

I created a sample script as per your description which would return multiple offers with some duplicates. Following is the state of jmeter variables before post processing.

offer_1=RUSSIA
offer_1_g=1
offer_1_g0=offer="RUSSIA"
offer_1_g1=RUSSIA
offer_2=UK
offer_2_g=1
offer_2_g0=offer="UK"
offer_2_g1=UK
offer_3=ICELAND
offer_3_g=1
offer_3_g0=offer="ICELAND"
offer_3_g1=ICELAND
offer_4=USA
offer_4_g=1
offer_4_g0=offer="USA"
offer_4_g1=USA
offer_5=UK
offer_5_g=1
offer_5_g0=offer="UK"
offer_5_g1=UK
offer_6=USA
offer_6_g=1
offer_6_g0=offer="USA"
offer_6_g1=USA
offer_7=USA
offer_7_g=1
offer_7_g0=offer="USA"
offer_7_g1=USA
offer_matchNr=7

As you can see above there are duplicates in the variables. Put the following Groovy code in a JSR223 post processor with groovy as language selected.

// Count of offers extracted by Regular Expression Extractor
def count = Integer.parseInt(vars.get("offer_matchNr"))
// An empty list which will store the offers
def offer_list = []

for (int i = 1; i <= count; i++){
    def offer = vars.get("offer_" + i)
    offer_list.add(offer)
}

// Removes the duplicates in the list
offer_list.unique()

// Following one liner adds new variables but with only unique offers in similar format as jmeter variable.
offer_list.eachWithIndex{ it, index -> vars.put("unique_offer_${index+1}", "${it}")}

After post processing:

unique_offer_1=RUSSIA
unique_offer_2=UK
unique_offer_3=ICELAND
unique_offer_4=USA