Thresholding the deviation in `http_req_duration` with k6?

Question

I want to create a custom metric with k6 for load testing of a service. I want to measure the deviation between the http_req_duration between the old version (before the code change), and the new version (with the code change), and put a threshold on the difference. Is it possible to do in k6? If yes, how should I do it?

Answer 1

Using this code below:

import http from "k6/http";
import {Trend} from "k6/metrics";

var diffT = new Trend("DiffTrend", true); // this true is just so in the summary it shows them as times

var oldURL = "https://test.loadimpact.com";
var newURL = "https://test.k6.io";

export default function() {
    // you can use http.get, but this way the two requests will be done at the same time which will save time and probably be a better comparison
    var res = http.batch([
        ["GET", oldURL, null, {"tags": {"varaint": "old"}}],
        ["GET", newURL, null, {"tags": {"varaint": "new"}}],
    ])
    diffT.add(res[0].timings.duration - res[1].timings.duration);
}

You can use this in two different ways:

1. use that diffT metrics which is probably the simplest, but I would argue won't be good for anything but the simplest cases, as it handles only request to request differences, not the overall picture ...But you are going to diff 10 requests to the same API this is probably going to be good enough.

2. You can use the tag "variant" to distinguish and compare whichever metric you want for each and all of the requests between the old and the new. This is IMO the better approach, using Grafana this will be a lot more powerful and will give you a much better picture.