Bundle npm module 'cheerio' in K6 test

Question

I am trying to create some tests using K6 framework from LoadImpact, but I am struggelig with including external NPM module following the instructions on their documentation site.

On loadImpacts documentations site they include a detailed example on just what I am after, modules that enable me to parse xml from a soap service response. But, I am unable to get this working! Now, I am a total javascript newbie, but I have been coding for many years and would really like to solve this. The can be found here: https://docs.k6.io/docs/modules#section-npm-modules can anyone get this working? I need to run this on servers isolated from the Internet, so I am totaly dependent on creating the packages and transfer the required files.

According to the documentation a package is created like this

-- bundle `cheerio` npm module
git clone git@github.com:cheeriojs/cheerio.git
npm install browserify index.js -s cheerio > cheerio.js

My first question: In the folder I am residing when running this command a 'cheerio.js' file is created along with a a 'cheerio' folder and a 'node_modules' folder. the cheerio.js in my "root" directory only contains the following:

+ cheerio@0.22.0
+ index.js@0.0.3
+ browserify@16.2.3
updated 3 packages and audited 2829 packages in 2.221s
found 0 vulnerabilities

Back to LoadImpacts example on how to reference this package in a k6 javascript:

import cheerio from "./vendor/cheerio.js";
export default function() 
{
  const res = http.get("https://loadimpact.com/");
  const $ = cheerio.load(res.body);

What file is this, and where in the structure generated by browserify can I find it? I have tried to change this to point to 'index.js' in the 'cheerio' folder or cheerio.js found in 'cheerio/lib'. I will then receive a complaint about the first line in cheerio.js which defines a "parse" variable it cannot find: var parse = require("./parse'), if I change this to var parse = require("./parse.js') it goes on to complain about missing 'htmlparser2' which I can also find in this structure, but it seems like the entire dependency structure is not working.

Can anybody give me some guidance on how to create a browserify package with dependencies for cheerio and how/what I need to copy to my k6 project to make this work like on the loadImpact site.

Answer 1

The k6 docs for this definitely need some clarification, which I'll later do. The vendor folder currently mentioned there isn't something special, the docs are just missing a step to copy the cheerio.js and xml2js.js files that were generated by browserify to a new vendor folder in your k6 project.

For now, I'll try to offer a simplified explanation on how to achieve the same thing in a simpler way:

1. Create a new empty folder and go to it in a terminal
2. Run npm install browserify cheerio there (ignore the npm warnings about missing package.json or description)
3. Run ./node_modules/.bin/browserify ./node_modules/cheerio/ -s cheerio > cheerio.js in that folder
4. The resulting cheerio.js file in the folder root should be the file you import from the k6 script:

import http from "k6/http";
import cheerio from "./cheerio.js";

export default function () {
    const res = http.get("https://loadimpact.com/");

    const $ = cheerio.load(res.body);

    console.log($('head title').text())
}

That should be it for a single npm library.

And if you need to use multiple npm packages, it might be better to invest some time into bundling them in a single browserified .js file. For example, if you need both the cheerio and the xml2js libraries mentioned in the k6 docs, you can do something like this:

1. Create a new empty folder

2. Add something like the following package.json file in it:

   {
     "name": "k6-npm-libs-demo",
     "version": "0.0.1",
     "description": "just a simple demo of how to use multiple npm libs in k6",
     "main": "npm-main.js",
     "dependencies": {},
     "devDependencies": {
       "browserify": "*",
       "cheerio": "*",
       "xml2js": "*"
     },
     "scripts": {
       "install": "./node_modules/.bin/browserify npm-main.js -s npmlibs  > vendored-libs.js"
     },
     "author": "",
     "license": "ISC"
   }
   
   

   Of course, if you need different libraries than cheerio and xml2js, you need to adjust the devDependencies options.

3. Add an npm-main.js file like this (again, adjusting for the libraries you want):

   exports.xml2js = require('xml2js');
   exports.cheerio = require('cheerio');
   

4. Open that folder in a terminal and run npm install. That should result in the creation of a vendored-libs.js file in the root of the folder, which you can use in k6 like this:

   import http from "k6/http";
   import { cheerio, xml2js } from "./vendored-libs.js";
   
   export default function () {
       const res = http.get("https://loadimpact.com/");
   
       const $ = cheerio.load(res.body);
       console.log($('head title').text())
   
       var xmlString = '<?xml version="1.0" ?>' +
           '<items xmlns="http://foo.com">' +
           ' <item>Foo</item>' +
           ' <item color="green">Bar</item>' +
           '</items>'
   
       xml2js.parseString(xmlString, function (err, result) {
           console.log(JSON.stringify(result));
       });
   }