I have e2e tests for a javascript library, which works with the canvas element. It's organized this way: there is an html page with a list of links, the canvas, and an image element. When I click on a link the library draws a particular test case on the canvas and loads an image with the expected view. Also I have a node.js script to automate these actions with webdriverio, resemblejs (the library to compare images) and tape (the testing framework).
Now this quirky part (this example below works as expected, tests fail, if the image differs from the canvas, etc.), where I take all the links, compare the image and canvas with resemblejs, and store data (which is the result of resemblejs comparison in an array), then I run another loop where I call tape and test stuff:
const links = await browser.$$('#ul li a');
const testsData = [];
for (const link of links) {
const data = await compare(link);
const testName = await link.getText();
testsData.push([testName, data]);
}
for (const [testName, data] of testsData) {
tape(testName, t => {
t.equal(data.rawMisMatchPercentage < 1, true);
t.end();
});
}
await browser.deleteSession();
But if I try to use a single loop, like here:
const links = await browser.$$('#ul li a');
let counter = 1;
for (const link of links) {
console.log('LOOP START ' + counter);
const data = await compare(link);
const testName = await link.getText();
tape(testName, t => {
console.log('TEST START ' + counter);
t.equal(data.rawMisMatchPercentage < 1, true);
t.end();
console.log('TEST END ' + counter);
});
console.log('LOOP END ' + counter++);
}
await browser.deleteSession();
the script executes this way:
- LOOP START 1 // the actions on the page started, then the comparison was made and returned the result
- LOOP END 1 // tape was called, put the callback somewhere and returned
- LOOP START 2 // the actions on the page started (requests have been sent to webriver), which created spare time for the node.js script, so the callback from tape's first call has a chance to run
- TEST START 1
- TEST END 1
- LOOP END 2
- LOOP START 3 // for some reason the actions on the page don't allow the previous tests to run anymore, like it was on LOOP START 2
- LOOP END 3
- LOOP START 4
- LOOP END 4
- etc.
And all the test except test 1 fail with the message: not ok TEST_NUM test exited without ending. Apparently the node.js script exits without tape callbacks ever run (except the first one).
I understand, that tape doesn't run the test immediately, it accepts a callback, which is run later, but what puzzles me: how come, that the script exits, when there are still callbacks to be run?
Any hint is greatly appreciated
Currently, I'm suspecting, that this is a bug or something else with the tape library. This is what's been happening (what I was able to research): tape accepts callbacks, runs the first one, reports the results but then doesn't run the remaining callbacks, but reports them as "failed" even without an attempt to call them.
