8

so long story short I'm developing RESTapi that takes a movie title on POST request to the /movies route, fetches info about that movie from external api and saves that object to the database. On POST /comments you add a comment to the different collection but every comment has a 'movie_id' property that has an associated movie.

That's my first bigger project so I'm trying to write integrational tests. Everything is great, at least in my opinion, except 3 weird test cases that are failing just out of nowhere. Tests could pass 10 times in a row and then suddenly that weird 'jest' timer shows up and 3 cases fail.

I'm using native mongodb driver, express and jest with supertest for testing, dropping test-database BeforeAll and AfterEach, I have no idea whats the reason of that.

Timer thingy: enter image description here

And after timer this shows up, failed tests: enter image description here

Full source code is here GITHUB

Other failed cases: enter image description here enter image description here

Any ideas, tips?

skyboyer
  • 22,209
  • 7
  • 57
  • 64
lekterable
  • 889
  • 1
  • 9
  • 18
  • 1
    `Full source code is here` You should post the relevant code in your question rather than expect helpers to try to search through your application... – CertainPerformance Apr 20 '18 at 08:37
  • 1
    if I had any idea what the reason might be i would do that – lekterable Apr 20 '18 at 08:40
  • I'd wrap some sort of try/catch around your failing test assertions (expectations). Wack a break point on the catch part and start running your tests through. Once one fails, go and inspect the database, you'll then understand what part of your data is incorrect during that test execution. That should give you an idea of what might be causing the issue. You'll also know the tests that ran before it from the CLI output which might help. – Elliot Blackburn Apr 20 '18 at 09:05

2 Answers2

18

I was in the same hell "jest parallel tests" problem and i find a solution, maybe not the best but now jest run tests in "queue mode" so when i delete datas in beforeAll my next group of tests is ready to go with "fresh" new inserted datas.

--runInBand Alias: -i. Run all tests serially in the current process, rather than creating a > worker pool of child processes that run tests. This can be useful for debugging.

jest source

So in my config.json i have :

    "scripts": {
        "test": "set NODE_ENV=test&& jest ./tests --runInBand --detectOpenHandles --forceExit",
        "server": "set NODE_ENV=development&& nodemon app.js"
    }
nathan lebreton
  • 196
  • 1
  • 3
4

The code seems to return one entry, while the test expects zero. This looks very much like an issue with test independence: Your tests seem to depend on each other (through the database).

I would guess that one test creates the movie and then clears it again. When everything works fine, the second test does not find the movie. But, in some unforunate cases (bad timing, different execution order, ...), the database is not cleared fast enough and the second test finds the movie.

So, you should work hard on making your tests independent. This is not easy with integrated tests, especially when they involve a real database.

Maybe you can create smaller tests (Unit tests, micro tests, ...) to achieve independence. If this is not possible, the test could check it's precondition (database empty or whatever) and wait until it's fulfilled (or until a timeout happens).

Dropping the database in BeforeAll and AfterEach might not be enough, because Jest even runs your tests in parallel: Are tests inside one file run in parallel in Jest?

Also, dropping might not be a completely synchronous and atomic operation, especially when there is some caching in the DB driver. But I don't know mongodb and it's node.js integration well enough to judge that.

David Tanzer
  • 2,732
  • 18
  • 30
  • I believe they are independent tbh, I clear database after each but when I need a movie in test I create it in that test case, same thing with those which randomly fail – lekterable Apr 20 '18 at 08:45
  • See the update to my answer. I am almost certain that your tests are not independent, I just cannot help you figure out why exactly this is the case :( – David Tanzer Apr 20 '18 at 08:50
  • I don't think that the depending on the other tests is the case, I really tried to make them independent and I don't think I depend on objects created in other test cases. But the part with dropping database sounds more probable, ill have to take a look at that. – lekterable Apr 20 '18 at 08:53
  • Thank's for trying tho! That's the failing test you talked about with an empty array https://pastebin.com/LqPrTTeb – lekterable Apr 20 '18 at 08:56
  • As you can see, i first create that movie and after that im testing – lekterable Apr 20 '18 at 08:57