Nestjs bullmq worker processor don't works with jest

Question

Step to reproduce

Create a new nest project : nest new nest-app
Install dependencies : npm i @nestjs/bullmq ioredis
Start a redis instance on default port 6379
Create this 2 files

app.service.ts

import { Injectable } from '@nestjs/common';
import { Job, Queue, Worker } from 'bullmq';
import Redis, { RedisOptions } from 'ioredis';

@Injectable()
export class AppService {
  private queue: Queue;
  private worker: Worker;
  private redis: Redis;

  connect() {
    const redisConfig: RedisOptions = {
      host: 'localhost',
      port: 6379,
      autoResubscribe: false,
      lazyConnect: true,
      maxRetriesPerRequest: 0,
      reconnectOnError: null,
      enableOfflineQueue: true,
    };
    this.redis = new Redis(redisConfig);
    this.queue = new Queue('queue1', { connection: this.redis });
    this.worker = new Worker(
      'queue1',
      async (job) => {
        console.log('PROCESS')
        await this.wait({});
        console.log('DONE')
        return job;
      },
      { connection: this.redis },
    );
    // Replace by this line and it works
    // this.worker = new Worker('queue1', '', { connection: this.redis });
  }

  async disconnect(): Promise<void> {
    await this.queue.close();
    await this.worker.close();
    await this.redis.quit();
  }

  async wait({ seconds = 1 }: { seconds?: number }): Promise<void> {
    await new Promise((resolve) => setTimeout(resolve, seconds * 1000));
  }
}

app.service.spec.ts

import { Test, TestingModule } from '@nestjs/testing';
import { AppService } from './app.service';

describe('Queue Service', () => {
  let appService: AppService;

  beforeAll(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [AppService],
    }).compile();
    appService = module.get<AppService>(AppService);
    appService.connect();
  });

  afterAll(async () => {
    await appService.disconnect();
  });

  it('Should pass', async () => {
    expect(true).toBeTruthy();
  });
});

Actual result

The test is still running with

Jest did not exit one second after the test run has completed.

'This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.

Expected result

The test just succeed

Versions

node : 16.20.1

npm : 9.8.1

"dependencies": {
    "@nestjs/bullmq": "^10.0.1",
    "@nestjs/common": "^10.0.0",
    "@nestjs/core": "^10.0.0",
    "@nestjs/platform-express": "^10.0.0",
    "ioredis": "^5.3.2",
    "reflect-metadata": "^0.1.13",
    "rxjs": "^7.8.1"
  },
  "devDependencies": {
    "@nestjs/cli": "^10.0.0",
    "@nestjs/schematics": "^10.0.0",
    "@nestjs/testing": "^10.0.0",
    "@types/express": "^4.17.17",
    "@types/jest": "^29.5.2",
    "@types/node": "^20.3.1",
    "@types/supertest": "^2.0.12",
    "@typescript-eslint/eslint-plugin": "^5.59.11",
    "@typescript-eslint/parser": "^5.59.11",
    "eslint": "^8.42.0",
    "eslint-config-prettier": "^8.8.0",
    "eslint-plugin-prettier": "^4.2.1",
    "jest": "^29.5.0",
    "prettier": "^2.8.8",
    "source-map-support": "^0.5.21",
    "supertest": "^6.3.3",
    "ts-jest": "^29.1.0",
    "ts-loader": "^9.4.3",
    "ts-node": "^10.9.1",
    "tsconfig-paths": "^4.2.0",
    "typescript": "^5.1.3"
  },

Yaroslavm · Answer 1 · 2023-08-08T16:49:41.467

0

It looks like Jest process was shutdown before disconnect() function promise was resolved.

Try

afterAll(async () => {
  await appService.disconnect();
});

Also add --detectOpenHandles to jest execution in package.json

Example:

"test:e2e": "jest --detectOpenHandles --config ./test/jest-e2e.json"

edited Aug 08 '23 at 16:49

answered Aug 08 '23 at 14:19

Yaroslavm

1,762
2
7
15

just forgot it but there is still the issue – TheSmartMonkey Aug 08 '23 at 14:21
Still the same issue? – Yaroslavm Aug 08 '23 at 14:22
yes I've tested outside nestjs and it works but not with nest – TheSmartMonkey Aug 08 '23 at 14:23
I think, there also should be await before connect. I see there are also async operations – Yaroslavm Aug 08 '23 at 14:24
connect is not async, where you able to reproduce it ? – TheSmartMonkey Aug 08 '23 at 14:25
@TheSmartMonkey but new Worker callback is async. It is not called during it's init, right? – Yaroslavm Aug 08 '23 at 14:29
yes it's a processor it's a function that is only called when you create a new job from what I understand but I may be wrong – TheSmartMonkey Aug 08 '23 at 14:31
Can you just try also await for connect to exclude this possibility? – Yaroslavm Aug 08 '23 at 14:33
just tryed same issue :( – TheSmartMonkey Aug 08 '23 at 14:37
1

@TheSmartMonkey I've set up everything you described and test passed for me. Maybe redis is hung or smth like that? – Yaroslavm Aug 08 '23 at 16:32
@TheSmartMonkey this was slient warning in my case. Was able to solve it by adding `--detectOpenHandles` for jest execution in `package.json` – Yaroslavm Aug 08 '23 at 16:46
It didn't solve the issue in my case – TheSmartMonkey Aug 08 '23 at 17:06
@TheSmartMonkey has it pointed to an issue? Or just failed? Also, can you kill redis and launch again? – Yaroslavm Aug 08 '23 at 17:07
If I had --detectOpenHandles it only removes the warning but the test is still stuck with an async process – TheSmartMonkey Aug 08 '23 at 21:13
So, I set up your infra in package.json, and used you npm / node versions and still test passes. So I have 2 last conclusions 1. After detectOpenHandles try `--testTimeout=25000` 2. You have another tests in you package and one of them has unresolved promise. But this test runs last in queue, so fails this one and not the one with unresolved promise. – Yaroslavm Aug 09 '23 at 08:13

Nestjs bullmq worker processor don't works with jest

Step to reproduce

Actual result

Expected result

Versions

1 Answers1