How to properly E2E test production | consumption of messages to Kafka with NestJS and Jest

Question

I want to be able to test from e2e the production and consumption of messages from a NestJS application to a broker. In my case I have put up a public repository here with the problem in total.

The application uses cp-all-in-one from confluent for the broker/schema registry and a NestJS application using the microservices library from NestJS to facilitate the production and consumption of messages.

My specific issue in my app.e2e.spec.ts is the the inability for me to spy on or confirm that messages where produced to the right topic and vice versa for consumption. I can see the messages in the topic in the confluent UI as well as in logs but I am unable to confirm by automated testing in Jest.

I have tried spying on the controller that has the consumer. Here is a snippet of the controller event handler.

  @EventPattern("queuing.restaurant.ordered_hotdogs")
  async handleOrderedHotdogs(@Payload() data: any): Promise<any> {
    // console.log(`ordered dogs event handler!!!!`);
    // console.log(`buffer off the topic`, data);

    const id = await registry.getLatestSchemaId(
      KafkaTopics.ORDERED_HOTDOGS_SUBJECT
    );

    const schema = await registry.getSchema(id);

    // console.log(`here is the schema for id:${id}`, schema);

    const decodedMessage = await registry.decode(data);

    // console.log(`decoded message`, decodedMessage);
    return decodedMessage;
  }

Here is the test case:

 test('Should produce message on GET / and then confirm production via consumption ', async  () => {

    // I want to be able to confirm that messages produced by getHello are
    // in fact produced to the right topic by 'spying on' the eventHandler
    // in the controller

    const consumerSpy = jest.spyOn(controller, "handleOrderedHotdogs");

    await request(app.getHttpServer())
      .get('/')
      .expect(200)
      .expect('Hello World!');


    expect(consumerSpy).toBeCalledTimes(1);

    const expectedMsg = {
      id: "1",
      name: "Hotdog",
      topping: "pickles",
    };

    expect(consumerSpy).lastReturnedWith(expectedMsg);

    // Here to allow jest to run long enough to facilitate consumption of messages
    return new Promise((Res) => {
      setTimeout(() => {
        console.log(`CONSUME DEBUG`);
        expect(true).toBeTruthy();
        Res(null);
      }, 5000);
    });

  });

Here is the output:

> resteraunt@0.0.1 test:e2e
> jest --config ./test/jest-e2e.json

 FAIL  test/app.e2e-spec.ts (14.123 s)
  AppController (e2e)
    ✕ Should produce message on GET / and then confirm production via consumption  (11335 ms)

  ● AppController (e2e) › Should produce message on GET / and then confirm production via consumption 

    expect(jest.fn()).toBeCalledTimes(expected)

    Expected number of calls: 1
    Received number of calls: 0

      55 |
      56 |
    > 57 |     expect(consumerSpy).toBeCalledTimes(1);
         |                         ^
      58 |
      59 |     const expectedMsg = {
      60 |       id: "1",

      at Object.<anonymous> (app.e2e-spec.ts:57:25)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        14.201 s