0

Hi I am working in a NestJS project using mongodb and mongoose.

I have to create transactions with a lot of promises inside, so i think it was a good idea to use Promise.all() inside my transaction for performace issues.

Unfortunately when i started working with my transactions i have a first issue, i was using session.startTransaction(); and my code was throwing the following error: Given transaction number 2 does not match any in-progress transactions. The active transaction number is 1, the error was thrown sometimes, not always but it was a problem

So i read the following question Mongoose `Promise.all()` Transaction Error, and i started to use withTransaction(), this solved the problem, but now mi code does not work propertly.

the code basically takes an array of bookings and then creates them, also needs to create combos of the bookings, what I need is that if a creation of a booking or a combo fail nothing should be inserted, for perfomance I use Promise.all().

But when i execute the function sometimes it creates more bookings than expected, if bookingsArray is from size 2, some times it creates 3 bookings and i just don't know why, this occurs very rarely but it is a big issue.

If i remove the Promise.all() from the transaction it works perfectly, but without Promise.all() the query is slow, so I wanted to know if there is any error in my code, or if you just cannot use Promise.all() inside a mongodb transaction in Nestjs

Main function with the transaction and Promise.all(), this one sometimes create the wrong number of bookings

async createMultipleBookings(
    userId: string,
    bookingsArray: CreateBookingDto[],
  ): Promise<void> {
    const session = await this.connection.startSession();
    await session.withTransaction(async () => {
      const promiseArray = [];
      for (let i = 0; i < bookingsArray.length; i++) {
        promiseArray.push(
          this.bookingRepository.createSingleBooking(
            userId,
            bookingsArray[i],
            session,
          ),
        );
      }
      promiseArray.push(
        this.bookingRepository.createCombosBookings(bookingsArray, session),
      );
      await Promise.all(promiseArray);
    });
    session.endSession();
  }

Main function with the transaction and withot Promise.all(), works fine but slow

  async createMultipleBookings(
    userId: string,
    bookingsArray: CreateBookingDto[],
  ): Promise<void> {
    const session = await this.connection.startSession();
    await session.withTransaction(async () => {
      for (let i = 0; i < bookingsArray.length; i++) {
        await this.bookingRepository.createSingleBooking(
          userId,
          bookingsArray[i],
          session,
        );
      }
      await this.bookingRepository.createCombosBookings(bookingsArray, session);
    });
    session.endSession();
  }

Functions called inside the main function

async createSingleBooking(
    userId: string,
    createBookingDto: CreateBookingDto,
    session: mongoose.ClientSession | null = null,
  ) {
    const product = await this.productsService.getProductById(
      createBookingDto.productId,
      session,
    );

    const user = await this.authService.getUserByIdcustomAttributes(
      userId,
      ['profile', 'name'],
      session,
    );

    const laboratory = await this.laboratoryService.getLaboratoryById(
      product.laboratoryId,
      session,
    );

    if (product.state !== State.published)
      throw new BadRequestException(
        `product ${createBookingDto.productId} is not published`,
      );

    const bookingTracking = this.createBookingTraking();

    const value = product.prices.find(
      (price) => price.user === user.profile.role,
    );
    const bookingPrice: Price = !value
      ? {
          user: user.profile.role,
          measure: Measure.valorACotizar,
          price: null,
        }
      : value;

    await new this.model({
      ...createBookingDto,
      userId,
      canceled: false,
      productType: product.productType,
      bookingTracking,
      bookingPrice,
      laboratoryId: product.laboratoryId,
      userName: user.name,
      productName: product.name,
      laboratoryName: laboratory.name,
      facultyName: laboratory.faculty,
      createdAt: new Date(),
    }).save({ session });

    await this.productsService.updateProductOutstanding(
      createBookingDto.productId,
      session,
    );
  } 
  async createCombosBookings(
    bookingsArray: CreateBookingDto[],
    session: mongoose.ClientSession,
  ): Promise<void> {
    const promiseArray = [];
    for (let i = 1; i < bookingsArray.length; i++) {
      promiseArray.push(
        this.combosService.createCombo(
          {
            productId1: bookingsArray[0].productId,
            productId2: bookingsArray[i].productId,
          },
          session,
        ),
      );
    }
    await Promise.all(promiseArray);
  } 

also this is how i create the connection element:

export class BookingService {
  constructor(
    @InjectModel(Booking.name) private readonly model: Model<BookingDocument>,
    private readonly authService: AuthService,
    private readonly bookingRepository: BookingRepository,
    @InjectConnection()
    private readonly connection: mongoose.Connection,
  ) {}
  • I believe you are trying to insert all the bookings with condition that if one of them fails, you want none to be inserted. Now, let's break your problem into parts. First, you want a transaction because if one of the insertions fails you want all others to fail too. Second, You want to insert in bulk, so you should use [insertMany](https://mongoosejs.com/docs/api.html#model_Model.insertMany). See [this question](https://stackoverflow.com/questions/58144439/why-insertmany-not-working-using-mongoos-with-transactions). – Devesh Mar 25 '22 at 17:50
  • Yes, you are right, i can optimize the create bookings part using insertmany, but still i need the transaction for the combos part, which yes i can optimize too with the insertmany, but still why my code does not work propertly? i mean yeah i need to use insertmany but its weird – Juan Pablo Betancourt Mar 25 '22 at 17:59
  • I am going to change my code so that it uses insertMany to avoid using promise.all() inside the transaction, i think this might solve the problem, but i have another question haha, if insertmany fails without a transaction this insert the data partially or insert nothing? i was reading the mongo docs and dont explain this – Juan Pablo Betancourt Mar 25 '22 at 19:22
  • "insertMany()" with the transaction will insert all or none. The question link in my previous comment will provide you exact demo. – Devesh Mar 26 '22 at 01:06

1 Answers1

0

I reworked my entire code to use insertMany() instead of using loops to do multiple single insertions and removed the promise.all inside the mongo transaction.

  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 10 '23 at 19:22