Nested async await not working as expected

Question

I'm trying to add a table in my existing code for some data and for this i have setted up everything similar to what mentioned in the exceljs docs. Here is a code snippet of the condition where i'm trying to get the data i need in order to get the table. When i'm printing the modRows within the forEach, it's showing the data but when i'm printing it outside the loop it's getting blank. Is there any way or other workaround to this? Is it possible to have async within async?

const generateCatReports = async (mode = 'monthly', months = 1, toDate = false) => {
  if (!['monthly', 'weekly'].includes(mode)) {
    throw new Error(InvalidParamsError);
  }
 const sortedTRIds = obtainSortedCentreIds(studiesTRMap, centreNameIDMap);
  const modRows = [];
  sortedTRIds.forEach(async (trId) => {
    console.log("trid",trId);
    const statsParams = {
      catId: trId,
      createdAt,
    };
    const statsPipeline = studiesStatsPipeline(statsParams, capitalize(mode));
    const statsInfo = await StudyModel.aggregate(statsPipeline).allowDiskUse(true).exec();
    Object.entries(statsInfo[0]).slice(1).forEach(([key, val]) => {
      modRows.push([key, val]);
    });
    console.log("inside sortedTr loop>>>>>" ,modRows);
  });
  console.log("outside sortedTr loop>>>>>>>>",modRows);

}

Result:

trId 1cf1eb1324322bbebe
inside sortedTr loop>>>>>>>>>>
['TOTAL', 10]
['white', 5]
['black', 5]

trId 1cf1eb1324322bbebe
inside sortedTr loop>>>>>>>>>>
['TOTAL', 10]
['white', 5]
['black', 5]

trId 21e1eb21322bbebe
inside sortedTr loop>>>>>>>>>>
['TOTAL', 8]
['white', 6]
['black', 2]

outside sortedTr loop>>>>>>>>>>
[]

Do not use forEach with async/await. Use a for loop instead. — Naveen Chahar, Nov 18 '20 at 06:22

score 4 · Accepted Answer · edited Nov 18 '20 at 09:01

Whenever you see an async callback like .forEach(async (trId) => { the chance that something is wrong is pretty high.

The problem what you have is that async functions are actually promises and therefore they don't block the main thread. That means that your callback function gets queued in the job queue and will be executed in the future.

Its simplified the same like this:

let arr = []
setTimeout(() => {
   arr.push("hy")
})
console.log(arr)

arr will be simply empty,

However you can use an for ... of loop

const generateCatReports = async (
  mode = "monthly",
  months = 1,
  toDate = false
) => {
  if (!["monthly", "weekly"].includes(mode)) {
    throw new Error(InvalidParamsError);
  }
  const sortedTRIds = obtainSortedCentreIds(studiesTRMap, centreNameIDMap);
  const modRows = [];
  for (const trId of sortedTRIds) {
    const statsParams = {
      catId: trId,
      createdAt
    };
    const statsPipeline = studiesStatsPipeline(statsParams, capitalize(mode));
    const statsInfo = await StudyModel.aggregate(statsPipeline)
      .allowDiskUse(true)
      .exec();
    Object.entries(statsInfo[0])
      .slice(1)
      .forEach(([key, val]) => {
        modRows.push([key, val]);
      });
    console.log("inside sortedTr loop>>>>>", modRows);
  }
  console.log("outside sortedTr loop>>>>>>>>", modRows);
};

Here you have no callback that will be queued.

A better solution would be to use Promise.all()

const generateCatReports = async (
  mode = "monthly",
  months = 1,
  toDate = false
) => {
  if (!["monthly", "weekly"].includes(mode)) {
    throw new Error(InvalidParamsError);
  }
  const sortedTRIds = obtainSortedCentreIds(studiesTRMap, centreNameIDMap);
  const modRows = await Promise.all(
    sortedTRIds.flatMap(async (trId) => {
      const statsParams = {
        catId: trId,
        createdAt
      };
      const statsPipeline = studiesStatsPipeline(statsParams, capitalize(mode));
      const statsInfo = await StudyModel.aggregate(statsPipeline)
        .allowDiskUse(true)
        .exec();
      return Object.entries(statsInfo[0]).slice(1);
    })
  );

  console.log(modRows);
};

Okay, So i got the problem so far. But what's the difference between using a forEach and for in this case. I mean, at the fundamental basis of looping both are pretty much the same? — Ashish Bairwa, Nov 18 '20 at 06:35
yes if you dont use `async` infront of it then they are the same. If you use `async` infront of an function, no matter what function, it will be a promise and therefore the promise will be added to the job que instead of being executed right immedeatly. — bill.gates, Nov 18 '20 at 06:38
@Stark try it out. declare a function and declare an async function. then execute the both functions you will see that the normal function always executes first — bill.gates, Nov 18 '20 at 06:39
@Stark here is a resource about async functions: https://javascript.info/async-await — bill.gates, Nov 18 '20 at 06:40
Let me give it a try and see the difference. Any good references for promises and promises/async await within a loop @Ifaruki — Ashish Bairwa, Nov 18 '20 at 06:40
@Stark if you log an function you will see an pending promise in the console instead of the value. because they are promises the function will be thenable that means you can attach `.then` after your function like: `someFunction().then(result => console.log(result))` — bill.gates, Nov 18 '20 at 06:42
One last thing @Ifaruki, what about the errors been thrown by eslint like https://eslint.org/docs/rules/no-await-in-loop and the restricted syntax one due to use of for in — Ashish Bairwa, Nov 18 '20 at 08:44
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/224714/discussion-between-stark-and-ifaruki). — Ashish Bairwa, Nov 18 '20 at 08:47

Anarno · Answer 2 · 2020-11-18T11:20:59.637

2

The async/await cannot be used in forEach cycle. You can use it in a simple for cycle or a while, do ...

For example:

export async function asyncForEach(array, callback) {
  for (let index = 0; index < array.length; index++) {
    await callback(array[index], index, array);
  }
}

edited Nov 18 '20 at 11:20

answered Nov 18 '20 at 06:22

Anarno

1,470
9
18

Why? I mean, What's the difference? Both are loops I guess? Can you point some reference. – Ashish Bairwa Nov 18 '20 at 06:24
You can use this example, as discussed in this article https://codeburst.io/javascript-async-await-with-foreach-b6ba62bbf404 – Anarno Nov 18 '20 at 11:22

score 2 · Answer 3 · answered Nov 18 '20 at 06:37

This is why you should not use async/await in forEach loop,

forEach loop accepts a callback, and won't wait even if you pass an async function as callback. Your async callback will wait for any promises inside of it but forEach loop will keep on executing callback function for all the elements in the array.

instead use for...in loop, as expected, an await inside of for...in loop will halt the loop execution and only continue iterating when you're done awaiting on a promise.

Nested async await not working as expected

3 Answers3