parallelizing the for loop in javascript

Question

I have a nested for loop with

for(let i=0; i<500; i++){
   //some work
   for(let j=0; j<100000; j++){ 
     //some work
  }
}

is there any way I can make this parallel or fast like a web worker, asynchronous?

I could not think of how I can make any of these work.

Each work inside the inner for loop is independent of than watch others.

I tried to parallelize the code using the Promise.all() but no fruitful

   for (let m = 0; m < keyCountMap.length; m += 2) {
        fetchWorker.postMessage([nodePages, keyCountMap]);
        // console.log(m, keyCountMap[m], keyCountMap.length);
        let myRoot = nodePages[keyCountMap[m]];
        const view = await Copc.loadPointDataView(filename, copc, myRoot);
        let getters = ["X", "Y", "Z", "Intensity"].map(view.getter);
        let chunkCount = 2;
        let totalCalled = 0;
        for (let j = 0; j < keyCountMap[m + 1]; j += chunkCount) {
          let remaining = keyCountMap[m + 1] - totalCalled;
          let localChunkCount = Math.min(chunkCount, remaining);
          totalCalled += localChunkCount;
          const pointTemp = new Array(localChunkCount).fill(null);
          const promises = pointTemp.map((element, index) =>
            readPoints(index + j, getters)
          );
          const done = await Promise.all(promises);
        }
      }

and

    const readPoints = (id, getters) => {
        return new Promise((resolve, reject) => {
          let returnPoint = getXyzi(id, getters);
          positions.push(
            returnPoint[0] - x_min - 0.5 * width,
            returnPoint[1] - y_min - 0.5 * width,
            returnPoint[2] - z_min - 0.5 * width
          );
          const vx = (returnPoint[3] / 65535) * 255;
          color.setRGB(vx, vx, vx);
          colors.push(color.r, color.g, color.b);
          return resolve(true);
        });
      };
    
      function getXyzi(index, getters) {
        return getters.map((get) => get(index));
      }

Almost certainly not. How many CPUs are available in your computer? — Pointy, Feb 01 '23 at 22:20
what about creating promises and resolving them parallel with `Promise.all` or `Promise.allSettled`? Can create a simple exmaple if you want :) — Sysix, Feb 01 '23 at 22:21
You saying web worker so I am guessing nodes child process would not be an option? What is the process inside the inner loop? Is it like an HTTP request something which can be resolved by a promise or is it something with a static run time? — Snake_py, Feb 01 '23 at 22:22
@Sysix thank you !! so i can make 100000 number of async call ? — Pravin Poudel, Feb 01 '23 at 22:23
@Snake_py I am getting a post-processed data using some other function which is inside the program so not HTTP — Pravin Poudel, Feb 01 '23 at 22:23
You can split up CPU-bound computation (there are libraries that wrap this up in various ways, it it can be done manually). Whether or not it makes sense to do so depends very much on the actual problem being solved. — Dave Newton, Feb 01 '23 at 22:29
@pravinpoudel its not sure how the browser will call each task and how many of them in parallel. But I think it's a simple start. else look at this thread for bit more details: https://stackoverflow.com/questions/65166535/multithreading-with-javascript-promises — Sysix, Feb 01 '23 at 22:40
JavaScript is single threaded by design. Is this in the browser or Node.js? For the latter you can create child processes, https://medium.com/@NorbertdeLangen/communicating-between-nodejs-processes-4e68be42b917 — Peter Thoeny, Feb 02 '23 at 02:00
@PeterThoeny i am in Front end but nothing related to DOM or DOM manipulation — Pravin Poudel, Feb 02 '23 at 03:02
I tried with async promise all but i am not getting speed up !! Any help please, https://stackoverflow.com/questions/75318854/async-promise-not-speeding-up-the-execution-time — Pravin Poudel, Feb 02 '23 at 05:02

score 0 · Answer 1 · answered Feb 02 '23 at 06:32

In single thread, you will only get performance boost for "parallelising" IO calls or network calls. In your case it is probably Copc.loadPointDataView, so the most important part is to make the body of the outer for-loop into promises. The inner loops don't matter so much as they do not contains IO/network calls so there is no "waiting time" to safe. But I'd demonstrate how to use Promise.all on all layers anyway.

let promises = [];

for (let m = 0; m < keyCountMap.length; m += 2) {
  promises.push((async () => {
    fetchWorker.postMessage([nodePages, keyCountMap]);
    // console.log(m, keyCountMap[m], keyCountMap.length);
    let myRoot = nodePages[keyCountMap[m]];
    const view = await Copc.loadPointDataView(filename, copc, myRoot);
    let getters = ["X", "Y", "Z", "Intensity"].map(view.getter);
    let chunkCount = 2;
    let totalCalled = 0;
    let innerPromises = [];
    for (let j = 0; j < keyCountMap[m + 1]; j += chunkCount) {
      innerPromises.push((async ()=>{
        let remaining = keyCountMap[m + 1] - totalCalled;
        let localChunkCount = Math.min(chunkCount, remaining);
        totalCalled += localChunkCount;
        const pointTemp = new Array(localChunkCount).fill(null);
        const readPointPromises = pointTemp.map((element, index) => readPoints(index + j, getters) );
        const done = await Promise.all(readPointPromises);
      })());
    }
    await Promise.all(innerPromises);
  })());
}

await Promise.all(promises);

Hi Ricky, thank you so much !! do i need to anything more on readPoint function? because still i am not getting any gain on changing chunkCount — Pravin Poudel, Feb 02 '23 at 06:51

parallelizing the for loop in javascript

1 Answers1