0

I would like to know if it is possible to implement an event with a maximum number of operations at a time. Example:

const myEvent = EventEmitter();
myEvent.on("calculate", (file) => {
  // async operation here
  // calls data event passing the data when finished. -> myEvent.emit("data", data);
});

arrayOfFiles.forEach((file), () => {
  myEvent.emit("calculate", file);
});

myEvent.on("data", (data) => {
  // do something here.
});

I want to allow only 500 files at a time. So I did this with the async.eachLimit:

const myEvent = EventEmitter();
myEvent.on("calculate", (file) => {
  // async operation here
  // calls data event passing the data when finished. -> myEvent.emit("data", data);
});

async.eachLimit(arrayOfFiles, 500, (file, callback) => {
  myEvent.emit("calculate", file);
  myEvent.on("data", (data) => {
    callback();
  });
}, (error) => {
  if (error) return console.error(error);
  console.log("done!");
});

Which works fine, but maybe there is a better way to do this without using the async.eachLimit.

Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • Better than a method for exactly this use case? – Andreas Mar 27 '17 at 17:07
  • What do you mean by "allow"? To whom? What should happen if it is still called faster than you want? – Bergi Mar 27 '17 at 17:43
  • I'm spawning child processes... if i allow faster, i may cause errors like opening to many files, memory exceptions, etc –  Mar 27 '17 at 17:52
  • Why are you using an event emitter at all? Directly plugging into `async.eachLimit` looks much easier – Bergi Mar 27 '17 at 20:54
  • I'm constructing yet, but i'm going to include other events. The whole idea is to download a bunch of files of s3, extract the text of each one and extract data... In the currently situation i'm doing async-sync operations... first of all i download everything, then i extract the text and then i extract the data.... i want to do everything asynchronous, but still limit the number of async operations at a time (with eachLimit).. So, the api will download the file => extract the text => extract the data => waits until the last one arrivies. –  Mar 27 '17 at 21:02

0 Answers0