1

How do you run multiple concurrent executions with puppeteer-cluster?

I have 5 as max concurrency, but doing await cluster.execute() only runs 1 at a time.

const cluster = await Cluster.launch({
   monitor: false,
   concurrency: Cluster.CONCURRENCY_BROWSER,
   maxConcurrency: 5,
})

for(let url of urls) {
   await cluster.execute(url)
}

I want all 5 to be executed concurrent. The alternative is await cluster.queue(), but after a while, the memory is being eaten up, because there is no way to check for current queue length/size.

user2912391
  • 67
  • 8

1 Answers1

0

change concurrency: Cluster.CONCURRENCY_BROWSER to concurrency: Cluster.CONCURRENCY_CONTEXT

Ahmed Belghith
  • 1
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 16 '22 at 00:04