I want to create Infinite loop in puppeteer-cluster

Question

I want create infinite loop in puppeteer-cluster

My Code

but After about 30 seconds, it caused idle What should I do?

Answer 1

There are probably timeout errors being thrown in your code. You have two options:

Option 1: Listen to the thrown errors

As you are not listing to taskerror events any errors thrown during the process will not be handled.

cluster.on('taskerror', (err, data) => {
      // handle error
});

Option 2: Increase timeout value

In case your task takes very long, you can increase the timeout value:

const cluster = await Cluster.launch({
    // ...
    timeout: 120000 // 2 minutes
});

---

In general, you should not create infinite loops inside the cluster.queue function. The queue handles that exact task for you. You queue your jobs and then the cluster.task function takes care of them. No need to create an infinite loop for tasks.

Answer 2

Thirty seconds is the default time for Timeouts in puppeteer so you may try to disable Timeouts using page.setDefaultNavigationTimeout(0) and page.setDefaultTimeout(0)

see : https://github.com/GoogleChrome/puppeteer/issues/1514
and : https://github.com/GoogleChrome/puppeteer/issues/2079