puppeteer page.evaluate() randomly fails with: Execution context was destroyed, most likely because of a navigation

Question

The following code randomly fails with:

Execution context was destroyed, most likely because of navigation.

Why is that? Any workarounds?

I use puppeteer version 1.19.0

Note: I'm looking for a general solution that allows navigating to both pages that have redirects, and pages that don't.

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();

  try {
    const page = await browser.newPage();

    await page.setCacheEnabled(false);

    const response = await page.goto("https://docs.cypress.io/", {
      waitUntil: "networkidle0",
      timeout: 60000
    });

    const pageUrls = await page.evaluate(() => {
      const links = Array.from(document.querySelectorAll("a"));

      return links.map(link => link.href);
    });

    console.log({ pageUrls });
  } catch (error) {
    console.log(error.message);
  }

  await browser.close();
})();

Rahul L · Accepted Answer · 2019-08-27T03:52:18.083

As a Workarounds - Add below code

await page.waitForNavigation()

after calling page.goto() Or if same issue occurs for page.click() you can wait for navigation using above method.

const puppeteer = require("puppeteer");

(async () => {
  const browser = await puppeteer.launch();

  try {
    const page = await browser.newPage();

    await page.setCacheEnabled(false);

    const response = await page.goto("https://docs.cypress.io/", {
      waitUntil: "networkidle0",
      timeout: 60000
    });

    await page.waitForNavigation();

    const pageUrls = await page.evaluate(() => {
      const links = Array.from(document.querySelectorAll("a"));

      return links.map(link => link.href);
    });

    console.log({ pageUrls });
  } catch (error) {
    console.log(error.message);
  }

  await browser.close();
})();

Output :

{ pageUrls:
   [ 'https://twitter.com/amirrustam',
     'https://www.componentsconf.com.au/workshops',
     'https://www.cypress.io/',
     'https://docs.cypress.io/guides/overview/why-cypress.html',
     'https://docs.cypress.io/api/api/table-of-contents.html',
     'https://docs.cypress.io/plugins/',
     'https://docs.cypress.io/examples/examples/recipes.html',
     'https://docs.cypress.io/faq/questions/using-cypress-faq.html',.................]

Edit

const puppeteer = require("puppeteer");

(async () => {
    const browser = await puppeteer.launch();
    try {
        const page = await browser.newPage();
        await page.setCacheEnabled(false);

        await Promise.all([
            page.waitForNavigation({ timeout: 60000 }),
            page.goto("https://www.google.com/", {
                waitUntil: "networkidle0",
                timeout: 60000
            })
        ])

        const pageUrls = await page.evaluate(() => {
            const links = Array.from(document.querySelectorAll("a"));
            return links.map(link => link.href);
        });
        console.log({ pageUrls });
    } catch (error) {
        console.log(error.message);
    }
    await browser.close();
})();

Output:

{ pageUrls:
   [ 'https://mail.google.com/mail/?tab=wm&ogbl',
     'https://www.google.co.in/imghp?hl=en&tab=wi&ogbl',
     'https://www.google.co.in/intl/en/about/products?tab=wh',
     'https://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=https://www.google.com/',
     'https://www.google.com/#',............

The problem with `await page.waitForNavigation()` is that you need to know in advance that there is a redirect to another page. If you try a different URL that doesn't redirect (e.g. https://google.com), `waitForNavigation` eventually times out. I'm looking for a general solution that follows redirects if they exist, but doesn't time out on pages that don't have redirects. — Misha Moroshko, Aug 26 '19 at 10:57
You can refer to https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagewaitfornavigationoptions You can increase the timeout . You can try in case of clicks . `await Promise.all([ page.waitForNavigation(), // The promise resolves after navigation has finished page.click('a.my-link'), // Clicking the link will indirectly cause a navigation ]);` — Rahul L, Aug 26 '19 at 12:06
If you are observing time out in waitfornavigation then it may be because of race condition https://github.com/GoogleChrome/puppeteer/issues/3338#issuecomment-426584706 — Rahul L, Aug 26 '19 at 14:34
@RahulL , I used the same code, but there was an error， TypeError: Cannot read property 'request' of undefined. — lyerox, Feb 25 '21 at 09:14

puppeteer page.evaluate() randomly fails with: Execution context was destroyed, most likely because of a navigation

1 Answers1