puppeteer wait for page update after button click (no navigation)

Question

Trying puppeteer for the first time and it is amazing. I need to wait for a page to fetch and render new data after a button click. This does not cause a navigation (the url is the same) so the following code does not work (it gives timeout exception):

await Promise.all([
    myButton.click()
    page.waitForNavigation()
])

What's the correct way to wait for the page to fetch/render async data on click?

Tom Faltesek · Answer 1 · 2019-08-23T20:51:07.320

Assuming the DOM changes in some way, you can wait for a specific element or selector.

Maybe an image appears.

await myButton.click();
await page.waitForSelector('img.success');

Maybe some element with an ID attribute is inserted into the DOM.

await myButton.click();
await page.waitForSelector('#newElementThatAppeared');

If you're unfamiliar with DOM selectors, you can read up here and here. They're powerful and easy to use.

Update - Custom wait predicate.

If we always know the length...

await myButton.click();
await page.waitFor(() => document.querySelectorAll('ul.specialList li').length > 5);

If we know the length will increase

const listSize = await page.evaluate(() => document.querySelectorAll('ul.specialList li').length);
await myButton.click();
await page.waitFor(() => document.querySelectorAll('ul.specialList li').length > listSize);

Hi! The data fetched will update an already non empty list of
Hmm. Could you update the question with a relevant snippet of the **before click** state of the DOM and the **after click** state? I may be able to help a bit more with those details. — Tom Faltesek, Aug 23 '19 at 15:52

storenth · Answer 2 · 2019-08-23T15:59:03.283

1

First of all await Promise.all some kind of concurrency and if you need click and then wait split this with

await page.click('#selector');
const finalResponse = await page.waitForResponse(response => response.url() === 'https://example.com' && response.status() === 200);
return finalResponse.ok();

And be noted:

This resolves when the page navigates to a new URL or reloads.

edited Aug 23 '19 at 15:59

answered Aug 23 '19 at 15:51

storenth

967
11
18

@Faltesek I fixed my typo, please remove your minus. – storenth Aug 23 '19 at 16:00
@Tom Faltesek Why your -1 needed under my answer? I try to help you try to troll? – storenth Aug 23 '19 at 17:08
1

Not trying to troll you. Just pointing out that the OP stated that the navigation and URL does **NOT** change. So your solution won't be helpful. – Tom Faltesek Aug 23 '19 at 20:37
Answer is perfect. – storenth May 23 '22 at 09:08

score 0 · Answer 3 · answered Aug 23 '19 at 20:15

0

Well, kind of an hack but didn't find any better solution for now. Basically I am comparing the number of specific li elements before and after the button click to verify when new data has been loaded.

const wait = time => new Promise(resolve => setTimeout(resolve, time))

let data = await page.evaluate(_ => document.querySelectorAll("li.className").length) 

let isDataLoaded = false
await page.click("#myButton")

while(!isDataLoaded) {
    if (data.length !== await.page.evaluate(_ => document.querySelectorAll("li.className").length)) {
        isDataLoaded = true
    }
    await wait(100)
}

// new data should be loaded and rendered now in the page
// [...]

answered Aug 23 '19 at 20:15

revy

3,945
7
40
85

Instead of that while loop, could you just use the `waitFor` like in my last example? – Tom Faltesek Aug 23 '19 at 20:34
`await page.waitFor(() => document.querySelectorAll('li.className').length != listSize);` – Tom Faltesek Aug 23 '19 at 20:35
I think you have used an incorrect syntax: await page.waitFor(listSize => document.querySelectorAll('li.className').length > listSize, {}, listSize) this works, but from my tests this is slower respect to the while solution – revy Aug 23 '19 at 21:35
I called my variable `listSize` in my example. You called it `data`. Is that the issue? – Tom Faltesek Aug 23 '19 at 21:37
await page.waitFor(() => document.querySelectorAll('li.className').length !== data); – Tom Faltesek Aug 23 '19 at 21:38
Even better, specifying the polling time between evaluation of the predicate, in milliseconds: `await page.waitFor(listSize => document.querySelectorAll('li.className').length > listSize, {polling: 100}, listSize)` This works basically the same as the while solution and probably is the way to go. Thanks! – revy Aug 23 '19 at 21:42

puppeteer wait for page update after button click (no navigation)

3 Answers3

Linked