Puppeteer - using querySelectorAll() to access elements in a dynamic HTML environment

Question

Searching the documentation for querySelectorAll() i got this:

A NodeList object, representing all elements in the document that matches the specified CSS selector(s). The NodeList is a static collection, meaning that changes in the DOM has NO effect in the collection. Throws a SYNTAX_ERR exception if the selector(s) is invalid.

What if you delete some elements. Then new elements appear (with the same class name as the old ones) due to dynamic html.

But now you want to access the new ones. Will i be able to rerun the querySelectorAll()? Or the old elements will be in the array?

score 1 · Answer 1 · answered Jun 14 '18 at 23:15

1

Events can only be put on to existing Elements, and don't exist when the Elements don't, so you have to assign Events to the Elements after you make them. A good solution to avoid the reassignment, is to use jQuery's .on(). Either that, or make a function that you just run again.

answered Jun 14 '18 at 23:15

StackSlave

10,613
2
18
35

I am using puppeteer so i don't think i am able to run jQuery. – user1584421 Jun 14 '18 at 23:20
Of course you're able to run jQuery: [inject jquery into puppeteer page](https://stackoverflow.com/questions/46987516/inject-jquery-into-puppeteer-page) – Vaviloff Jun 15 '18 at 06:42

score 1 · Accepted Answer · answered Jun 15 '18 at 06:40

You will of course be able to rerun querySelectorAll() and each time it will return the elements currently corresponding to the query — the important thing is you will have to rerun it to get new elements.

An example:

(async() => { 

    // ... usual create browser and page stuff

    var items = [];
    while(items = await page.$$eval('p', pp => pp.map( p => p.textContent ) ))
    {
        console.log(items);        
        await page.waitFor(1000);
    }
})()

page.$$eval runs Array.from(document.querySelectorAll(selector)) within the page and passes it as the first argument to pageFunction.

The result:

Puppeteer - using querySelectorAll() to access elements in a dynamic HTML environment

2 Answers2