Errors with page.$$eval() using Puppeteer

Question

I'm trying to use page.$$eval() to get data from an <ul>, store the values in two arrays and then iterate through the arrays to put the data into key/value pairs. I'm struggling with getting the data into the arrays.

The HTML looks like this:

<ul class="b-profile__sections">
    <li class="b-profile__sections__item">
        <span class="b-profile__sections__link">
            <span class="b-profile__sections__count">585</span>
            <span class="b-profile__sections__name"> Posts </span>
        </span>
    </li>
    <li class="b-profile__sections__item">
        <span class="b-profile__sections__link m-no-hover">
            <span class="b-profile__sections__count">110.3K</span>
            <span class="b-profile__sections__name"> Likes </span>
        </span>
    </li>
</ul>

My code:

let user = {};
let properties = [];
let values = [];
await page.$$eval('.b-profile__sections__name', (elems, properties) => elems.forEach(elem => {
    properties.push(elem.innerText); // push result into array
}), properties) // array passed as argument to page.$$eval()
await page.$$eval('.b-profile__sections__count', (elems, values) => elems.forEach(elem => {
    values.push(elem.innerText); // push result into array
}), values) // array passed as argument to page.$$eval()
for (let i = 0; i < properties.length; i++) {
    user[properties[i]] = values[i] //iterate through arrays to make key/value pairs
}

Error:

Error: Evaluation failed: ReferenceError: values is not defined
    at __puppeteer_evaluation_script__:2:37
    at Array.forEach (<anonymous>)
    at __puppeteer_evaluation_script__:1:19

or:

Error: Evaluation failed: TypeError: Cannot read property 'push' of undefined
    at __puppeteer_evaluation_script__:2:44
    at Array.forEach (<anonymous>)
    at __puppeteer_evaluation_script__:1:27

I'm guessing it's to do with the browser (function passed to page.$$eval())/Node.js (the two arrays) context. Is there a better way?

Sorry, Smithfield, but while your question is well worded and nicely presented it gives just too much information. Could you single out the absolute minimal piece of code that is trouble and show that instead? — Vaviloff, Dec 20 '20 at 17:32
BTW, `values` will be copied by serialization/deserialization while transferring from the Node.js context to the browser context, so `values.push(elem.innerText)` will not update the array in the Node.js context. (I cannot see how the code in the fragment can produce the mentioned error though.) — vsemozhebuty, Dec 20 '20 at 20:49
@Vaviloff thank you for the feedback; I've tried to simplify the problem. — , Dec 21 '20 at 11:10

score 0 · Answer 1 · answered Dec 23 '20 at 11:50

let properties = await page.$$eval('.b-profile__sections__name', elems => elems.map(elem => elem.innerText));

let values = await page.$$eval('.b-profile__sections__count', elems => elems.map(elem => elem.innerText));

for (let i = 0; i < properties.length; i++) {
 user[properties[i]] = values[i]
}

Errors with page.$$eval() using Puppeteer

1 Answers1