Puppeteer Getting List of Elements with Same Selector

Question

Background:

Using NodeJS/CucumberJS/Puppeteer to build end-to-end regression test for an emberJS solution.

Problem:

Selecting (page.click) and getting textContent of one of the elements when there are several dynamic elements with the same selector? (In my case, I have 4 elements with the same selector = [data-test-foo4="true"])

I know, that with:

const text = await page.evaluate( () => document.querySelector('[data-test-foo4="true"]').textContent );

I can get the text of the first element, but how do I select the other elements with the same selector? I've tried:

var text = await page.evaluate( () => document.querySelectorAll('[data-test-foo4="true"]').textContent )[1];
console.log('text = ' + text);

but it gives me 'text = undefined'

Also, the following:

await page.click('[data-test-foo4="true"]');

selects the first elements with that selector, but how can I select the next one with that selector?

score 49 · Answer 1 · edited Jul 19 '22 at 09:14

49

You can use Array.from() to create an array containing all of the textContent values of each element matching your selector:

const text = await page.evaluate(() => Array.from(document.querySelectorAll('[data-test-foo4="true"]'), element => element.textContent));

console.log(text[0]);
console.log(text[1]);
console.log(text[2]);

If you need to click more than one element containing a given selector, you can create an ElementHandle array using page.$$() and click each one using elementHandle.click():

const example = await page.$$('[data-test-foo4="true"]');

await example[0].click();
await example[1].click();
await example[2].click();

edited Jul 19 '22 at 09:14

user4035

22,508
11
59
94

answered Sep 07 '18 at 17:54

Grant Miller

27,532
16
147
165

Thanks Grant, I tried: `await page.click('[data-test-button-sort-asc-desc-toggle-type-selection="true"]'); await page.waitFor(settings._3000); const text = await page.evaluate( () => Array.from( document.querySelectorAll('[data-test-button-sort-asc-desc-toggle-type-selection=="true"]' ), element => element.textContent ) );` – Huckleberry Carignan Sep 07 '18 at 19:18
- but after the first part ran `page.click` successfully, I then got a`'Error: Evaluation failed: DOMException: Failed to execute 'querySelectorAll' on 'Document': '[data-test-button-sort-asc-desc-toggle-type-selection=="true"]' is not a valid selector. at __puppeteer_evaluation_script__:1:29` – Huckleberry Carignan Sep 07 '18 at 19:21
This is weird to me, usually when the selector is incorrect, I get the following error: `Error: No node found for selector: [data-test-button-sort-asc-desc-toggle-type-selection-GARBAG="true"]` – Huckleberry Carignan Sep 07 '18 at 19:22
You have a double equal sign (`==`) in `querySelectorAll(...)` which is causing an error. It should be one equal sign: `=`. – Grant Miller Sep 07 '18 at 19:23
Ah ha! I put a around the element(s) with the tag and it worked ` {{option.label}} ` – Huckleberry Carignan Sep 07 '18 at 21:26

score 4 · Answer 2 · answered Dec 13 '20 at 08:52

4

https://github.com/puppeteer/puppeteer/blob/v5.5.0/docs/api.md#frameselector-1

const pageFrame = page.mainFrame();
const elems = await pageFrame.$$(selector);

answered Dec 13 '20 at 08:52

Dmytro

304
2
13

ggorlen · Answer 3 · 2023-01-28T19:28:17.360

Not mentioned yet is the awesome page.$$eval which is basically a wrapper for this common pattern:

page.evaluate(() => callback([...document.querySelectorAll(selector)]))

For example,

const puppeteer = require("puppeteer"); // ^19.1.0

const html = `<!DOCTYPE html>
<html>
<body>
<ul>
  <li data-test-foo4="true">red</li>
  <li data-test-foo4="false">blue</li>
  <li data-test-foo4="true">purple</li>
</ul>
</body>
</html>`;

let browser;
(async () => {
  browser = await puppeteer.launch();
  const [page] = await browser.pages();
  await page.setContent(html);

  const sel = '[data-test-foo4="true"]';
  const text = await page.$$eval(sel, els => els.map(e => e.textContent));
  console.log(text); // => [ 'red', 'purple' ]
  console.log(text[0]); // => 'red'
  console.log(text[1]); // => 'purple'
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close());

If you want to pass additional data from Node for $$eval to use in the browser context, you can add additional arguments:

const text = await page.$$eval(
  '[data-test-foo4="true"]',
  (els, data) => els.map(e => e.textContent + data),
  "X" // 'data' passed to the callback
);
console.log(text); // => [ 'redX', 'purpleX' ]

You can use page.$$eval to issue a native DOM click on each element or on a specific element:

// click all
await page.$$eval(sel, els => els.forEach(el => el.click()));

// click one (hardcoded)
await page.$$eval(sel, els => els[1].click());

// click one (passing `n` from Node)
await page.$$eval(sel, (els, n) => els[n].click(), n);

or use page.$$ to return the elements back to Node to issue trusted Puppeteer clicks:

const els = await page.$$('[data-test-foo4="true"]');

for (const el of els) {
  await el.click();
}

// or click the n-th:
await els[n].click();

Pertinent to OP's question, you can always access the n-th item of these arrays with the usual syntax els[n] as shown above, but often, it's best to select based on the :nth-child pseudoselector. This depends on how the elements are arranged in the DOM, though, so it's not as general of a solution as array access.

Puppeteer Getting List of Elements with Same Selector

3 Answers3

Linked