1

I want to get the sizes of the product from the adidas.com site using Puppeteer. The sizes are not loaded in the initial loading and the HTML code is as follows:

<div class="sizes___3Stmf">
<div class="gl-label size___TqqSo" data-testid="size-placeholder"><span class="notranslate placeholder___yXpD2">AAA</span></div>
</div>

But after full loading, the elements will change as below and the sizes will be displayed:

<div class="sizes___3Stmf gl-vspace" data-auto-id="size-selector">
<button class="gl-label size___TqqSo selected___2CqxQ"><span>36</span></button>
</div>

How can I wait for the changed elements to load and then get it? My code is as follows. I even tried using the waitForSelector() but it didn't work.

await page.goto(url, {waitUntil: 'domcontentloaded'});
await page.waitForSelector('.sizes___3Stmf button');        
const outputApp = await page.$(".sizes___3Stmf");
const outputHtml = await page.evaluate((el) => el.innerHTML, outputApp);
console.log(outputHtml);

I get the following error by running the above code:

timeouterror: waiting for selector `.sizes___3stmf button` failed: timeout 30000ms exceeded.
mohsen
  • 11
  • 1
  • It's a good idea to share the actual link because pages are super complex and there are a lot of reasons why a selector wait might fail that go deeper than you might suspect. – ggorlen Aug 05 '22 at 14:31

1 Answers1

0

You should rely on 100% stable element attributes. sizes___3Stmf class name doesn't look like one of them.

Try to wait for div with attribute data-auto-id:

await page.waitForSelector('div[data-auto-id="size-selector"] button');
Animir
  • 1,121
  • 10
  • 23
  • TimeoutError: waiting for selector `div[data-auto-id="size-selector"] button` failed: timeout 30000ms exceeded – mohsen Aug 05 '22 at 15:07
  • I think the full page is not loaded with waitUntil: 'domcontentloaded'. Is there a way to wait for the javascript to finish completely? I tried all the ways (load, networkidle2, networkidle0) but it didn't work – mohsen Aug 05 '22 at 15:11
  • @mohsen Try to save screenshot after page is loaded with `await page.screenshot({path: 'test.png'});` – Animir Aug 05 '22 at 16:05