1

I'm stuck here.

I got multiple rows with class rowcontent.

I get them like that:

const rows = await page.$$('.row-content');

Almost every row in rows got many spans with the class named cashspan.

I would like to get those values in an array called 'values'.

I've tried much to many things with no success

for (let m = 0; m < rows.length; m++) {
        const row = await rows[m];
const values = await row.evaluate(() => Array.from(row.getElementsByClassName('cashspan'), element => element.textContent));
 console.log(values)
}

this was the latest thing I've tried.

With

const spancashs = await page.evaluate(() => Array.from(document.querySelectorAll('[class="cashspan"]'), element => element.textContent));

I get all the elements on the page. But i need them for every row. Hope that makes sense.

Update1

Example:

<div class="container">
        <div class="row-content">
            <div class="someclass1">
                <div class="someclass2">
                    <span class="cashspan">1</span>
                </div>
            </div>
            <div class="someclass3">
                <div class="someclass4">
                    <span class="cashspan">2</span>
                </div>
            </div>
            <div class="someclass5">
                <div class="someclass6">
                    <span class="cashspan">3</span>
                </div>
            </div>
        </div>
        <div class="row-content">
            <div class="someclass7">
                <div class="someclass8">
                    <span class="cashspan">4</span>
                </div>
            </div>
            <div class="someclass9">
                <div class="someclass10">
                    <span class="cashspan">5</span>
                </div>
            </div>
            <div class="someclass11">
                <div class="someclass12">
                    <span class="cashspan">6</span>
                </div>
            </div>
        </div>
        <div class="row-content">
            <div class="someclass13">
                <div class="someclass14">
                    <span class="cashspan">7</span>
                </div>
            </div>
            <div class="someclass15">
                <div class="someclass16">
                    <span class="cashspan">8</span>
                </div>
            </div>
            <div class="someclass17">
                <div class="someclass18">
                    <span class="cashspan">9</span>
                </div>
            </div>
        </div> 
     </div>

Code:

 const rows = await page.$$('.row-content');
    
 for (let i = 0; i < rows.length; i++) {
        const row = await rows[i];
        const values = await row.evaluate(() => 
        Array.from(row.getElementsByClassName('cashspan'), element => 
        element.textContent));
   console.log(values)
 }

I'm trying to get all cashspan values in every row-content container. The output for this example should be:

[ 1, 2, 3 ]
[ 4, 5, 6 ]
[ 7, 8 ,9 ]
Jinkle
  • 11
  • 2
  • `() => Array.from(row` ... what's `row` supposed to be here? Try adding a parameter: `row => ...`. The callback to `evaluate` is run in browser context, not Node, so it doesn't have access to any data you don't pass into it. I strongly recommend showing the page, or a [mcve] of it, rather than describing it in English, which makes it a lot of guesswork to figure out what's actually there. – ggorlen Mar 31 '22 at 14:32
  • Thank you very much for answering. I Updated my question so it may be clearer now. If you got any more suggestions please let me know :) – Jinkle Mar 31 '22 at 15:21
  • Thanks, looks good. Did adding the `row` parameter to `evaluate` work for you? If not, please provide more details or a link to the actual site because the code worked for me on your provided HTML with that one change. – ggorlen Mar 31 '22 at 15:24
  • Could you give me your working Code for the provided example? So I can make sure I tried it the 'right' way? I can't provide a link for now, because it's a site where you got to log in. – Jinkle Mar 31 '22 at 15:55

1 Answers1

0

Following up on the comments, the row variable inside of evaluate()'s callback was never defined in browser context. Adding that variable to the evaluate() callback parameter list worked for me on the provided example. This is the only non-cosmetic change below:

const puppeteer = require("puppeteer"); // ^13.5.1

const html = `
<body>
<div class="container">
  <div class="row-content">
    <div class="someclass1">
      <div class="someclass2">
        <span class="cashspan">1</span>
      </div>
    </div>
    <div class="someclass3">
      <div class="someclass4">
        <span class="cashspan">2</span>
      </div>
    </div>
    <div class="someclass5">
      <div class="someclass6">
        <span class="cashspan">3</span>
      </div>
    </div>
  </div>
  <div class="row-content">
    <div class="someclass7">
      <div class="someclass8">
        <span class="cashspan">4</span>
      </div>
    </div>
    <div class="someclass9">
      <div class="someclass10">
        <span class="cashspan">5</span>
      </div>
    </div>
    <div class="someclass11">
      <div class="someclass12">
        <span class="cashspan">6</span>
      </div>
    </div>
  </div>
  <div class="row-content">
    <div class="someclass13">
      <div class="someclass14">
        <span class="cashspan">7</span>
      </div>
    </div>
    <div class="someclass15">
      <div class="someclass16">
        <span class="cashspan">8</span>
      </div>
    </div>
    <div class="someclass17">
      <div class="someclass18">
        <span class="cashspan">9</span>
      </div>
    </div>
  </div>
</div>
</body>
`;

let browser;
(async () => {
  browser = await puppeteer.launch({headless: true});
  const [page] = await browser.pages();
  await page.setContent(html);
  const rows = await page.$$('.row-content');

  for (let i = 0; i < rows.length; i++) {
    const row = await rows[i];
    const values = await row.evaluate(row => Array.from(
      row.getElementsByClassName('cashspan'),
      element => element.textContent
    ));
    console.log(values);
  }
})()
  .catch(err => console.error(err))
  .finally(() => browser?.close())
;

If this isn't working on the live site, the problem could be due to any number of JS behaviors, visibility or timing issues, so more detail would be necessary to accurately reproduce the problem.

ggorlen
  • 44,755
  • 7
  • 76
  • 106
  • Thank you ggorlen! It actually works now. Only change I made to your code snippet was changing textContent to innerHTML as it is the last element in the DOM and gives me the raw number in my case. Have a great Day Sir! – Jinkle Apr 01 '22 at 12:32