1

Hi I'm looking for a way to scrape badge and point numbers from Trailblazer.

Here is the code I am currently using, based off another stackoverflow post I found.

`

const puppeteer = require("puppeteer");
const cheerio = require("cheerio");

const scrape = async () => {
  const url = "https://trailblazer.me/id/hverma99";

  async function getPage(url) {
    const browser = await puppeteer.launch({ headless: true });
    const page = await browser.newPage();
    await page.goto(url, { waitUntil: "networkidle0" });

    const html = await page.content(); // serialized HTML of page DOM.
    await browser.close();
    return html;
  }

  const html = await getPage(url);
  const $ = cheerio.load(html);
  const span = $(".tds-tally__count.tds-tally__count_success");
  console.log(span.text());
  console.log("hit");
};

scrape();

`

The "hit" console.log does appear in console, but the span.text console.log does not.

Am I missing a step? Did the website update since the post I saw? Any help would be appreciated.

Connor Breault
  • 11
  • 1
  • can you try the following: `console.log('span.text='+span.text())` or something along those lines? (so you see the log even if its an empty/null string) – NickSlash Oct 28 '22 at 22:45
  • Looking at your target page, looks like the values you are interested in are part of a `shadow-dom` this may help: https://stackoverflow.com/a/29629737/212869 – NickSlash Oct 28 '22 at 22:52

0 Answers0