How can I check if selector exists in Puppeteer?

Question

In Puppeteer, how can I check if, for example, #idProductType exists and if not, set producttype to ""? I tried many many things but it doesn't work.

const urls = [myurls, ...]
const productsList = [];
for (let i = 0; i < urls.length; i++) {
    const url = urls[i];
    await Promise.all([
        page.goto(url),
        page.waitForNavigation({ waitUntil: 'networkidle0' }),
    ]);

    let products = await page.evaluate(() => {

  //here i want to test if #idProductType exists do : 
        let producttype = document.querySelector('#idProductType').innerText;
  //else 
        let producttype = "";
  //and same thing for other selectors

        let productsubtype = document.querySelector('#idProductSubType').innerText;
        let product = document.querySelector('#idProduct').innerText;
        let description = document.querySelector('td.js-orderModelVer').innerText;
        let reference = document.querySelector('td.text-nowrap').innerText;
        let prixpub = document.querySelector('td.text-nowrap.text-right.js-pricecol-msrp').innerText;
        let dispo = document.querySelector('td.text-nowrap.text-center.js-pricecol-availability').innerText;
        let retire = document.querySelector('td.js-retired-filler-cell').innerText;

        let results = [];
        results.push({
            producttype: producttype,
            productsubtype: productsubtype,
            product: product,
            description: description,
            reference: reference,
            prixpub: prixpub,
            dispo: dispo,
            retire: retire
        })
        return results
    })
    productsList.push(products);
}

Answer 1

You can use page.$(selector) which is similar to document.querySelector(selector)

let producttype = (await page.$('#idProductType')) || "";

Answer 2

return innerText or empty string if not found:

let productType = await page.evaluate(() => {
  let el = document.querySelector(".foo")
  return el ? el.innerText : ""
})

Answer 3

Puppeteer throws an error when could not find a matched element.

So to confirm the existence,

try {
  await page.$(selector)
  // Does exist
} catch {
  // Does not
}

Or to make whether it exists a flag

const exists = await page.$eval(selector, () => true).catch(() => false)

Answer 4

Another aproach that may be more suitable for such usecases:

let producttype

if ((await page.$('#idProductType')) !== null) {
  // do things with its content
  producttype = await page.evaluate(el => el.innerText, await page.$('#idProductType'))
} else {
  // do something else
  producttype = ''
}

Answer 5

querySelector() returns null value if there are not available element in DOM with specific selector

So you could write simple helper function:

const getInnerTextForSelector = (selector) => {
    const element = document.querySelector(selector);
    if (element)
        return element.innerText;
    return '';
};

and run for example for #idProductType selector:

const producttype = getInnerTextForSelector('#idProductType');

Or you could write helper which will operate on puppeteer Page and ElementHandle's:

const getElementForSelector = async (page, selector) => {
    return (await element.$(selector)) || undefined;
};

export const getInnerText = async (page, selector) => {
    const elementForSelector = await getElementForSelector(page, selector);
    try {
        if (elementForSelector)
            return (
                (await elementForSelector.evaluate(element => {
                    return element.innerText;
                })) || ''
            );
    } catch {
        return '';
    }
};

and then run for example for #idProductType selector:

const producttype = await getInnerText(page, '#idProductType');

Answer 6

If the element is generated by Javascript it's a good idea to use a timeout:

let producttype;
try {
  await page.waitForSelector("#idProductType", {timeout: 1000});
  producttype = document.querySelector('#idProductType').innerText;
} catch (error) {
  producttype = "";
} 

Answer 7

Here is the method I use to do this

Here is the way I do it

if(page.$(selector) !== null) {
  // selector was found in the page
} else {
  // selector not found
}

for reference, you may check this discussion

also here you can see it in action

Answer 8

I think i find a way to do it, i don't know if it is the best solution :

            var producttype = "";
            try {
                var producttype = document.querySelector('#idProductType').innerText;
            } catch (err) {
                console.log("The element didn't appear.")