Stack Overflow
Stack Overflow

Questions tagged [puppeteer]

Puppeteer is a Node.js library which provides a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome or Chromium.

Puppeteer is a Node.js library which provides a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome or Chromium. You can also use Puppeteer with Firefox Nightly (experimental support).

Puppeteer is mostly used for:

  1. Generate screenshots and PDFs of pages.
  2. Crawl an SPA and generate pre-rendered content (i.e. "SSR").
  3. Scrape content from websites.
  4. Automate form submission, UI testing, keyboard input, etc.
  5. Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
  6. Capture a timeline trace of your site to help diagnose performance issues.

Resources:

Books:

7753 questions
16
votes
1 answer

Run jest unit tests in real browser environment with window access using puppeteer

What i want My code widely depends on global window object (and it's methods) which isn't fully implemented by jsdom, but available in real browser environment. So i want to run unit tests inside puppeteer page context environment so covered code…
user3388811
  • 318
  • 3
  • 13
16
votes
5 answers

Puppeteer page.waitForNavigation() timeout error handling

Using puppeteer, i open up a page when i enter a value - it outputs the result. await page.click('button[class="button form-button rs-gcbalance-btn"]') await page.waitForSelector('div[class="small-4 large-4 rs-gcbalance-result-num-col').catch(err…
almarc
  • 1,598
  • 3
  • 11
  • 31
16
votes
4 answers

Puppeteer CORS mistake

Hello I have problem in my code using puppeteer, CORS error happens randomly but in 80% of my tests. Here is my code thanks for help. BTW the server respond is Access to fetch at…
bredart
  • 178
  • 1
  • 1
  • 10
16
votes
2 answers

Make Puppeteer use local profile's cookies

I want to use my local user's profile with Puppeteer. However, it doesn't seem to work. I launch it with these args. const browser = await puppeteer.launch({ executablePath: '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', …
Puppeteer user
  • 171
  • 2
  • 5
16
votes
1 answer

Puppeteer - wait for element to have certain value

How can I wait until an element has a certain value? For example, some button on a page changes the value of an input field from "No Value" to "Value X". The problem is, I don't know how much time will it take so page.waitFor() is not an option. I…
kmeshavkin
  • 325
  • 1
  • 2
  • 12
16
votes
1 answer

How does an ElementHandle differ from a DOM Element?

The documentation on page.$(selector) says it returns a promise containing an ElementHandle. But the documentation on ElementHandle is a bit lacking. It says it "represents a DOM element", but what does that really mean? And if does represent the…
Atav32
  • 1,788
  • 3
  • 23
  • 33
16
votes
6 answers

puppeteer element.click() not working and not throwing an error

I have a situation where a button, on a form, that is animated into view, if the element.click() happens while the animation is in progress, it doesn't work. element.click() doesn't throw an error, doesn't return a failed status (it returns…
Austin France
  • 2,381
  • 4
  • 25
  • 37
16
votes
4 answers

How can I disable cache in puppeteer?

I want to disable cache in puppeteer, can anyone please tell me how I can do so? I found this page.setCacheEnabled(enabled) but I couldn't understand how to use the same. I am aware that the browser is launched without cache or cookies but in my…
Nagarjun Prasad
  • 802
  • 3
  • 17
  • 31
16
votes
3 answers

How can I dynamically inject functions to evaluate using Puppeteer?

I am using Puppeteer for headless Chrome. I wish to evaluate a function inside the page that uses parts of other functions, defined dynamically elsewhere. The code below is a minimal example / proof. In reality functionToInject() and…
mikemaccana
  • 110,530
  • 99
  • 389
  • 494
15
votes
2 answers

How to get around Error: net::ERR_CONNECTION in Puppeteer

I tried taking a proxy from this site: https://hidemy.name/en/proxy-list/?type=4#list Here is my Puppeteer scraping code (deployed to Heroku), which is returning the error in the title on the .goto() line: const preparePageForTests = async (page) =>…
nickcoding2
  • 142
  • 1
  • 8
  • 34
15
votes
2 answers

Pupeteer - how can I accept cookie consent prompts automatically for any URL?

When taking a screenshot of a website using pupeteer, cookie consent prompts are displayed. I want to dismiss or accept these prompts before taking the screenshot. The problem I am facing is that most websites present the cookie prompt in different…
drs
  • 1,165
  • 3
  • 13
  • 19
15
votes
3 answers

How do you wait for a selector, but only for a specific time period in puppeteer?

I'm scraping data from youtube and trying to get the number of comments. I'm trying to grab the element that contains the value, but in case the comments are disabled for a video, that element doesn't exist at all and waitForSelector() waits, I…
Rohit
  • 1,385
  • 2
  • 15
  • 21
15
votes
10 answers

How to wait for all downloads to complete with Puppeteer?

I have a small web scraping application that downloads multiple files from a web application where the URLs require visting the page. It works fine if I keep the browser instance alive in between runs, but I want to close the instance in between…
Jiri
  • 177
  • 1
  • 1
  • 7
15
votes
2 answers

Puppeteer: How to get the contents of each element of a nodelist?

I'm trying to achieve something very trivial: Get a list of elements, and then do something with the innerText of each element. const tweets = await page.$$('.tweet'); From what I can tell, this returns a nodelist, just like the…
i.brod
  • 3,993
  • 11
  • 38
  • 74
15
votes
2 answers

How can I wait for an ajax request and process the result?

I open a website, then wait for all redirects to be done. Then I capture a captcha image, and send it via nodejs to a user. Then I recive the typed captcha: const browser = await puppeteer.launch({headless: false}); const page = await…
hretic
  • 999
  • 9
  • 36
  • 78
1 2 3
…
99 100