Stack Overflow
Stack Overflow

Questions tagged [puppeteer]

Puppeteer is a Node.js library which provides a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome or Chromium.

Puppeteer is a Node.js library which provides a high-level API to control headless Chrome or Chromium over the DevTools Protocol. It can also be configured to use full (non-headless) Chrome or Chromium. You can also use Puppeteer with Firefox Nightly (experimental support).

Puppeteer is mostly used for:

  1. Generate screenshots and PDFs of pages.
  2. Crawl an SPA and generate pre-rendered content (i.e. "SSR").
  3. Scrape content from websites.
  4. Automate form submission, UI testing, keyboard input, etc.
  5. Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
  6. Capture a timeline trace of your site to help diagnose performance issues.

Resources:

Books:

7753 questions
13
votes
4 answers

How to enable parallel tests with puppeteer?

I am using the chrome puppeteer library directly to run browser integration tests. I have a few tests written now in individual files. Is there a way run them in parallel? What is the best way to achieve this?
Matt
  • 487
  • 1
  • 6
  • 16
13
votes
3 answers

How to catch an exception inside an event listener?

I use Puppeteer library to open an URL and process all requests' responses. Sometimes inside the event listener page.on('response') I need to throw an error like in the example below. But I'm unable to catch these exceptions in any way, I always got…
pronngo
  • 820
  • 11
  • 26
12
votes
1 answer

Puppeteer - ProtocolError: Protocol error (Page.printToPDF): Printing is not available

I get the below error when trying to print PDF with Puppetter. I've not been able to find much information about this error online. Does this mean this particular page just doesn't support PDF or is there a setting in my code that could ammend this?…
Matt Herbstritt
  • 4,754
  • 4
  • 25
  • 31
12
votes
3 answers

Webscraping TimeoutError: Navigation timeout of 30000 ms exceeded

I'm trying to extract some table from a company website using puppeteer. But I don't understand why the browser open Chromium instead my default Chrome, which then lead to "TimeoutError: Navigation timeout of 30000 ms exceeded", not let me enough…
Thang Dang
  • 385
  • 2
  • 4
  • 22
12
votes
2 answers

How do I set multiple custom HTTP headers in puppeteer?

I am trying to login using puppeteer at https://kith.com/account/login?return_url=%2Faccount When I login and solve the captcha with audio, it detects me as a bot, so I am trying to change the request headers to see if that helps but cannot find…
Salvatore Timpani
  • 407
  • 3
  • 6
  • 26
12
votes
2 answers

How do print the console output of the page in puppeter as it would appear in the browser?

I keep seeing this WRONG CODE page.on('console', msg => console.log(msg.text())); That FAILS console.log('Hello %s', 'World'); produces Hello World // browser Hello %s World // puppeteer Ok, So I thought maybe I could do…
gman
  • 100,619
  • 31
  • 269
  • 393
12
votes
2 answers

Is there a way to trigger standard zoom (not headless)

I am running puppeteer with headless mode off in order to automate and remotely control a visible Chromium browser on another computer. Is there a way to trigger or emulate zooming on the browser just like in the UI menus or ctrl +/crtl -…
Steve H
  • 946
  • 9
  • 22
12
votes
3 answers

How to access current value in a text input field with puppeteer

I'm trying to automate retrieving form values from an already filled out form with puppeteer and xpath. I've already automated FILLING a text input field as follows, but doing the reverse with .evaluate() doesn't work: [fieldHandle] = await…
remed.io
  • 168
  • 1
  • 1
  • 9
12
votes
2 answers

How to get children of elements by Puppeteer

I understand that puppeteer get its own handles rather than standard DOM elements, but I don't understand why I cannot continue the same query by found elements as const els = await page.$$('div.parent'); for (let i = 0; i < els.length; i++) { …
Googlebot
  • 15,159
  • 44
  • 133
  • 229
12
votes
1 answer

How to get the download stream (buffer) using puppeteer?

I want to get the download content (buffer) and after soon, store the data at my S3 account. So far I wasn't able to find out some solution... Looking for some examples in the web, I noticed that there is a lot of people with this problem. I tried…
Rogério Oliveira
  • 414
  • 1
  • 4
  • 15
12
votes
1 answer

Puppeteer not launching chromium in Mac 10.14

I am trying to open a page through puppeteer but it's not throwing any error and entire code is getting executed by chromium doesn't show up. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const…
Akash Verma
  • 344
  • 3
  • 12
12
votes
3 answers

Get elements from page.evaluate in Puppeteer?

I'm working with Node.js and Puppeteer for the first time and can't find a way to output values from page.evaluate to the outer scope. My algorithm: Login Open URL Get ul Loop over each li and click on it Wait for innetHTML to be set and add it's…
Bong2000
  • 473
  • 1
  • 5
  • 9
12
votes
5 answers

How to stop all JS scripts in Puppeteer

I would like to be able to stop any scripts from being able to run in puppeteer after the page has loaded. The reason for this is to stop carousel images and lazy loading images and essentially get the page to behave as statically as possible to…
RexFuzzle
  • 1,412
  • 2
  • 17
  • 30
12
votes
3 answers

How to find or pick a suitable Chrome/Chromium revision number?

I couldn't figure out a nice way to determine a suitable revision number to use with Puppeteer Sharp. What I did was use the version lookup feature on the "OmahaProxy - Google Chrome" site. I looked-up the version of Chrome I'm running on my…
Kenny Evitt
  • 9,291
  • 5
  • 65
  • 93
12
votes
2 answers

Puppeteer can't find selector

I'm attempting to do a bit of web scraping using Puppeteer, but the script seems unable to find the selector I'm looking for. Basically this code: const puppeteer = require('puppeteer'); let scrape = async () => { const year = 18; const browser =…
Nicholas Jennings
  • 201
  • 2
  • 3
  • 8
1 2 3
…
99 100