Chrome Headless is the headless mode of Chrome and Chromium used for automation, testing, and CI scenarios. Use this tag for questions related to the usage of Chrome Headless and the DevTools protocol.
Questions tagged [google-chrome-headless]
1156 questions
34
votes
10 answers
How can I download images on a page using puppeteer?
I'm new to web scraping and want to download all images on a webpage using puppeteer:
const puppeteer = require('puppeteer');
let scrape = async () => {
// Actual Scraping goes Here...
const browser = await puppeteer.launch({headless:…
supermario
- 2,625
- 4
- 38
- 58
32
votes
2 answers
Why is puppeteer reporting "UnhandledPromiseRejectionWarning: Error: Navigation failed because browser has disconnected!"?
I've got a simple node.js script to capture screenshots of a few web pages. It appears I'm getting tripped up somewhere along the line with my use of async/await, but I can't figure out where. I'm currently using puppeteer v1.11.0.
const puppeteer =…
Al R.
- 2,430
- 4
- 28
- 40
30
votes
4 answers
How do you send a POST request in Puppeteer?
I use this code for sending a GET request:
(async() => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.example.com/search');
const data = await page.content();
…
J. Doe
- 447
- 1
- 4
- 7
28
votes
5 answers
Crawling multiple URLs in a loop using Puppeteer
I have an array of URLs to scrape data from:
urls = ['url','url','url'...]
This is what I'm doing:
urls.map(async (url)=>{
await page.goto(url);
await page.waitForNavigation({ waitUntil: 'networkidle' });
})
This seems to not wait for page…
ahhmarr
- 2,296
- 4
- 27
- 30
27
votes
2 answers
Node puppeteer take screenshot full page SPA
I have a single page application with scrolling. I am trying to take a screenshot of the whole page, but it only gives me the visible part. How can I make it shoot the whole page?
const browser = await puppeteer.launch(options);
const page =…
JsFan
- 393
- 2
- 4
- 13
26
votes
3 answers
How to specify browser language in Puppeteer
I would like to launch a Google Chrome browser with language Spanish es using Puppeteer.
I've tried puppeteer.launch(args:['--lang=es',...],...) but it didn't work.
I've tried passing the environment variable LANGUAGE=es mocha puppeteer-test.js but…
Giorgio
- 13,129
- 12
- 48
- 75
25
votes
7 answers
Download file through Google Chrome in headless mode
I'm do me code in Cromedrive in 'normal' mode and works fine. When I change to headless mode it don't download the file. I already try the code I found alround internet, but didn't work.
chrome_options =…
CBury
- 411
- 2
- 6
- 9
25
votes
2 answers
Limit chrome headless CPU and memory usage
I am using selenium to run chrome headless with the following command:
system "LC_ALL=C google-chrome --headless --enable-logging --hide-scrollbars --remote-debugging-port=#{debug_port} --remote-debugging-address=0.0.0.0 --disable-gpu --no-sandbox…
Ahmad Hijazi
- 635
- 2
- 9
- 27
24
votes
3 answers
Puppeteer Launch Incognito
I am connected to a browser using a ws endpoint (puppeteer.connect({ browserWSEndpoint: '' })).
When I launch the browser that I ultimately connect to, is there a way to launch this in incognito?
I know I can do something like this:
const incognito…
rpascal
- 723
- 1
- 8
- 25
24
votes
6 answers
Wait for element to appear when using Puppeteer
I wonder if there's a similar way as in Selenium to wait for text to appear for a particular element. I've tried something like this, but it doesn't seem to wait:
await page.waitForSelector('.count', {visible: true});
elena
- 3,740
- 5
- 27
- 38
24
votes
5 answers
How to handle popups in puppeteer
how to handle the popup and access the popup to do some operations on it.
const puppeteer = require('puppeteer');
async function run() {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await…
vinay kumar
- 555
- 1
- 6
- 16
24
votes
7 answers
Alter the default header/footer when printing to PDF
I'm trying to use Google Chrome as a replacement of PhantomJS to render HTML into PDF. So far it's been working well for me. The only issue I have that I have not found any way to translate the following PhantomJS code:
page.paperSize = {
footer:…
apokryfos
- 38,771
- 9
- 70
- 114
23
votes
4 answers
How to set window size in Selenium Chrome Python
The following code to resize a selenium chrome window does not work:
driver.set_window_size(1920, 1080)
time.sleep(5)
size = driver.get_window_size()
print("Window size: width = {}px, height = {}px.".format(size["width"], size["height"]))
From…
David Miller
- 512
- 2
- 4
- 17
23
votes
3 answers
How to use puppeteer to dump WebSocket data
I want to get websocket data in this page https://upbit.com/exchange?code=CRIX.UPBIT.KRW-BTC, its websocket URL is dynamic and only valid during the first connection, the second time you connect to it it will not send data anymore.
So I wonder that…
soulmachine
- 3,917
- 4
- 46
- 56
23
votes
6 answers
Retrieving JavaScript Rendered HTML with Puppeteer
I am attempting to scrape the html from this NCBI.gov page. I need to include the #see-all URL fragment so that I am guaranteed to get the searchpage instead of retrieving the HTML from an incorrect gene page…
Sir_Zorg
- 331
- 1
- 2
- 5