Stack Overflow
Stack Overflow

Questions tagged [pyppeteer]

Unofficial Python port of puppeteer JavaScript (headless) chrome/chromium browser automation library.

Unofficial Python port of puppeteer JavaScript (headless) chrome/chromium browser automation library.

Pyppeteer is mostly used for:

  1. Generate screenshots and PDFs of pages.
  2. Crawl an SPA and generate pre-rendered content (i.e. "SSR").
  3. Scrape content from websites.
  4. Automate form submission, UI testing, keyboard input, etc.
  5. Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
  6. Capture a timeline trace of your site to help diagnose performance issues.

Resources:

Differences from puppeteer

185 questions
1
vote
1 answer

Connect to a browser with Pyppeteer

I want to connect to a chrome browser that i have started with the launch command await launch(headless=False, \ executablePath ="C:/Program Files/Google/Chrome/Application/chrome.exe",\ …
Teodoros
  • 459
  • 4
  • 14
1
vote
2 answers

How to check if Pyppeteer browser has closed?

I can't seem to find any information regarding Python's version of Puppeteer on how to check if my browser has closed properly, following browser.close(). I have limited knowledge of JavaScript, so can't properly follow the answer puppeteer : how…
MasayoMusic
  • 594
  • 1
  • 6
  • 24
1
vote
0 answers

Pypeeter prints blank list in PDF instead of full document

I'm using Django=3.1 pyppeteer=0.2.5 #for printing page as PDF I don't know where is the problem here, because it has worked fine before. I'm using new ability of Django for making async response. Maybe here is the problem? the code for printing…
Ian
  • 43
  • 1
  • 7
1
vote
0 answers

Pyppeteer - timeout = 1000 ms but page.goto still hangs forever

I noticed that one url makes pyppeteer hang forever. That's weird because I've set timeout. Do you know where is the problem? Or is it asyncio problem? async def test(): url = 'https://ig.com.br/' browser = await launch(headless=True) …
Milano
  • 18,048
  • 37
  • 153
  • 353
1
vote
1 answer

Build scraper REST API server with pyppeteer or selenium

I need to create a server to which I can make REST requests by obtaining the scraped data from the indicated site. For example a url like this: http://myip/scraper?url=www.exampe.com&token=0 I have to scrape a site built in javascript that…
tecn603
  • 211
  • 5
  • 14
1
vote
2 answers

pyppeteer-install behind proxy

I'm behind a corporate proxy. I can get pip working by doing set https_proxy=http://myproxy:port so I can install pyppeteer but whatever I've tried - I can't get pyppeteer to download chromium. I run pyppeteer-install, and it just says downloading…
Darren Oakey
  • 2,894
  • 3
  • 29
  • 55
1
vote
2 answers

Getting the URL of a new tab in pyppeteer

I'm struggling to figure how to: Detect when a new tab is opened in chrome (e.g. from clicking something on a page) Get the url of that newly opened tab. I have connected pyppeteer to my current chrome instance with this: browser = await…
Duckyy
  • 11
  • 3
1
vote
1 answer

using pyppeteer in a continuous scraping mode

Every example and use case uses pyppeteer where browser is opened and close immediately. e.g. import asyncio from pyppeteer import launch async def main(): browser = await launch() page = await browser.newPage() await…
desmond
  • 1,853
  • 4
  • 21
  • 27
1
vote
1 answer

Why does this pyppeteer code only work on windows?

I have written a small program in python using pyppeteer. It runs fine on my Windows computer, but when I tried running it on a Unix-based system it did not work. Here's a minimal reproducible example: import asyncio from pyppeteer import…
ThatCoolCoder
  • 249
  • 3
  • 16
1
vote
0 answers

Pyppeteer: error when clicking on link with href "javascript:void(0);"

A beginner in JS/Html, I'm trying to click on a link in an unordered list, through Pyppeteer0.2.5 (Python3.6.9/Chromium 87.0.4280.66). The html code of the link is:
  • The Authors
    • 131
    • 5
    • 1
    vote
    1 answer

    Pyppeteer / puppeteer / Angular JS selection of radio button doesn't work

    I have a div function which has a radio button which I would like to select 'fire' using Pyppeteer. However, when I try to do this, all attempts I try fail Div
    Christian Papathanasiou
    • 321
    • 1
    • 3
    • 12
    1
    vote
    1 answer

    How to error check pyppeteer page.goBack()

    How would I error check if there is a previous page for the chromium browser to go to? I have a button which freezes out if the chromium instance has just been launched and there is not a previous page to go to using the command page.goback()
    TheOuz
    • 47
    • 7
    1
    vote
    0 answers

    Print the Navigator attributes in pyppeteer?

    I want to know what navigator attributes are loaded when my pages load. Here are some attributes that I should be able to retrieve, https://www.w3schools.com/js/js_window_navigator.asp. However, there seems to be more such…
    MasayoMusic
    • 594
    • 1
    • 6
    • 24
    1
    vote
    0 answers

    How to get element attribute by xpath in pyppeteer

    I'm trying to use ElementHandle.Jeval() to get element style attribute through Pyppeteer. But the problem is I can only address the element by xpath since the element class is dynamically generated. However, the ElementHandle.Jeval() requries…
    Josh Liu
    • 374
    • 1
    • 2
    • 12
    1
    vote
    0 answers

    how to use async functions with multiprocessing in python?

    I am using pyppeteer to take a screenshot https://pypi.org/project/pyppeteer/. But it takes a lot of time so wanted to use multiprocessing. This is my code. import nest_asyncio nest_asyncio.apply() import asyncio from pyppeteer import…
    Abhishek Verma
    • 13
    • 3
    1 2 3
    …
    12 13