Stack Overflow
Stack Overflow

Questions tagged [pyppeteer]

Unofficial Python port of puppeteer JavaScript (headless) chrome/chromium browser automation library.

Unofficial Python port of puppeteer JavaScript (headless) chrome/chromium browser automation library.

Pyppeteer is mostly used for:

  1. Generate screenshots and PDFs of pages.
  2. Crawl an SPA and generate pre-rendered content (i.e. "SSR").
  3. Scrape content from websites.
  4. Automate form submission, UI testing, keyboard input, etc.
  5. Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
  6. Capture a timeline trace of your site to help diagnose performance issues.

Resources:

Differences from puppeteer

185 questions
0
votes
1 answer

Trying to websrap with python, pypeteer

Goal is to pull the information off of a website that tracks tiktok followers and post it in console/send in discord channel. Currently using discord to initiate it but having it print in console. Current code listed below…
pythonguy
  • 41
  • 4
0
votes
1 answer

asyncio pyppeeter stop loop and close browser

im trying to open a website in chrome whith pyppeteer, capture all the requests the website makes and look at the headers. If my code finds a specific header, it should close the browser and stop runnig. my code: import asyncio import json import…
Pejko
  • 25
  • 5
0
votes
1 answer

Web scraping with pyppeteer, but site is blocking me

I'm trying to web scraping to get some coupom values applyed in a product in site, but it keeps blocking me. I've already tried to use proxy with my country IP, changing headers, searching for some "id" to access, like some aws auths, with and…
israel
  • 350
  • 1
  • 2
  • 9
0
votes
2 answers

Pyppeteer: how to extract text from div element on Linux (Ubuntu 16.04)?

Given HTML snippet:
About 3,260 results
. It works on Windows however it does not work on Linux (Ubuntu 16.04). I've already installed following extra packages mentioned here: Why does this…
cloudSAPiens
  • 1
  • 1
  • 2
0
votes
1 answer

I am facing this problem in pyppeteer in simply login to page and logout by clicking on div anchor tag

Where i am going wrong? I just want to go on some URL fill login file fields and click on submit button and select logout div click on it and exit.. pyppeteer.errors.NetworkError: Execution context was destroyed, most likely because of a…
Zafar Abbas
  • 27
  • 5
0
votes
2 answers

Webscraping with requests_html but it says a chromium file is missing

I trying to web scrape using requests-html but it returns an error saying there is a missing file even though I pip install requests-html and it said all req fulfilled. how do I get around this. from requests_html import HTMLSession import time url…
Houston Khanyile
  • 75
  • 8
0
votes
1 answer

Pyppeteer (python) - clink a tag and after scraping the page

I am new to Pyppeteer (Python) and I am trying to know how to (in order): log into the page clink a tag take the data from the tag which I have clinked The website is 'https://quotes.toscrape.com/login' I think I managed to solve the first part…
Omerge
  • 69
  • 1
  • 7
0
votes
0 answers

Why do I get ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host?

Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: Traceback (most recent call last): File "C:\ProgramData\Anaconda3\lib\asyncio\events.py", line 81,…
Prady
  • 19
  • 2
0
votes
1 answer

Is there way to access chrome extension/plugin through python library pyppeteer?

I have a use case where I need to click the browser extension/plugin while navigating the URL on headless browser in python library puppeteer.
Prady
  • 19
  • 2
0
votes
1 answer

Pyinstaller script error when converting a python file into an executable

I'm trying to use pyinstaller to convert my python file into an executable, but I keep getting this error. Traceback (most recent call last): File "sr_gui_test.py", line 12, in File "", line 991, in…
Ayo
  • 81
  • 1
  • 1
  • 8
0
votes
1 answer

Python pyppeteer, how to use a socks proxy

I m trying to make my code use a socks proxy to browse a website with pyppeteer. I couldn't find a way to make it work, here the code: browser = await PYP.launch() page= await browser.newPage() await page.goto(url) Is there any way to run…
FrancescoL
  • 89
  • 7
0
votes
2 answers

page.waitForTimeout() in pyppeteer?

I have a pyppeteer (not puppeteer) browser with many pages opened, and I'd like to wait for example 2 second before doing other stuff on some of these pages. I tried to use time.sleep() but it looks like it blocks the execution of all pages. Is…
varjoll
  • 29
  • 5
0
votes
1 answer

Puppeteer - browser.pages()

I am running await browser.pages() in puppeteer (pyppeteer), however every time it is run, my browser pages go to a very small window size. Is there any way to avoid this? I am for example scanning every 1s to evaluate all browser URLs, and…
ShivP98
  • 11
  • 1
0
votes
1 answer

Click a button within an iframe - pyppeteer

I am trying to click a button inside an iframe using pyppeteer - the python version of puppeteer However, I have drawn a blank. A simple page.click('#selector') does not seem to draw results and await page.waitForSelector('#selector') does not pick…
Shivy011
  • 1
  • 1
0
votes
1 answer

My requests-html does not work and gives error 'No module named email.utils.'

I am learning web scraping and I installed requests-html. Now I ran this script I saw in a tutorial, and can't get it to work. I don't understand as it is a part of the standard library. I tried pip install email but it just returned another…
Uttam Sharma
  • 1
  • 2
1 2 3
…
12 13