Stack Overflow
Stack Overflow

Questions tagged [pyppeteer]

Unofficial Python port of puppeteer JavaScript (headless) chrome/chromium browser automation library.

Unofficial Python port of puppeteer JavaScript (headless) chrome/chromium browser automation library.

Pyppeteer is mostly used for:

  1. Generate screenshots and PDFs of pages.
  2. Crawl an SPA and generate pre-rendered content (i.e. "SSR").
  3. Scrape content from websites.
  4. Automate form submission, UI testing, keyboard input, etc.
  5. Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
  6. Capture a timeline trace of your site to help diagnose performance issues.

Resources:

Differences from puppeteer

185 questions
0
votes
0 answers

headless chrome being detected by Akamai

I scrape website https://www.microchipdirect.com/api/Product/ProductInfo?CPN=STM8L052C6T6, it will show 'accesss deny' if you do not goto https://www.microchipdirect.com first.It was OK. Problem: I use pyppeteer,it runs successfully, when I use…
PRETTY TOM
  • 1
  • 1
0
votes
0 answers

Getting Error even after installing Pyppeteer in Windows

Even after installing pyppeteer in windows using the following commands pip install pyppeteer pip install pipenv pipenv shell pipenv install pyppeteer I am still getting this error while executing this command Traceback (most recent call last): …
kg973171001
  • 1
0
votes
1 answer

Weird button behavior while scrapping bet365

I am trying to scrap odd data by Pyppeteer from bet365 follow their UI flow and need for help. The pages are just like this. https://www.bet365.com/#/AC/B1/C1/D8/E123540970/F3/ If the match is outdated, just get into one of the matches that not…
yomao
  • 15
  • 4
0
votes
0 answers

tradingview websocket is blocked when receiving message with pyppeteer

thanks to this thread i was able to connect to tradingview socket with pyppeteer. But i am not able to work with the response otherwise i block the whole program. I can also not make the function "printResponse" async because then i can not call it…
McMoe
  • 1
0
votes
0 answers

Get current browser url in Python

I want to return the current url of a web browser in python. I have come from this link Get current URL from browser using python but has not solved my problem. I have the following: import asyncio from pyppeteer import launch async def main(): …
Ranji Raj
  • 778
  • 4
  • 18
0
votes
1 answer

Pyppeteer code gives error after executing

so I've saw an example code of pyppeteer and tried to execute it and it gave me this error: "Exception has occurred: OSError [WinError 14001] The application has failed to start because its side-by-side configuration is incorrect. Please see the…
Sagi Lolam
  • 3
  • 4
0
votes
0 answers

how keep function printing the websocket response using pyppeteer

how i keep my async function opened priting a response from websocket? just like await asyncio.sleep(1000) import asyncio from pyppeteer import launch async def main(): browser = await launch(headless=True, autoClose=False) page = await…
st1np
  • 1
  • 1
  • 1
0
votes
2 answers

Pyppeteer and Docker Error: Browser closed unexpectedly:

Receiving this error in a Docker and Python3.10 Container An example link is as follow: https://finance.yahoo.com/quote/BABA/options?p=BABA&date=1653004800 Browser closed unexpectedly: Here is my Dockerfile FROM python:3.10 RUN apt-get update #…
Michael Paccione
  • 2,467
  • 6
  • 39
  • 74
0
votes
1 answer

Why does pyppeteer take such a long time to load a single webpage on AWS Lambda

I am currently trying to crawl MVN Repository using puppeteer on AWS Lambda. However, my test function would run for 15 minutes and proceed to fail after that (See below). It seems like the browser is opened but it doesn't crawl. Here is my current…
Wayne
  • 61
  • 1
  • 2
  • 7
0
votes
0 answers

What is causing Pypppeteer/Puppeteer to Timeout?

I'm trying to scrape the contents of this site http://www.intoaqua.com.au and I can't seem to identify what causes pypetteer to believe that the site has not finished loading/rendering. I suspect it's the vimeo video animation, but I'm not sure. I…
Christian
  • 1
0
votes
1 answer

Visual Studio Code doesn´t move to next line usign asyncio and pyppeteer

My goal is to open an browser using pyppeteer and creating a new tab. But Visual Studio stops executing the code after the browser = await launch(headless=False) line. But executing the same code in for Example Thonny does work. Any Help? Here is…
98coolmen
  • 5
  • 2
0
votes
1 answer

Issue puppeteer/pyppeteer headless doesn't render SSR page

Trying to scrape a page using pyppeteer (https://loja.meo.pt/Equipamentos/gaming/Sony/PS5-Digital-Comando-DS-Plus-Card-365-dias?cor=Branco&modo-compra=PromptPayment) -- the screenshot works and i see the modal to consent cookies but the background…
Luis Lopes
  • 506
  • 4
  • 14
0
votes
1 answer

Pytest-asyncio not moving to next statement when using with pyppeteer

I am trying to run Pyppeteer with pytest but after launching chromium it's not going to the next statement. import asyncio import pytest from pyppeteer import launch @pytest.mark.asyncio async def test_BackendWeb(): browser = await…
Manish Saini
  • 5
  • 1
  • 3
0
votes
1 answer

Why is intercepting requests in the pyppeteer not working?

Minimal example: import asyncio from pyppeteer import launch from pyppeteer.network_manager import Request from pyppeteer_stealth import stealth browser = await launch() page = await browser.newPage() await stealth(page=page) async def…
nosheyakku
  • 316
  • 2
  • 11
0
votes
0 answers

How to make requests work with debugging?

I have the following that does not work. import asyncio from pyppeteer import launch import logging logging.basicConfig(level=logging.DEBUG) async def main(): browser = await launch() page = await browser.newPage() await…
user1424739
  • 11,937
  • 17
  • 63
  • 152
1 2 3
…
12 13