I noticed that one url
makes pyppeteer
hang forever. That's weird because I've set timeout
. Do you know where is the problem? Or is it asyncio
problem?
async def test():
url = 'https://ig.com.br/'
browser = await launch(headless=True)
page = await browser.newPage()
page.setDefaultNavigationTimeout(1000)
await page.setUserAgent(fake_useragent.UserAgent().random)
await page.goto(url, {'waitUntil': 'networkidle0','timeout':1000})
await browser.close()
asyncio.run(test())
Other websites work well - it timeouts because the timeout value is too small. But this website makes it hang forever.
Do you know why? I want to make sure that any website will either be fetched under 20 seconds or raise error.