Is there a way to optimize page scrapping using pyppeteer?
My below code sometimes takes more than 20s to scrape page.
launch(
executablePath=path,
handleSIGINT=False,
handleSIGTERM=False,
handleSIGHUP=False,
args=[
'--disable-dev-shm-usage',
'--disable-gpu',
'--single-process',
'--no-zygote',
'--no-sandbox',
'--disable-web-security',
'--disable-notifications'
],
userDataDir='/tmp/',
headless=True)
page = await browser.newPage()
res = await load_page(page, url, headers, timeout)
I am using pyppeteer==0.2.6