0

Is there a way to optimize page scrapping using pyppeteer?

My below code sometimes takes more than 20s to scrape page.

launch(
            executablePath=path,
            handleSIGINT=False,
            handleSIGTERM=False,
            handleSIGHUP=False,
            args=[
                '--disable-dev-shm-usage',
                '--disable-gpu',
                '--single-process',
                '--no-zygote',
                '--no-sandbox',
                '--disable-web-security',
                '--disable-notifications'
            ],
            userDataDir='/tmp/',
            headless=True)
        page = await browser.newPage()

        res = await load_page(page, url, headers, timeout)

I am using pyppeteer==0.2.6

ALTAF HUSSAIN
  • 205
  • 2
  • 13

0 Answers0