0

I want to use pyppeteer to open a webpage, wait for the search box to appear, fill in a search query, submit and wait for the new page's content (obtained after clicking the submit button) to appear.

I have tried using the following code with page.evaluate and javascript to simulate the above tasks.

Javascript

script = """
    () => {
    async function waitForElm(selector) {
            return new Promise(resolve => {
                if (document.querySelector(selector)) {
                    return resolve(document.querySelector(selector));
                }

        const observer = new MutationObserver(mutations => {
            if (document.querySelector(selector)) {
                resolve(document.querySelector(selector));
                observer.disconnect();
            }
        });
                observer.observe(document.body, {
                    childList: true,
                    subtree: true
                });
            });
        }
        
        waitForElm('input').then( function() {

        document.getElementsByClassName("px-3 py-2 outline-none rounded-none disabled:bg-gray-200")[0].setAttribute("value", "%s");
        document.getElementsByClassName("px-3 py-2 outline-none rounded-none disabled:bg-gray-200")[0].click();
        document.getElementsByClassName("px-3 py-2 outline-none rounded-none disabled:bg-gray-200")[0].dispatchEvent(new Event('change', { 'bubbles': true }));
        document.getElementsByClassName('bg-secondary-600 text-white py-1 font-bold shadow-lg')[0].click();
        }).then(function() { return waitForElm('button[class="px-6 py-2 bg-primary-500 hover:bg-primary-700 text-white font-medium rounded"]');});
    }
    """%(query)

Code not giving correct content

url='https://pricehistoryapp.com/'
page = await self.browser.newPage()
await page.goto(url, options={'timeout': 125000})
result = await page.evaluate(script)
content = await page. Content()

But I am still getting the output 'content' as the page which should be before doing the search query.

I have also tried using waitUntil: 'networkidle0', Page.waitForNetworkIdle(), waitForSelector methods but still no help.

But I get the correct content of page (i.e. after button click) if I use sleep i.e. :

Code giving correct content but constant sleep

page = await self.browser.newPage()
await page.goto(url, options={'timeout': 125000})
result = await page.evaluate(script)
await asyncio.sleep(29)
content = await page. Content()

I don't know what mistake I am making please help.

ggorlen
  • 44,755
  • 7
  • 76
  • 106
SUNITA GUPTA
  • 121
  • 4

0 Answers0