So I was learning to use Splash 3.5v for scraping Dynamic websites and Splash was successful in sending text splash:send_text()
to the search bar of the website https://www.olx.com.pk/. But Splash was not able to click the search button. I have tried using my User-Agent, tried using this command splash.private_mode_enabled = false
in start, also tried pressing enter with search_bar:send_keys('<Enter>')
command, even tried splash:runjs("JSpath")
but nothing seems to work on this website. I think there is some kind of protection on the website that is not allowing Splash to click the button. It is not just the search button but also the same problem with the category buttons as well. Before I just send the splash request by manually typing the product in the SplashRequest()
https://www.olx.com.pk/items/q-UPS I wanted to ask this question to see if anyone knows about this and can answer why this is happening. Thank you. the snippet is as follows:
function main(splash, args)
splash.private_mode_enabled = false
assert(splash:go(args.url))
assert(splash:wait(1))
search_bar = splash:select("input[type='search']")
search_bar:focus()
search_bar:send_text("UPS")
--[[btn = splash:select("button[aria-label='Search']")
btn:mouse_click()
assert(splash:wait(3))--]]
search_bar:send_keys('<Enter>')
assert(splash:wait(5))
splash:set_viewport_full()
return {
html = splash:html(),
png = splash:png(),
}
end