I am trying to scrape this site using google colab. https://www.blibli.com/p/facial-tissue-tisu-wajah-250-s-paseo/is--LO1-70001-00049-00001?seller_id=LO1-70001&sku_id=LO1-70001-00049-00001&sclid=7zuGEaS4hh5SowAA6tnfd5i2wKjR6e3p&sid=c5746ccfbb298d3b&pid=LO1-70001-00049-00001
The idea here is to get the fingerprint to be used and parsed in the requests.
currently my code is
from seleniumwire import webdriver
options = webdriver.ChromeOptions()
options.set_capability(
"goog:loggingPrefs", {"performance": "ALL", "browser": "ALL"}
)
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
# open it, go to a website, and get results
driver = webdriver.Chrome('chromedriver',options=options)
dataranch=[1]
dataulrs=['https://www.blibli.com/p/facial-tissue-tisu-wajah-250-s-paseo/is--LO1-70001-00049-00001?seller_id=LO1-70001&sku_id=LO1-70001-00049-00001&sclid=7zuGEaS4hh5SowAA6tnfd5i2wKjR6e3p&sid=c5746ccfbb298d3b&pid=LO1-70001-00049-00001']
for prod_id,urls in zip(dataranch,dataulrs):
try:
driver.get(urls)
sleep(randint(3,5))
product_name=driver.find_element(By.CSS_SELECTOR, ".product-name").text
try:
normal_price=driver.find_element(By.CSS_SELECTOR, ".product-price__before").text
except:
normal_price="0"
normal_price=normal_price.replace('Rp',"").replace(".","")
try:
discount=driver.find_element(By.CSS_SELECTOR, ".product-price__discount").text
except:
discount="0"
compid=urls.split(".")[4].split("?")[0]
dat={
'product_name':product_name,
'normal_price':normal_price,
'discount':discount,
'competitor_id':compid,
'url':urls,
'prod_id':prod_id,
'date_key':today,
'web':'ranch market'
}
dat=pd.DataFrame([dat])
except Exception as e:
print(f"{urls} error")
print(e)
What am I doing wrong here? Can someone help? Because I tried inspecting the elements and the css . is there. Is there a way to scrape the data needed? Do I need to use a different module just selenium to get the data?
