0

I have tried to fetch data from a website. Here's the code I have, but I think my header is wrong or something. I just keep end up with the same issue. That I'm denied.

How can I fix this to avoid that?

from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

import time

user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.50'
sec_ch_ua = 'Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24"'
referer = 'strict-origin-when-cross-origin'

def interceptor(request):
    # delete the "User-Agent" header and
    # set a new one
    del request.headers["user-agent"]  # Delete the header first
    request.headers["user-agent"] = user_agent
    # set the "Sec-CH-UA" header
    request.headers["sec-ch-ua"] = sec_ch_ua
    # set the "referer" header
    request.headers["referer"] = referer

Options = Options()
#Options.add_argument('--headless=new')
Path = "C:\Temp\chromedriver.exe"

driver = webdriver.Chrome(Path,
    options=Options,
    service=ChromeService(ChromeDriverManager().install())
)
# set the Selenium Wire interceptor
driver.request_interceptor = interceptor

url = "https://www.sas.dk/api/offers/flights?to=NYC&from=CPH&outDate=%2020230703&adt=1&chd=0&inf=0&yth=0&bookingFlow=points&pos=dk&channel=web&displayType=upsell"
#driver = webdriver.Chrome(Path,options=Options)
try:
    driver.get(url)

    time.sleep(5)
finally:
    driver.quit

Best regards

Valnurat
  • 71
  • 1
  • 7
  • That website bot detector must be flagging you. You can try a different user agent string, a proxy – Learning is a mess May 23 '23 at 21:56
  • The user agent sting I use now, is the one from the url. I have examined the data with the Developer tool. What do you mean by proxy? Should I add that to the header aswell? – Valnurat May 24 '23 at 08:41

0 Answers0