I am trying to make a api call to get reviews from best buy website.
If I execute the following curl in the terminal, I do get a response from the server.
curl --location 'https://www.bestbuy.com/ugc/v2/reviews?page=2&pageSize=8&sku=6509757&sort=BEST_REVIEW&variant=A' \
--header 'authority: www.bestbuy.com' \
--header 'accept: */*' \
--header 'accept-language: en-US,en;q=0.9,hi;q=0.8,de;q=0.7,ur;q=0.6,pa;q=0.5,es;q=0.4' \
--header 'dnt: 1' \
--header 'referer: https://www.bestbuy.com/site/doctor-strange-in-the-multiverse-of-madness-steelbookdigital-copy-4k-ultra-hd-blu-ray-blu-ray-2022/6509757.p?skuId=6509757' \
--header 'sec-ch-ua: "Not_A Brand";v="99", "Google Chrome";v="109", "Chromium";v="109"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--header 'sec-fetch-dest: empty' \
--header 'sec-fetch-mode: cors' \
--header 'sec-fetch-site: same-origin' \
--header 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36' \
--header 'x-client-id: ratings-and-reviews-user-generated-content-ratings-and-reviews-v1'
But if I use python requests to make the same api call, the requests never succeeds. the code i am using to do this is following -
import requests
url = "https://www.bestbuy.com/ugc/v2/reviews?page=2&pageSize=8&sku=6509757&sort=BEST_REVIEW&variant=A"
payload={}
headers = {
'authority': 'www.bestbuy.com',
'accept': '*/*',
'accept-language': 'en-US,en;q=0.9,hi;q=0.8,de;q=0.7,ur;q=0.6,pa;q=0.5,es;q=0.4',
'dnt': '1',
'referer': 'https://www.bestbuy.com/site/doctor-strange-in-the-multiverse-of-madness-steelbookdigital-copy-4k-ultra-hd-blu-ray-blu-ray-2022/6509757.p?skuId=6509757',
'sec-ch-ua': '"Not_A Brand";v="99", "Google Chrome";v="109", "Chromium";v="109"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"macOS"',
'sec-fetch-dest': 'empty',
'sec-fetch-mode': 'cors',
'sec-fetch-site': 'same-origin',
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36',
'x-client-id': 'ratings-and-reviews-user-generated-content-ratings-and-reviews-v1'
}
response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
HELP!
I want to what is the reason curl works but no python library works.
