When I send a get requests to https://twitter.com/ using Burp Suite or CUrl, I get HTTP 302 with following body: Found. Redirecting to /i/flow/login?redirect_after_login=%2F. However, when I make the same get request from Python-requests, I get HTTP 200, with a long html saying Something went wrong, but don't fret, let's give it another shot.. I cannot make requests to https://twitter.com/.
I used the following code with and without headers:
import requests
url = "https://twitter.com/"
headers = {
'Host' : 'twitter.com',
'Sec-Ch-Ua' : '" Not A;Brand";v="99", "Chromium";v="104"',
'Sec-Ch-Ua-Mobile' : '?0',
'Sec-Ch-Ua-Platform' : '"Windows"',
'Upgrade-Insecure-Requests' : '1',
'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.102 Safari/537.36',
'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'Sec-Fetch-Site' : 'none',
'Sec-Fetch-Mode' : 'navigate',
'Sec-Fetch-User' : '?1',
'Sec-Fetch-Dest' : 'document',
'Accept-Encoding' : 'gzip, deflate',
'Accept-Language' : 'en-US,en;q=0.9'
}
response = requests.get(url=url, headers=headers)
It returns the same response regardless of the headers.
But with Burp Suite, using the same headers, or no headers, or with CUrl, with no headers, it returns the correct "HTTP 302" response.
