Python requests WWW-Authenticate missing in headers

Question

I'm trying to access the WWW-Authenticate header after a GET request using Python's requests library. When I do this in Postman it returns it but in Python it doesn't.

Python:

import requests
from requests.auth import HTTPBasicAuth

headers = {
    # "Python-Token": "<calculated when request is sent>",
    # "Host": "<calculated when request is sent>",
    "Accept": "application/json, */*", 
    "User-Agent": "PythonRuntime/3.9.7",
    # "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36",
    "Accept-Encoding": "gzip, deflate, br", 
    "Connection": "keep-alive", 
    "Access-Control-Expose-Headers": "x-WWW-Authenticate",
    'x-requested-with': 'XMLHttpRequest',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    "Authorization": "Bearer"
}
site = "https://YourSPOSiteURL/_vti_bin/client.svc/"
auth = HTTPBasicAuth('something@email.com', 'somepasswordhere')
headers = dict( requests.get(f"{site}/_vti_bin/client.svc/", auth=auth).headers )

headers:

{'Cache-Control': 'private, max-age=0',
 'Content-Length': '0',
 'Content-Security-Policy': "frame-ancestors 'self' teams.microsoft.com "
                            '*.teams.microsoft.com *.skype.com '
                            '*.teams.microsoft.us local.teams.office.com '
                            '*.powerapps.com *.yammer.com '
                            '*.officeapps.live.com *.office.com '
                            '*.stream.azure-test.net *.microsoftstream.com '
                            '*.dynamics.com;',
 'Date': 'Fri, 04 Feb 2022 14:53:16 GMT',
 'Expires': 'Thu, 20 Jan 2022 14:53:16 GMT',
 'Last-Modified': 'Fri, 04 Feb 2022 14:53:16 GMT',
 'MS-CV': 'xxxxxxxxxxxxx.0',
 'MicrosoftSharePointTeamServices': 'xx.x.x.xxxxx',
 'P3P': 'CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR '
        'SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"',
 'SPRequestGuid': '697da8340-a0d3-30123-9fc9-98c88419387426',
 'Server': 'Microsoft-IIS/10.0',
 'Strict-Transport-Security': 'max-age=31536000',
 'Vary': 'Origin',
 'X-AspNet-Version': '4.0.xxxxx',
 'X-Content-Type-Options': 'nosniff',
 'X-FRAME-OPTIONS': 'SAMEORIGIN',
 'X-MS-InvokeApp': '1; RequireReadOnly',
 'X-Powered-By': 'ASP.NET',
 'X-SharePointHealthScore': '2',
 'request-id': '697da8340-a0d3-30123-9fc9-98c88419387426'}

Postman:

I've tried every headers param I can find but can't seem to return the WWW-Authenticate response header which I need...any help would be much appreciated.

Edit:

Within Postman, it looks to be the Host key in the headers get request that's generating the WWW-Authenticate in the response headers. When I remove it (in Postman) I get only a handful of headers in the response headers. But how would I replicate this in Python's requests library?

This is what it says when hovering over the Host header in GET:

score 1 · Accepted Answer · answered Feb 08 '22 at 01:20

1

Pass the request headers to the server:

# requests.get(f"{site}/_vti_bin/client.svc/", auth=auth)
requests.get(f"{site}/_vti_bin/client.svc/", headers=headers, auth=auth)

answered Feb 08 '22 at 01:20

aaron

39,695
6
46
102

score 1 · Answer 2 · answered Feb 08 '22 at 06:08

You are not using that headers dict when sending the request.

You can easily debug what was actually sent accessing the request property from the response you got without additional software

response = requests.get(
    f"{site}/_vti_bin/client.svc/",
    auth=('something@email.com', 'somepasswordhere'),
    headers=headers,
)
print(response.request.headers)
print(response.headers)

so you can inspect all those additional headers (but not the Host header https://stackoverflow.com/a/57771006/11715259)

See https://docs.python-requests.org/en/latest/user/advanced/#prepared-requests

Yet, I truly recommend you get used to throw in a reverse proxy for debugging.

https://mitmproxy.org/ is quite great for this, much more playful than tcpdump/wireshark if you only work with the http protocol.

score 1 · Answer 3 · answered Feb 08 '22 at 09:35

When I do this in Postman it returns it but in Python, it doesn't.

You are passing only the basic auth parameter to your request. You should provide headers whose are set by Postman to get a response like it.

import requests


headers = {
    "Accept": "application/json, */*",
    "User-Agent": "PythonRuntime/3.9.7",
    "Accept-Encoding": "gzip, deflate, br",
    "Connection": "keep-alive",
    "Access-Control-Expose-Headers": "x-WWW-Authenticate",
    "x-requested-with": "XMLHttpRequest",
    "sec-fetch-site": "same-origin",
    "sec-fetch-mode": "cors",
    "Authorization": "Bearer <token>"
}

response = requests.get(
   url="https://YourSPOSiteURL/_vti_bin/client.svc/",
   auth=("something@email.com", "somepasswordhere"),
   headers=headers
)

print(response.headers.get("WWW-Authenticate"))

Python requests WWW-Authenticate missing in headers

3 Answers3