2

I'm puzzled at seeing how a HttpClient instance is not sending DefaultRequestHeaders (specifically the Authorization header) when following a 301 redirect. Considering the name of the collection (i.e., DefaultRequestHeaders) I would expect these to be always sent. Is this a bug or a behavior to be expected?

To provide a bit more context to the failure I'm facing, I'm trying to call the Microsoft Graph getOffice365GroupsActivityDetail API and starting yesterday this API is forcing with a 301 http response code a redirect to location https://reportsweu.office.com/graph/v1.0/data/[tenant-id]/Microsoft.O365Reporting.getOffice365GroupsActivityDetail and the unexpected thing is that a different token seems to be required since making a call with the bearer token valid for calling the original API (i.e. https://graph.microsoft.com/v1.0/reports/getOffice365GroupsActivityDetail) fails with a "S2S auth failed" error.

whatever
  • 2,492
  • 6
  • 30
  • 42
  • What are you trying to do? What you ask would introduce a security vulnerability which is why most if not all tools and apps reset Authorization. Redirecting in general is performed by a middleware HttpClientHandler, [RedirectHandler](https://github.com/dotnet/runtime/blob/8c5cf110b4d2fa0b98cad4b60e4abba5a6562c58/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs). If you really, really, need to pass the `Authorization` header and have ways of mitigating against leaks, you could create your own handler – Panagiotis Kanavos Mar 21 '23 at 10:54
  • On the other hand, if you want to use JWT, API tokens, OpenID or any other established protocol, you don't need to reuse the Authorization header. – Panagiotis Kanavos Mar 21 '23 at 10:55
  • Another option, is to disable automatic redirects for that call, inspect the HttpResponseMessage and if the status is 302, redirect to the new `Location` URL. You should validate that URL though, and only try to reuse the `Authorization` header in safe calls, eg when calling the Login endpoint only – Panagiotis Kanavos Mar 21 '23 at 11:07
  • I'm calling a microsoft graph api (https://learn.microsoft.com/en-us/graph/api/reportroot-getoffice365groupsactivitydetail?view=graph-rest-1.0), passing the required jwt bearer token, and yesterday something that worked fine up to now started facing failures as this call is now forcing a redirect and the fun thing is that even following the redirect and making a new call passing along the previously retrieved token is now failing with a nice "S2S auth failed" error message :) – whatever Mar 21 '23 at 11:09
  • Are you sure your token hasn't expired? You aren't supposed to use the same token indefinitely. If there was a problem with the API *millions* of people would be affected. Why aren't you using Graph's own [NuGet package](https://learn.microsoft.com/en-us/graph/sdks/sdk-installation)? – Panagiotis Kanavos Mar 21 '23 at 11:35
  • Totally sure, and if you think about it, it could not be otherwise, this same token allows a successful call to the original API url, the one for which I get the redirect response. The thing that completely astonishes me is that the redirect location is under office.com which makes sense might require a different token but it would be the first time for me seeing that a redirect requires fetching a new authentication token :) – whatever Mar 21 '23 at 12:12
  • Why aren't you using the Graph SDK? You're trying to call an Office 365 so you need a token that validates you to Office 365's domain, ie `office.com`. It's not the redirect that needs a new token, it's the service. You'd get the same behavior with Google, as `docs.google.com` is in a different domain than `https://www.googleapis.com/auth/contacts` – Panagiotis Kanavos Mar 21 '23 at 12:22
  • Check for example's Google's docs on [Access Tokens](https://developers.google.com/identity/protocols/oauth2#4.-send-the-access-token-to-an-api.): *Access tokens are valid only for the set of operations and resources described in the scope of the token request. For example, if an access token is issued for the Google Calendar API, it does not grant access to the Google Contacts API.* – Panagiotis Kanavos Mar 21 '23 at 12:28
  • In general, OAuth/OpenID Connect (which is used by most cloud services) behaves differently for [different scenarios](https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-auth-code-flow). There are different flows to authenticate end users interactively, devices (eg phone), services etc. It *is* confusing, and I can't say I understand how each flow works either. Using a service's SDK shields you from knowing and implementing the details of working with OpenID up to a point. You still need to know enough to pick the correct flow, but not actually implement it – Panagiotis Kanavos Mar 21 '23 at 12:36
  • PS: When I say it's confusing, I still have trouble occasionally with a background service calling Google Sheets. At least I only have to call `.Authenticate()` instead of having to handle all the redirects and token storage myself – Panagiotis Kanavos Mar 21 '23 at 12:38
  • Guess what, using the graph sdk does not seem to shield you from anything ...... same issue with another graph api and this time the sdk is used :) https://learn.microsoft.com/en-us/answers/questions/1191714/graph-sdk-reports-getsharepointsiteusagedetail-not – whatever Mar 21 '23 at 12:53

2 Answers2

2

I just now discovered that this behavior is by design as the documentation on HttpClientHandler.AllowAutoRedirect Property states:

The Authorization header is cleared on auto-redirects and the handler automatically tries to re-authenticate to the redirected location. No other headers are cleared. In practice, this means that an application can't put custom authentication information into the Authorization header if it is possible to encounter redirection. Instead, the application must implement and register a custom authentication module.

To me this behavior is not intuitive as after the 301 redirect the HttpClient still shows the header as present in the DefaultRequestHeaders collection and yet it will not be sent when the redirect is followed. At this point it's probably better to add this header to the Headers collection on the HttpRequestMessage object other than to the DefaultRequestHeaders collection on the HttpClient object as this allows greater control over adding it or not to the new HttpRequestMessage object required for following the redirect.

whatever
  • 2,492
  • 6
  • 30
  • 42
  • Maybe file it as a bug on the Github repo for dotnet? – Charlieface Mar 20 '23 at 22:33
  • That's not a bug. The very fact that `Authorization` is explicitly cleared should tell you that keeping it is the real bug. In a typical API case, even if you were using Basic Authentication, you'd be redirected to a login URL and once authenticated you'd be redirected to the final page. If the final URL wasn't using HTTPS and the `Authorize` header retained the original content you'd leak your credentials at the very least. – Panagiotis Kanavos Mar 21 '23 at 10:33
-2

That's no bug at all. It's actually a common security measure against authentication leaks. Just like other tools, if you want to retain the header you'll have to do it explicitly.

This measure is used to prevent credential leaks. Without this, a hacked site could redirect you to a malicious 3rd party site, record the credentials in the Authorization header and then redirect you back to the final site.

.NET Framework works this way as well. There was a bug in .NET Core 2.0 (Not Stripping Auth in HttpClientHandler redirects by default) that didn't reset the Authorization header but that was fixed in 2.1

Other tools work the same way too, including curl. In CVE-2018-1000007 for example, they describe exactly this scenario:

When asked to send custom headers in its HTTP requests, curl will send that set of headers first to the host in the initial URL but also, if asked to follow redirects and a 30X HTTP response code is returned, to the host mentioned in URL in the Location: response header value.

Sending the same set of headers to subsequent hosts is in particular a problem for applications that pass on custom Authorization: headers, as this header often contains privacy sensitive information or data that could allow others to impersonate the curl-using client's request.

The solution was to reset the header unless the user explicitly forces the tool to reuse it :

custom Authorization: headers will be limited the same way other such headers is controlled within curl: they will only be sent to the host used in the original URL unless curl is told that it is ok to pass on to others using the CURLOPT_UNRESTRICTED_AUTH option.

Panagiotis Kanavos
  • 120,703
  • 13
  • 188
  • 236
  • The fact that it is not a bug was already cleared, I believe you saw my answer, right? :) Microsoft suggests at https://github.com/dotnet/runtime/issues/26475#issuecomment-432734436 as a possible solution pre-populating a CredentialsCache and passing it to the HttpClientHandler which I feel is nonsense if you cannot anticipate what redirects you might be facing. The other solution that is hinted (i.e. a custom authentication module) is on the other hand an obscure solution. How do you implement a custom authentication module in .NET 5.0 or greater? – whatever Mar 21 '23 at 10:57
  • @whatever it's not Microsoft, it's everyone. What are you trying to do in the first place? Why do you want to pass `Authorization` everywhere? Are you trying to use a JWT ? – Panagiotis Kanavos Mar 21 '23 at 11:01
  • @whatever the `what` matters - OpenID flows don't require retaining `Authorization` headers. You don't need a `a custom authentication module` that's a `don't use unless you know what you're doing` phrase. At the most basic level, you can check the response status code for 302 and if set, make a call to the new URL specified in `Location`. That's 2-3 lines, at a point where you know that authentication redirects are expected and you can verify the new URL. – Panagiotis Kanavos Mar 21 '23 at 11:05
  • 1
    Retaining the Authorization header might not be need but if I don't provide an Authorization header to the redirect endpoint I get a "Valid Authorization header is needed" error while, as already said, if I provide what was working fine up to yesterday I get an "S2S auth failed" error. The bearer token valid for calling the https://graph.microsoft.com/v1.0/reports/getOffice365GroupsActivityDetail endpoint does not seem to be what is required for calling the redirect endpoint https://reportsweu.office.com/graph/v1.0/data//Microsoft.O365Reporting.getOffice365GroupsActivityDetail. – whatever Mar 21 '23 at 11:48
  • Today things have changed once more and now the redirect is to a URL such as the following https://reportsweu.office.com/data/v1.0/download?token=[YOUR-TOKEN]. Thank you Microsoft :) – whatever Mar 22 '23 at 11:34
  • And yet, none of the millions of people using this has noticed anything. Post the code using the Graph SDK in the question. – Panagiotis Kanavos Mar 22 '23 at 11:40
  • 1
    Calm down, it has been noticed even by sdk consumers, https://learn.microsoft.com/en-us/answers/questions/1191714/graph-sdk-reports-getsharepointsiteusagedetail-not – whatever Mar 22 '23 at 11:46