2

IIS site configured to use windows authentication with default options. The client is written in C# and uses single HttpClient instance to perform requests. Requests success, but every request triggers 401 Challenge:

enter image description here

Traffic captured with Wireshark. We performed loud test, and noticed, that with anonymous authentication client performs 5000 reqeusts per second, but with windows authentication - 800. So, looks like wireshark does not impact to authentication, performance slowdown indicates, that 401 Challenge also occurs without wireshark.

Wirehshark log: https://drive.google.com/file/d/1vDNZMjiKPDisFLq6ZDhASQZJJKuN2cpj/view?usp=sharing

Code of client is here:

var httpClientHandler = new HttpClientHandler();
httpClientHandler.UseDefaultCredentials = true;
var httpClient = new HttpClient(httpClientHandler);
while (working)
{
    var response = await httpClient.GetAsync(textBoxAddress.Text + "/api/v1/cards/" + cardId);
    var content = await response.Content.ReadAsStringAsync();
}

IIS site settings:

enter image description here

enter image description here

enter image description here

How to make HttpClient to persist authentication between requests, to prevent waste negotiate handshakes on every request?

UPD: Code of client: https://github.com/PFight/httpclientauthtest Steps to reproduce:

  1. Create folder with simple file index.html, create application 'testsite' in IIS for this folder. Enable anonymous authentication.
  2. Run client (https://github.com/PFight/httpclientauthtest/blob/main/TestDv5/bin/Debug/TestDv5.exe), press start button - see count of requests per second. Press stop.
  3. Disable anonymous atuhentication, enable windows authentication.
  4. Press start button in client, see count of reqeusts per second.

On my computer I see ~1000 requests per second on anonymous, and ~180 on windows. Wireshark shows 401 challenges on every request for windows authentication. keep-alive header enabled in IIS.

IIS version: 10.0.18362.1 (windows 10)

Version of System.Net.Http.dll loaded to process: 4.8.3752.0

Pavel
  • 2,602
  • 1
  • 27
  • 34

1 Answers1

2

Firstly I tried to save the Authorization header for re-use it with every new request.

using System.Net.Http.Headers;

Requester requester = new Requester();
await requester.MakeRequest("http://localhost/test.txt");
await Task.Delay(100);
await requester.MakeRequest("http://localhost/test.txt");

class Requester
{
    private readonly HttpClientHandler _httpClientHandler;
    private readonly HttpClient _httpClient;
    private AuthenticationHeaderValue _auth = null;

    public Requester()
    {
        _httpClientHandler = new HttpClientHandler();
        _httpClientHandler.UseDefaultCredentials = true;
        _httpClient = new HttpClient(_httpClientHandler);
        _httpClient.DefaultRequestHeaders.Add("User-Agent", Guid.NewGuid().ToString("D"));
    }

    public async Task<string> MakeRequest(string url)
    {
        HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Get, url);
        message.Headers.Authorization = _auth;

        HttpResponseMessage resp = await _httpClient.SendAsync(message);
        _auth = resp.RequestMessage?.Headers?.Authorization;
        resp.EnsureSuccessStatusCode();
        string responseText = await resp.Content.ReadAsStringAsync();
        return responseText;
    }
}

But it didn't work. Every time there was http code 401 asking for authentication despite of Authorization header.

The IIS logs is listed below.

2021-12-23 15:07:47 ::1 GET /test.txt - 80 - ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 401 2 5 127
2021-12-23 15:07:47 ::1 GET /test.txt - 80 MicrosoftAccount\account@domain.com ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 200 0 0 4
2021-12-23 15:07:47 ::1 GET /test.txt - 80 - ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 401 1 2148074248 0
2021-12-23 15:07:47 ::1 GET /test.txt - 80 MicrosoftAccount\account@domain.com ::1 c75eeab7-a0ea-4ebd-91a8-21f5cd59c10f - 200 0 0 0

IIS's Failed Requests Tracing reports the following when receiving re-used Authentication Header:

Property Value
ModuleName WindowsAuthenticationModule
Notification AUTHENTICATE_REQUEST
HttpStatus 401
HttpReason Unauthorized
HttpSubStatus 1
ErrorCode The token supplied to the function is invalid (0x80090308)

I've made a research and I can say that this is not possible without alive connection.

Every time the connection is closed there will be new handshake.

According this and this answers NTLM authenticates a connection, so you need to keep your connection open.

NTLM over http is using HTTP persistent connection or http keep-alive.

A single connection is created and then kept open for the rest of the session.

If using the same authenticated connection, it is not necessary to send the authentication headers anymore.

This is also the reason why NTLM doesn't work with certain proxy servers that don't support keep-alive connections.


UPDATE:

I found the key point using your example.

First: You must enable keep-alive at your IIS

Second: You must set authPersistSingleRequest flag to false. Setting this flag to True specifies that authentication persists only for a single request on a connection. IIS resets the authentication at the end of each request, and forces re-authentication on the next request of the session. The default value is False. authPersistSingleRequest

Third: You can force the HttpClient to send keep-alive headers:

httpClient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpClient.DefaultRequestHeaders.Add("Keep-Alive", "600");

Using this three key points I have achieved only one NTLM handshake during connection lifetime. Fiddler screenshot

Also it's important which version of .NET \ .NET Framework do you use. Because HttpClient hides different realizations dependent on framework version.

Framework Realization of HttpClient
.Net Framework Wrapper around WebRequest
.Net Core < 2.1 Native handlers (WinHttpHandler / CurlHandler)
.Net Core >= 2.1 SocketsHttpHandler

I tried it on .NET 6 and it works great, but it didn't work on .Net Framework as I can see, so here is the question: which platform do you use?

UPDATE 2:

Found the solution for .Net Framework.

CredentialCache myCache = new CredentialCache();
WebRequestHandler handler = new WebRequestHandler()
{
    UseDefaultCredentials = true,
    AllowAutoRedirect = true,
    UnsafeAuthenticatedConnectionSharing = true,
    Credentials = myCache,
};
var httpClient = new HttpClient(handler);

httpClient.DefaultRequestHeaders.Add("Connection", "keep-alive");
httpClient.DefaultRequestHeaders.Add("Keep-Alive", "600");

var from = DateTime.Now;
var countPerSecond = 0;
working = true;
while (working)
{
    var response = await httpClient.GetAsync(textBoxAddress.Text);
    var content = await response.Content.ReadAsStringAsync();
    countPerSecond++;
    if ((DateTime.Now - from).TotalSeconds >= 1)
    {
        this.labelRPS.Text = countPerSecond.ToString();
        countPerSecond = 0;
        from = DateTime.Now;
    }

    Application.DoEvents();
}

The key point is to use WebRequestHandler with UnsafeAuthenticatedConnectionSharing enabled option and use credential cache.

If this property is set to true, the connection used to retrieve the response remains open after the authentication has been performed. In this case, other requests that have this property set to true may use the connection without re-authenticating. In other words, if a connection has been authenticated for user A, user B may reuse A's connection; user B's request is fulfilled based on the credentials of user A.

Caution Because it is possible for an application to use the connection without being authenticated, you need to be sure that there is no administrative vulnerability in your system when setting this property to true. If your application sends requests for multiple users (impersonates multiple user accounts) and relies on authentication to protect resources, do not set this property to true unless you use connection groups as described below.

Big thanks to this article for solution.

wireshark screenshot

Georgy Tarasov
  • 1,534
  • 9
  • 11
  • Thanks, for research, you confirmed our suggestion, that keeping tcp connection is only the way. The answer will be to determine, how to enable http keep-alive for HttpClient + IIS. All docs says, that HttpClient shoud keep connections by maximum, but as you can see - it does not. – Pavel Dec 23 '21 at 16:14
  • @Pavel , I used [this](https://support.optimidoc.com/073694-How-to-turn-off-Keep-Alive-in-IIS-server) article to disable the keep-alive on IIS, because with enabled keep-alive it worked as expected: only one handshake during exchange. I thought that disabled keep-alive is requirement of your IIS setup. – Georgy Tarasov Dec 23 '21 at 16:48
  • IIS logs with enabled keep-alive: 2021-12-23 13:43:21 ::1 GET /test.txt - 80 - ::1 e047713d-3eeb-4ff6-93ac-3311d33c5d85 - 401 2 5 0 2021-12-23 13:43:21 ::1 GET /test.txt - 80 MicrosoftAccount\account@domain.com ::1 e047713d-3eeb-4ff6-93ac-3311d33c5d85 - 200 0 0 0 2021-12-23 13:43:21 ::1 GET /test.txt - 80 MicrosoftAccount\account@domain.com ::1 e047713d-3eeb-4ff6-93ac-3311d33c5d85 - 200 0 0 0 – Georgy Tarasov Dec 23 '21 at 16:49
  • Thanks, I will try create more simple sample. If you doesn't experiense problem with keep-alive, then may be I will find difference between simpliest site and ours. – Pavel Dec 24 '21 at 09:30
  • Updated question, shared code of client. Problem reproduced on a simple index.html with keep-alive header. Please, try https://github.com/PFight/httpclientauthtest/blob/main/TestDv5/bin/Debug/TestDv5.exe with specified scenario on your machine. Will you reproduce problem? – Pavel Dec 27 '21 at 07:44
  • Thanks, looks like reason in [UnsafeAuthenticatedConnectionSharing](https://learn.microsoft.com/en-us/dotnet/api/system.net.httpwebrequest.unsafeauthenticatedconnectionsharing?redirectedfrom=MSDN&view=net-6.0#System_Net_HttpWebRequest_UnsafeAuthenticatedConnectionSharing), wich is default false. So, if HttpClient uses HttpWebRequest under the hood, then it does not keep NTLM connection at all.. Using WebClient with overriden GetWebRequest with UnsafeAuthenticatedConnectionSharing = true solved the problem on .Net. – Pavel Dec 27 '21 at 10:24
  • Great! Thanks a lot! Last answer update works with HttpClient! – Pavel Dec 27 '21 at 10:31