6

I am trying to use the System.Net.HttpWebRequest class to perform a HTTP GET request to a specific web server for web applications we have load balanced over numerous servers. In order to achieve this, I need to be able to set the Host header value for the request, and I have been able to achieve this by using the System.Net.WebProxy class.

However, this all breaks down when I try to perform a GET using SSL. When I attempt to do this, the call to HttpWebRequest.GetResponse throws a System.Net.WebException, with a HTTP status code of 400 (Bad Request).

Is what I'm trying to achieve possible with HttpWebRequest, or should I be looking for an alternative way to perform what I want?

Here is the code I've been using to try and get this all to work :-

using System;
using System.Web;
using System.Net;
using System.IO;

namespace UrlPollTest
{
    class Program
    {
        private static int suffix = 1;
        static void Main(string[] args)
        {
            PerformRequest("http://www.microsoft.com/en/us/default.aspx", "www.microsoft.com");
            PerformRequest("https://www.microsoft.com/en/us/default.aspx", "");
            PerformRequest("https://www.microsoft.com/en/us/default.aspx", "www.microsoft.com");

            Console.WriteLine("Press any key to continue");
            Console.ReadKey();
        }

        static void PerformRequest(string AUrl, string AProxy)
        {
            Console.WriteLine("Fetching from {0}", AUrl);
            try
            {
                HttpWebRequest request = WebRequest.Create(AUrl) as HttpWebRequest;
                if (AProxy != "")
                {
                    Console.WriteLine("Proxy = {0}", AProxy);
                    request.Proxy = new WebProxy(AProxy);
                }

                WebResponse response = request.GetResponse();
                using (Stream httpStream = response.GetResponseStream())
                {
                    using (StreamReader reader = new StreamReader(httpStream))
                    {
                        string s = reader.ReadToEnd();
                        File.WriteAllText(string.Format("D:\\Temp\\Response{0}.html", suffix++), s);
                    }
                }
                Console.WriteLine("  Success");
            }
            catch (Exception e)
            {
                Console.WriteLine("  " + e.Message);
            }
        }
    }
}
Cleggy
  • 715
  • 9
  • 24
  • Genuine question: How do you know you need a host-header? "HTTP 1.1 host headers are not supported when you use SSL" http://support.microsoft.com/kb/187504. – russau Aug 06 '09 at 03:46
  • Second thought: can you get the file over SSL in a browser? – russau Aug 06 '09 at 03:52
  • 1
    I need to set a Host header so I can specify the exact server to hit if the web application is load balanced across multiple servers. This code is part of a web monitoring and testing framework I'm writing. And yes, the SSL requests I'm attempting to make do work in the browser, so the problem lies somewhere in the way I am attempting to implement similar functionality in code. The article you linked suggests that Host header support for SSL requests was made available in Windows Server 2003 SP1, and all our servers should be fully patched. – Cleggy Aug 06 '09 at 04:03
  • There's an artile about SSL and host header's here: tinisles.blogspot.com/2008/11/… IIS still only allows one certificate per IP address - but that certificate can be a wildcard, i.e *.mydomain.com. Which would allow abc.mydomain.com and def.mydomain.com to be on the same IP differentiated by host header. That said - this probably isn't related to your problem. – russau Aug 06 '09 at 04:14
  • Yes, our servers are definitely patched beyond Windows Server 2003 SP1. – John Kaster Aug 06 '09 at 04:16

2 Answers2

2

I've used a hacky approach before - of having a test exe that modifies my "hosts" file, injecting an entry for the farm name to the specific server's address (and issuing an ipconfig /flushdns). After that, requests should get routed to the right server as though it were the only one there.

Obviously this requires admin access... I use it as part of an automated smoke-test to hit the farm as though it were individual machines.

The other thing you might try is something on the NLB, perhaps in TCL (in the case of F5) - maybe add a custom header that the NLB understands? (assuming it is doing SSL re-signing).

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • Thanks for the suggestion, but that won't work for us. We'll be running multiple threads at the same time, and there could definitely be HOSTS entry conflicts in that scenario. – John Kaster Aug 06 '09 at 04:29
0

In .NET 4.0 the Host header can be set independently of the URL that the request is going to so I would recommend that you use HttpwebRequest.Host to solve your problem. This (I think) will ship as part of .NET 4.0 beta2. There is NO way to re-write the host header to be different under previous versions of .Net without implementing your own custom subclass of WebRequest (believe me, it's been tried).

Jeff Tucker
  • 3,387
  • 22
  • 19