Trouble with ASP.NET HttpWebRequest POST to a site

Question

ASP.NET 4.0

Need some help with this vexing HTTP POST problem - I have looked at other posts on Stackoverflow but to no avail.

Problem summary: It's a classic case - I want to login to an external site which takes 2 parameters to login, and I need to use a POST to do it

What happens: I do a POST and the HTTP response that comes back is essentially the same page that I posted to in the first place (i.e. it hasn't really logged in)

Things I've done: I have fiddler (protocol analyzer) running, and I have comparisons of my POST and a working POST (from another desktop app), but I can't seem to reproduce the same behavior

[edit 1]: It appears to be a cookie issue (the code below is now outdated, I've made modifications), I have managed to set all the parameters correctly. Problem not resolved as of this edit.

Below is my code and the I have also reproduced the comparison headers

 private static void doPost(string URL)
    {
        HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(URL);
        myRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.126 Safari/533.4";
        CookieContainer cCookie = new CookieContainer();
        myRequest.CookieContainer = cCookie;
        myRequest.ContentType = "application/x-www-form-urlencoded";
        myRequest.Accept = "application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";            

        string postData = "param1=somevalue&param2=someothervalue";
        ASCIIEncoding ascii = new ASCIIEncoding();
        byte[] bData = ascii.GetBytes(postData);

        myRequest.Method = "POST";
        myRequest.ContentLength = bData.Length;

        Stream oStream = myRequest.GetRequestStream();
        oStream.Write(bData, 0, bData.Length);

        string oResp = string.Empty;

        using (var resp = myRequest.GetResponse())
        {
            using (var responseStream = resp.GetResponseStream())
            {
                using (var responseReader = new StreamReader(responseStream))
                {
                    oResp = responseReader.ReadToEnd();
                }
            }
        }
        Console.WriteLine(oResp);        
    }

I do get a HTTP 1.1 OK but the response text is the same page I posted to - i.e. the login page, which suggests that my post did not actually succeed.

Now here are the comparison POSTS

a) FROM MY APPLICATION (DOESN'T WORK)

POST https://[someURL] HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.126 Safari/533.4
Content-Type: application/x-www-form-urlencoded
Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,/;q=0.5
Host: xxxx.com
Content-Length: 64
Expect: 100-continue
Connection: Keep-Alive

param1=value1&param2=value2


b) FROM THE POST THAT WORKS (A DIFFERENT DESKTOP APP DOING THE SAME THING)

POST https:[someURL] HTTP/1.1
Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/x-shockwave-flash, /
Referer: [someURL]
Accept-Language: en-US
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; InfoPath.2; .NET4.0C; .NET4.0E)
Host: xxxxxx.com
Content-Length: 2815
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: ccccc1=10; blahblah=pppqqqrrr; ASP.NET_SessionId=jp4mjg45z34si545om3nouew



SomeField=False&__VIEWSTATE1=1H4sIAA3mnn8A%2F31WTbequfx[TRUNCATED]jF%2FLkgCgAA&__VIEWSTATE0=3&__VIEWSTATE=&__VIEWSTATE=&param1=value1&param2=value2&x=0&y=0

As you can see, the second POST is much bigger - the key differences are

1. There are some cookies, whereas mine don't show any
2. The POST header shows these additional fields (SomeField, VIEWSTATE1, etc.) - how I do access them to do the same?

Do I first need to do a GET, parse the body, extract these VIEWSTATE1--3 & SomeField, and then reset it in the post body data bytes? I could try many things but it would be much appreciated if someone tells me if I'm way off track or am doing something fundamentally wrong...

I plan to use the Html Agility pack to parse the HTML

thanks a bunch, g

Answer 1

The problem is probably that you need to supply a session cookie with your login request.

What you need to do is first make a normal GET request to the login page. Add all response cookies to your CookieContainer and then use the same CookieContainer when making the actual POST login request.

Also, try and mimic the real request as much as possible. Use the same Referer value etc.

Have a look at this question logged in to PureVolume.com programatically !

Edit: Depending on how the site you are trying to login to is designed you might need to parse and supply the ViewState data as you suggested. But since an initial GET request most likely already is required this shouldn't be to hard to do.