HttpWebResponse is wrong for the ContentLength

Question

I have a download method for HTTP protocol. But it seems it doesnt work correctly, something is wrong. I tested it with some url sources and it was correct except the last one. The ContentLength property is wrong for the url. It is shown as 210 kb in runtime, but it is 8 MB in fact. I will show it by sharing my code. How to fix it?

Code:

    void TestMethod(string fileUrl)
    {
        HttpWebRequest req = WebRequest.Create(fileUrl) as HttpWebRequest;
        HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
        long contentSize = resp.ContentLength;

        MessageBox.Show(contentSize.ToString());
    }
    private void TestButton_Click(object sender, EventArgs e)
    {
        string url1 = "http://www.calprog.com/Sounds/NealMorseDowney_audiosample.mp3";
        string url2 = "http://www.stephaniequinn.com/Music/Canon.mp3";

        TestMethod(url1); //This file size must be 8 MB, but it shows up as 210 kb. This is the problem

        TestMethod(url2); //This file size is correct here, about 2.1 MB 
    }

score 1 · Accepted Answer · answered Sep 18 '16 at 18:59

I think you are not allowed to access this url this way (with HttpWebRequest).

if you try to get the response text :

    HttpWebRequest req = WebRequest.Create(fileUrl) as HttpWebRequest;
    HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
    using (var streamreader = new StreamReader(resp.GetResponseStream()))
    {
        var r = streamreader.ReadToEnd();
        long contentSize = r.Length;
        Console.WriteLine(contentSize.ToString());
    }

You ll get this response :

<html><head><title>Request Rejected</title></head><body>The requested URL was rejected. If you think this is an error, please contact the webmaster. <br><br>Your support ID is: 2719994757208255263</body></html>

You must set the UserAgent to be able to get the full response. Like this :

req.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0";

By setting this value the server will think your program is a Firefox browser.

So these few lines of code should do the trick :

   void TestMethod(string fileUrl)
   {
       HttpWebRequest req = WebRequest.Create(fileUrl) as HttpWebRequest;
       req.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0";
       HttpWebResponse resp = req.GetResponse() as HttpWebResponse;
       long contentSize = resp.ContentLength;
       Console.WriteLine(contentSize.ToString());
    }

Have a good day !

It works fine. I didn't like using `StreamReader` because it waits the file to be downloaded. Setting `UserAgent` is better and can return the correct size. Lastly, does this Mozilla `UserAgent` works for all Http sources? Because I need it to be generalized. Thanks @Quentin. — Ali Tor, Sep 18 '16 at 19:12
Yes this user agent will works with all http sources. (If the url can be reachable with a standard browser) — Quentin Roger, Sep 18 '16 at 19:27

HttpWebResponse is wrong for the ContentLength

1 Answers1