HttpClient and WebClient return 503 server unavailable or 403 forbidden

Question

In C# I am trying to use either WebClient or HttpClient to retrieve text response from a URL. The URL works in a browser. However, when I use WebClient or HttpClient, I get either 503 or 403 errors with an http service I want to use.

I tested my WebClient and HttpClient routines with typical URLs and the routines work as expected.

The service URL is: "http://api.db-ip.com/v2/free/" with appended IP Address to complete the URL.

For example: "http://api.db-ip.com/v2/free/1.10.16.5"

The result displayed in a browser for the example is plain text:

{
    "ipAddress": "1.10.16.5",
    "continentCode": "AS",
    "continentName": "Asia",
    "countryCode": "CN",
    "countryName": "China",
    "stateProv": "Guangdong",
    "city": "Guangzhou Shi"
}

I have tried various suggested answers to this same problem in other postings. However, for this site, the problem persists. I verified I am well within the acceptable use of the service - and the URLs work in a browser on my development PC and work with curl.

The following is my typical C# code using WebClient. My HttpClient code is somewhat more complex involving async Task and await; but the results are about the same.

using (WebClient client = new WebClient())
{
    string testURL = "http://api.db-ip.com/v2/free/1.10.16.5"
    string testResult = ""
    try
    {
        // tried Proxy, Encoding, and user-agent settings, no difference
        // client.Proxy = null;
        // client.Encoding = Encoding.UTF8;
        // client.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)");
        testResult = client.DownloadString(testURL);
    }
    catch (Exception x)
    {
        Console.WriteLine("Error getting result from " + testURL);
        Console.WriteLine("Response Text> " + testResult);
        Console.WriteLine(x.Message);
        if (x.InnerException != null)
        {
            Console.WriteLine(x.InnerException.Message);
        }
    }
}

I tried various WebClient parameters without success. I expect that I am missing the correct parameters to make it work.

-=-=-=- Update: The following HttpClient method works (thanks @mxmissile)

// HttpClient instance lifecycle should be the Application's lifecycle per .NET recommendation
private static readonly HttpClient myHttpClient = new HttpClient();

public async Task<string> GeoLocationText_byIpAddressAsync(string ipAddress)
{
    string testURL = "http://api.db-ip.com/v2/free/1.10.16.5"
    string testResult = ""
    try
    {
        myHttpClient .DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0");
        string testResult = await myHttpClient .GetStringAsync(testURL );
    }
    catch (Exception x)
    {
        Console.WriteLine("Error getting result from " + testURL);
        Console.WriteLine("Response Text> " + testResult);
        Console.WriteLine(x.Message);
        if (x.InnerException != null)
        {
            Console.WriteLine(x.InnerException.Message);
        }
    }
}

The *free* quota is 1000 queries per day. Since I have not gotten it working yet, I may have tried about 10 to 20 IP addresses. Also, no problems using a web browser or curl — gridtrak, Jul 14 '21 at 17:24
Adding the UserAgent like @Riwen mentioned worked for me: https://gist.github.com/mxmissile/a32ceeaeda6f63365d9930d7399497e8 — mxmissile, Jul 14 '21 at 17:40
IT WORKS with @Riwen's User Agent suggestion. THANKS to both of you @mxmissile!!! — gridtrak, Jul 14 '21 at 19:03

score 0 · Answer 1 · answered Jul 14 '21 at 19:08

The following code works for me now using the the .UserAgent property described by @mxmissile and @Riwen. They should get credit for this answer.

// HttpClient instance lifecycle should be the Application's lifecycle per .NET recommendation
private static readonly HttpClient myHttpClient = new HttpClient();

public async Task<string> GeoLocationText_byIpAddressAsync(string ipAddress)
{
    string testURL = "http://api.db-ip.com/v2/free/1.52.235.161"
    string testResult = ""
    try
    {
        myHttpClient .DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0");
        string testResult = await myHttpClient .GetStringAsync(testURL );
    }
    catch (Exception x)
    {
        Console.WriteLine("Error getting result from " + testURL);
        Console.WriteLine("Response Text> " + testResult);
        Console.WriteLine(x.Message);
        if (x.InnerException != null)
        {
            Console.WriteLine(x.InnerException.Message);
        }
    }
}

The testResult is:

{
    "ipAddress": "1.52.235.161",
    "continentCode": "AS",
    "continentName": "Asia",
    "countryCode": "VN",
    "countryName": "Vietnam",
    "stateProv": "Hanoi",
    "city": "Hanoi"
}

HttpClient and WebClient return 503 server unavailable or 403 forbidden

1 Answers1