REST service switched to https, works in browser, throwing a "(403) Forbidden" error in c#

Question

NOAA recently switched their services from http to https and a c# call that has been working for years is now returning a "The remote server returned an error: (403) Forbidden."

Strangely enough the same call works from a browser and from Postman. Why would the server reject one request and not another, what am I missing?

URL: https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?zipCodeList=44113&product=time-series&begin=2017-03-23T00:00:00&temp=temp&appt=appt&pop12=pop12

Sample code revised per the accepted answer below. Both versions were never setting the UserAgent and apparently this is now required:

string xml = "";
string url = "";
try
{
    using (System.Net.WebClient wc = new System.Net.WebClient())
    {
        url = "https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?zipCodeList=44113&product=time-series&begin=2017-03-23T00:00:00&temp=temp&appt=appt&pop12=pop12";
        wc.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 1.1.4322; .NET CLR 3.5.20404)");
        xml = wc.DownloadString(new Uri(url));
    }
    //......  
}
catch (Exception ex)
{
    LogError(ex);
}

or this

string xml = ""; 
string url = "";
url = "https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?zipCodeList=44113&product=time-series&begin=2017-03-23T00:00:00&temp=temp&appt=appt&pop12=pop12";
HttpWebRequest httpWR = (HttpWebRequest)WebRequest.Create(url);
httpWR.Method = WebRequestMethods.Http.Get;
httpWR.Accept = "application/xml";
httpWR.UserAgent = ".NET Framework Client";
try
{
    using (HttpWebResponse response = (HttpWebResponse)httpWR.GetResponse())
    {
        using (StreamReader reader = new StreamReader(response.GetResponseStream()))
        {
            xml = reader.ReadToEnd();
        }
    }
    //......
}
catch (Exception ex)
{
    LogError(ex);
}

scartag · Answer 1 · 2017-03-23T19:03:39.880

3

I think the service requires a valid user agent.

I've modified your code to include that.

using (System.Net.WebClient wc = new System.Net.WebClient())
        {
            url = "https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?zipCodeList=44113&product=time-series&begin=2017-03-23T00:00:00&temp=temp&appt=appt&pop12=pop12";

            wc.Headers.Add("user-agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 1.1.4322; .NET CLR 3.5.20404)");


            xml = wc.DownloadString(url);
        }

edited Mar 23 '17 at 19:03

answered Mar 23 '17 at 18:54

scartag

17,548
3
48
52

@Plamen Kouzov i've tested and updated my answer. it works. – scartag Mar 23 '17 at 19:04
@Plamen Kouzov please put the user agent call before you call `wc.DownloadString(url)` i have it working without any errors. What error are you getting now? – scartag Mar 23 '17 at 19:15
I have ServicePointManager.ServerCertificateValidationCallback = delegate { return true; }; just before xml = wc.DownloadString(new Uri(url)); I wonder if they did not blacklist us for some reason. – Plamen Kouzov Mar 23 '17 at 19:20
@Plamen Kouzov Oh no! please refresh the page. i have since changed the answer to something else. That initial one was wrong. – scartag Mar 23 '17 at 19:21
Perfect! I wish I could upvote your answer a few more times. – Plamen Kouzov Mar 23 '17 at 19:37
@Plamen Kouzov Glad it helped. Would be nice if you could accept the answer please :) – scartag Mar 23 '17 at 20:03

REST service switched to https, works in browser, throwing a "(403) Forbidden" error in c#

1 Answers1