StatusCode for GetResponse() is OK but url Redirect

Question

When executing the following lines of code:

string link = "https://finance.yahoo.com/r/4951f719-c8e1-3b1d-b4db-684ef6739b8e/iphone-x-5-reasons-to-wait?utm_source=yahoo&utm_medium=partner&utm_campaign=yahootix&partner=yahootix&yptr=yahoo&.tsrc=rss"
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create(link);    
myHttpWebRequest.MaximumAutomaticRedirections=100;
myHttpWebRequest.AllowAutoRedirect=true;
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();

I get StatusCode "OK" and ResponseUri is still the same original URL and not the Redirected one.

How can I get the last redirected URL?

Answer 1

AllowAutoRedirect property works only if URL is redirected by HTTP protocol (when server returns HTTP code 30x). But this URL is redirected by javascript (or meta http-equiv='refresh' tag, if javascript is not supported by browser) contained in HTML page. So you must parse HTML page content and read URL from it. Following code uses HtmlAgilityPack library (available as NuGet package) to parse HTML page and read desired URL:

string link = "https://finance.yahoo.com/r/4951f719-c8e1-3b1d-b4db-684ef6739b8e/iphone-x-5-reasons-to-wait?utm_source=yahoo&utm_medium=partner&utm_campaign=yahootix&partner=yahootix&yptr=yahoo&.tsrc=rss";
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(link);
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();

using (var responseStream = myHttpWebResponse.GetResponseStream())
{
    using (var sr = new StreamReader(responseStream))
    {
        //content of HTML page
        var html = sr.ReadToEnd();

        //using HTMLAgilityPack library to parse HTML page
        var htmlDocument = new HtmlAgilityPack.HtmlDocument();
        htmlDocument.LoadHtml(html);

        //find URL in HTML page. Null-checks omitted for simplicity
        var contentAttribute = htmlDocument.DocumentNode.SelectSingleNode("noscript/meta").Attributes["content"].Value;
        var URL = contentAttribute.Split(';')[1];//skip "0;" in contentAttribute
        URL = URL.Substring(4);//skip 'URL='
        URL = URL.Trim('\'');//remove surrounding '
    }
}