How to prevent character unescaping with WebClient or HttpWebRequest or any .NET client?

Question

I am making a request to http://translate.google.com/translate_tts?q=da%C3%B1o&tl=es-ES to get an audio recording. When I use System.Net.WebClient, System.Net.Http.HttpClient or WebRequest.Create, the request is sent as http://translate.google.com/translate_tts?q=daño&tl=es-ES instead. This second url works, but produces a incorrect result.

How can I send a request with %C3%B1 instead of ñ in the path?

Note that you cannot download these urls with a referral header (like, by clicking them) so if you click them they will fail and the result may be cached so that copy'n'pasting the url fails too. If you want to test the urls, copy them to the clipboard, then add to the q parameter to avoid cached results.

Adding repro where I tried using PUrify (same results as without):

            var parameters = this.Bind<SpeechRequest>();
            parameters.q = "daño";
            parameters.tl = "es-MX";

            var url = String.Format("http://translate.google.com/translate_tts?q={0}&tl={1}", 
                WebUtility.UrlEncode(parameters.q), WebUtility.UrlEncode(parameters.tl));

            var uri = new Uri(url);
            System.Console.WriteLine("Uri constructed from string: " + url);
            System.Console.WriteLine("Before Purify: ");
            ShowUriDetails(uri);
            uri.Purify();
            System.Console.WriteLine("After Purify:");
            ShowUriDetails(uri);

            var request = System.Net.WebRequest.Create(uri) as HttpWebRequest;

Output:

    Uri constructed from string: http://translate.google.com/translate_tts?q=da%C3%B1o&tl=es-MX
    Before Purify:
            uri.ToString() - http://translate.google.com/translate_tts?q=daño&tl=es-MX
            uri.AbsoluteUri - http://translate.google.com/translate_tts?q=da%C3%B1o&tl=es-MX
            uri.Host - translate.google.com
            uri.Query - ?q=da%C3%B1o&tl=es-MX
            uri.PathAndQuery - /translate_tts?q=da%C3%B1o&tl=es-MX
            uri.AbsolutePath - /translate_tts
            uri.Fragment -
    After Purify:
            uri.ToString() - http://translate.google.com/translate_tts?q=daño&tl=es-MX
            uri.AbsoluteUri - http://translate.google.com/translate_tts?q=da%C3%B1o&tl=es-MX
            uri.Host - translate.google.com
            uri.Query - ?q=da%C3%B1o&tl=es-MX
            uri.PathAndQuery - /translate_tts?q=da%C3%B1o&tl=es-MX
            uri.AbsolutePath - /translate_tts
            uri.Fragment -

maybe I'm missing something here, but would htmlencoding the url be enough to fix it? https://msdn.microsoft.com/en-us/library/w3te6wfz(v=vs.110).aspx — user2366842, Jul 15 '15 at 19:01
I just used WebRequest and the URL is not changed. `GET http://translate.google.com/translate_tts?q=da%C3%B1o&tl=es-ES HTTP/1.1` — Brian from state farm, Jul 15 '15 at 19:25
@user2366842 I'll put in sample code to show how I am url encoding the special characters — Frank Schwieterman, Jul 15 '15 at 20:09
@Brianfromstatefarm how did you observe it hadn't change? Which namespace is your WebRequest from? — Frank Schwieterman, Jul 15 '15 at 20:10
Oh geez. After looking at it closer in Fiddler, I found the problem was the different in User-Agent :( — Frank Schwieterman, Jul 15 '15 at 20:34

How to prevent character unescaping with WebClient or HttpWebRequest or any .NET client?

0 Answers0