Unicode data in web response header

Question

I have developed web api which accept files using POST method, makes manipulation and return them back using HTTP Response. The web api return additional data in http header like output file name. The problem is that then I am posting and receiving response with HttpWebResponse I get scrambled file name in response header value and unicode characters are lost.

For example if I submit наталья.docx file I get наÑалÑÑ.pdf.

The full response header

Pragma: no-cache
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Result: True
StoreFile: false
Timeout: 300
OutputFileName: наÑалÑÑ.pdf
Content-Disposition: attachment; filename=наÑалÑÑ.pdf
Cache-Control: no-cache, no-store
Content-Type: application/pdf
Date: Wed, 12 Sep 2012 07:21:37 GMT
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4

I am reading header values like this

HttpWebResponse webResponse = FormUpload.MultipartFormDataPost(postdatatoserver);
using (Stream clientResponse = webResponse.GetResponseStream())
if (webResponse.StatusCode == HttpStatusCode.OK)
{
   Helpers.CopyStream(clientResponse, outStream);
   webHeaderCollection = webResponse.Headers;
}

I am not sure should I just decode scrambled characters to unicode when I read them from response header or maybe I need to include encoding into response header when I send data from web api server?

Answer 1

See http://msdn.microsoft.com/en-us/library/system.net.webresponse.getresponsestream.aspx:

Stream ReceiveStream = myWebResponse.GetResponseStream();
Encoding enc = System.Text.Encoding.UTF8;

// Pipe the stream to a higher level stream reader with the required encoding format. 
StreamReader readStream = new StreamReader(ReceiveStream, enc);

You might also try

System.Text.Encoding.Default
or
System.Text.Encoding.UTF7
or
System.Text.Encoding.Unicode
or 
System.Text.Encoding.GetEncoding(1251)
or 
System.Text.Encoding.GetEncoding(1252)
or
System.Text.Encoding.GetEncoding(20866)

See here for a longer list:
http://www.pcreview.co.uk/forums/system-text-encoding-getencoding-whatvalidstrings-t1406242.html

Edit:

Current [RFC 2045] grammar restricts parameter values (and hence Content-Disposition filenames) to US-ASCII.

So the HTTP-Headers are always transmitted in ASCII format, irrespective of the StreamReader encoding.
IE doesn't conform to the standard, so there is a workaround: UrlEncode the filename

So you need to do this when you write the file back:

// IE needs url encoding, FF doesn't support it, Google Chrome doesn't care
if (Request.Browser.IsBrowser ("IE"))
{
    fileName = Server.UrlEncode(fileName);
}

Response.Clear ();
Response.AddHeader ("content-disposition", String.Format ("attachment;filename=\"{0}\"", fileName));
Response.AddHeader ("Content-Length", data.Length.ToString (CultureInfo.InvariantCulture));
Response.ContentType = mimeType;
Response.BinaryWrite(data);

As per Unicode in Content-Disposition header you can add an asterisk, and append the proper encoding.