Read FTP file with international characters

Question

I am downloading a FTP location file using below code. Its working for all of the files except where the file name contains international character.

I have learned in the URI format such this is not allowed but how can download if there is existing file at mentioned location.

For testing I have setup local FTP server under IIS like below.

http://www.online-tech-tips.com/computer-tips/setup-ftp-server-in-windows-iis/

string mat_address = "ftp://localhost/";
StringBuilder result = new StringBuilder();
FtpWebRequest ftp = (FtpWebRequest)WebRequest.Create(mat_address);

ftp.Credentials = new NetworkCredential("userid", "Password");
ftp.Method = WebRequestMethods.Ftp.ListDirectory;

string[] downloadfile = null;
using (FtpWebResponse response = (FtpWebResponse)ftp.GetResponse())
using (StreamReader reader = new StreamReader(response.GetResponseStream(), System.Text.Encoding.Default, true))
{
    downloadfile = reader.ReadToEnd().Split(new string[] { "\r\n" }, StringSplitOptions.RemoveEmptyEntries);
}

foreach (string d in downloadfile)
{
    if (d.Contains("d"))
    {
        string temp = mat_address + HttpUtility.UrlPathEncode(d);
        FtpWebRequest ftp2 = (FtpWebRequest)WebRequest.Create(temp);

        ftp2.Credentials = new NetworkCredential("userid", "Password");

        ftp2.Method = WebRequestMethods.Ftp.GetDateTimestamp;

        ftp2.UseBinary = true;

        ftp2.Proxy = null;
        ftp2.KeepAlive = false;
        ftp2.UsePassive = false;

        FtpWebResponse response2 = ftp2.GetResponse() as FtpWebResponse;
        DateTime temp1 = response2.LastModified.Date;
        if (temp1 > DateTime.Now.AddDays(-10))
        {
           // Some extra work
        }
    }
}

I am getting error

The remote server returned an error: (550) File unavailable (e.g., file not found, no access).

Below is my FTP root folder with problematic file name as diá.png

I am using C# for coding and Visual Studio 2013 for development. Whats going wrong can someone help.

Update to Question: Changes in Encoding to UTF8. Using the local host everything works fine. But when using the FTP server from international domain like germany and sweden. The name is read like below.

I am getting error for the line below.

FtpWebResponse response2 = ftp2.GetResponse() as FtpWebResponse;

Hex Value for File Name: Suggested include by Martin. Thanks 31,30,31,33,36,2D,49,43,4F,4D,20,50,4A,C4,54,54,45,52,59,44,20,70,69,63,74,20,37‌​,38,78,31,31,38,20,61,6E,6E,69,2D,76,65,72,73,61,72,69,75,73,20,5B,77,31,33,32,31‌​,20,78,20,68,39,32,31,5D,20,6D,6D,20,44,49,46,46,55,53,45,2E,50,4E,47,

Answer 1

Most FTP servers should use UTF-8 encoding. So does your local (IIS) server.

So you need to use the Encoding.UTF8, when parsing the directory listing.

---

Though your real/production server seems to be broken in some way. It looks like it uses Windows-1252 encoding for the directory listing. Yet it claims (and seems to require) UTF-8 encoding for commands. That clearly (and rightfully) confuses FileZilla. But I do not see, why it does not work with the FtpWebRequest as it should use the UTF-8 (as the server positively responds to OPTS utf8 on command), and you have tried to explicitly use the Windows-1252 encoding, when parsing the listing.

---

Anyway, as you found (in chat) that WinSCP works, you can try to use the WinSCP .NET assembly. It will also make your code a way simpler:

SessionOptions sessionOptions = new SessionOptions();
sessionOptions.Protocol = Protocol.Ftp;
sessionOptions.HostName = "hostname";
sessionOptions.UserName = "username";
sessionOptions.Password = "password";

using (Session session = new Session())
{
    session.Open(sessionOptions);

    foreach (RemoteFileInfo fileInfo in session.ListDirectory("/textures").Files)
    {
        if (fileInfo.Name.Contains("d"))
        {
            if (fileInfo.LastWriteTime > DateTime.Now.AddDays(-10))
            {
                string sourcePath =
                    RemotePath.EscapeFileMask("/textures/" + fileInfo.Name);
                session.GetFiles(sourcePath, @"c:\local\path\").Check();
            }
        }
    }
}

Or, even simpler, using file mask with time constraint:

SessionOptions sessionOptions = new SessionOptions();
sessionOptions.Protocol = Protocol.Ftp;
sessionOptions.HostName = "hostname";
sessionOptions.UserName = "username";
sessionOptions.Password = "password";

using (Session session = new Session())
{
    session.Open(sessionOptions);

    session.GetFiles("/textures/*d*>=10D", @"c:\local\path\").Check();
}

See also WinSCP example How do I transfer new/modified files only?

Answer 2

I'd say, you'll have to transform the encoding of the recieved filename to match the needs of your local file system. Could you post what filename you revieve? I think you get an excaped string containing some illegal characters...