c# ftpwebrequest performance

Question

I have an application that download files from a Unix FTP server. It works fine, just have this performance problem: Files which size is <= 1K takes in average between 2084 and 2400 milliseconds to download, while applications like Filezilla download the same files in less than 1 second (per each file).

Maybe this time its OK for some average users, but is not acceptable for my application, since I need to download THOUSANDS of files.

I optimize the code as much as I could: - The cache and buffer to read the content are created 1 time in the constructor of the class. - I create 1 time the network credentials, and re-use on every file download. I know this is working, since for the first file it takes like 7s to download, and all subsequent downloads are on the range of 2s. - I change the size of the buffer from 2K until 32K. I dont know if this will help or not, since the files Im downloading are less than 1K, so in theory the buffer will be fill with all the information in 1 round from network.

Maybe is not related to the network, but to the way Im writing and/or windows handles the write of the file??

Can someone please give me some tips on how to reduce the time to something similar to filezilla?? I need to reduce the time, otherwise my ftp will be running for 3 days 24 hours a day to finish its task :( Many thanks in advance. The code here: Its not complete, it just show the download part.

//Create this on the constructor of my class
downloadCache = new MemoryStream(2097152);
downloadBuffer = new byte[32768];

public bool downloadFile(string pRemote, string pLocal, out long donwloadTime)
{
FtpWebResponse response = null;
Stream responseStream = null;

try
{
    Stopwatch fileDownloadTime = new Stopwatch();
    donwloadTime = 0;
    fileDownloadTime.Start();

    FtpWebRequest request = (FtpWebRequest)WebRequest.Create(pRemote);
    request.Method = WebRequestMethods.Ftp.DownloadFile;
    request.UseBinary = false;
    request.AuthenticationLevel = AuthenticationLevel.None;
    request.EnableSsl = false;
    request.Proxy = null;
    //I created the credentials 1 time and re-use for every file I need to download
    request.Credentials = this.manager.ftpCredentials; 

    response = (FtpWebResponse)request.GetResponse();
    responseStream = response.GetResponseStream();

    downloadCache.Seek(0, SeekOrigin.Begin);
    int bytesSize = 0;
    int cachedSize = 0;

    //create always empty file. Need this because WriteCacheToFile just append the file
    using (FileStream fileStream = new FileStream(pLocal, FileMode.Create)) { }; 

    // Download the file until the download is completed.
    while (true)
    {
        bytesSize = responseStream.Read(downloadBuffer, 0, downloadBuffer.Length);
        if (bytesSize == 0 || 2097152 < cachedSize + bytesSize)
        {
            WriteCacheToFile(pLocal, cachedSize);
            if (bytesSize == 0)
            {
                break;
            }
            downloadCache.Seek(0, SeekOrigin.Begin);
            cachedSize = 0;   
        }
        downloadCache.Write(downloadBuffer, 0, bytesSize);
        cachedSize += bytesSize;
    }

    fileDownloadTime.Stop();
    donwloadTime = fileDownloadTime.ElapsedMilliseconds;

    //file downloaded OK
    return true;
}
catch (Exception ex)
{
    return false;
}
finally
{
    if (response != null)
    {
        response.Close();
    }

    if (responseStream != null)
    {
        responseStream.Close();
    }
}
}

private void WriteCacheToFile(string downloadPath, int cachedSize)
{
using (FileStream fileStream = new FileStream(downloadPath, FileMode.Append))
{
    byte[] cacheContent = new byte[cachedSize];
    downloadCache.Seek(0, SeekOrigin.Begin);
    downloadCache.Read(cacheContent, 0, cachedSize);
    fileStream.Write(cacheContent, 0, cachedSize);
}
}

I forgot to say : Im using multiple threads to download many files at the same time. — user2232787, Jul 18 '13 at 18:10

score 1 · Answer 1 · answered Jul 18 '13 at 22:30

1

Sounds to me your problem is related to Nagels algorithm used in the TCP client.

You can try turning the Nagel's algorithm off and also set SendChunked to false.

answered Jul 18 '13 at 22:30

sean717

11,759
20
66
90

Thanks sean717. I read the info on Nagels algorithm. I also read msdn documentation on how to turn off this, and it says to set Socket.NoDelay property to either true or false. However, this is at Socket level, and my app its using higher classes: FtpWebRequest. How can I access the NoDelay property from my code?? – user2232787 Jul 19 '13 at 09:38
Hey, please take a look at [this question](http://stackoverflow.com/questions/2400949/what-might-cause-the-big-overhead-of-making-a-httpwebrequest-call). Basically FtpWebRequest is derived from System.Net.WebRequest from which you can turn off Nagel. – sean717 Jul 19 '13 at 16:18
I added the following lines: ServicePoint spTemp = request.ServicePoint; spTemp.UseNagleAlgorithm = false; I need to test if this works, since [link](http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.usenaglealgorithm.aspx) says that "the value of this property does not affect existing ServicePoint objects. Only new service points created after the change are affected" – user2232787 Jul 19 '13 at 17:08

c# ftpwebrequest performance

1 Answers1