Service crashes with large file using FTP Client Download

Question

I am using the following code to get data from an FTP host using a windows service.

We are calling

DownloadFile("movie.mpg","c:\movies\")

    /// <summary>
    /// This method downloads the given file name from the FTP server
    /// and returns a byte array containing its contents.
    /// </summary>
    public byte[] DownloadData(string path, ThrottledStream.ThrottleLevel downloadLevel)
    {
        // Get the object used to communicate with the server.
        WebClient request = new WebClient();

        // Logon to the server using username + password
        request.Credentials = new NetworkCredential(Username, Password);

        Stream strm = request.OpenRead(BuildServerUri(path));
        Stream destinationStream = new ThrottledStream(strm, downloadLevel);
        byte[] buffer = new byte[BufferSize];
        int readCount = strm.Read(buffer, 0, BufferSize);

        while (readCount > 0)
        {
            destinationStream.Write(buffer, 0, readCount);
            readCount = strm.Read(buffer, 0, BufferSize);
        }

        return buffer;
    }

    /// <summary>
    /// This method downloads the given file name from the FTP server
    /// and returns a byte array containing its contents.
    /// Throws a WebException on encountering a network error.
    /// Full process is throttled to 50 kb/s (51200 b/s)
    /// </summary>
    public byte[] DownloadData(string path)
    {
        // Get the object used to communicate with the server.
        WebClient request = new WebClient();

        // Logon to the server using username + password
        request.Credentials = new NetworkCredential(Username, Password);

        return request.DownloadData(BuildServerUri(path));
    }

    /// <summary>
    /// This method downloads the FTP file specified by "ftppath" and saves
    /// it to "destfile".
    /// Throws a WebException on encountering a network error.
    /// </summary>
    public void DownloadFile(string ftppath, string destfile)
    {
        // Download the data

        byte[] Data = DownloadData(ftppath);



        // Save the data to disk
        if(!System.IO.Directory.Exists(Path.GetDirectoryName(destfile)))
        {
            System.IO.Directory.CreateDirectory(Path.GetDirectoryName(destfile));
        }
        FileStream fs = new FileStream(destfile, FileMode.Create);
        fs.Write(Data, 0, Data.Length);
        fs.Close();
    }

My software keeps crashing when I get to about 500mb, Im asuming its when byte array / memory of PC runs out (it has only 1gb). Anyone know a way I can write the data directly to disk as a temp file and not store in a byte array. When the download is complete then basically move the temp file to a new location.

I'm not familiar with ThrottledStream, but if it's backed by memory then, as you point out, this will be an issue. Why not write directly to a FileStream? You should be able to replace your destinationStream with FileStream (http://msdn.microsoft.com/en-us/library/system.io.filestream.aspx) and your code should still work. — dash, Jun 06 '12 at 14:05
Also - see http://stackoverflow.com/questions/2269607/how-to-programmatically-download-a-large-file-in-c-sharp — dash, Jun 06 '12 at 14:16

score 0 · Answer 1 · answered Jun 06 '12 at 14:07

0

you can write directly to the stream something like the below:

using (Stream stream = request.GetRequestStream())
{
    stream.Write(file.Data, 0, file.Data.Length);
}

answered Jun 06 '12 at 14:07

Massimiliano Peluso

26,379
6
61
70

Service crashes with large file using FTP Client Download

1 Answers1