Exception of type 'System.OutOfMemoryException' was thrown in a SFTP connection

Question

Scenario: a i have an app that makes connection with a SFTP server and download a file. The code that do this:

while (true)
{
    try
    {
        var connectionInfo = // creating a new connection info of a Renci.SshNet lib
        var client = new SftpClient(connectionInfo);
        client.Connect();

        using (SftpFileStream sftpFileStream = client.OpenRead(path)
        {
            using (FileStream fileStream = new FileStream(filePath, FileMode.CreateNew))
            {
                int BUFFER_SIZE = 2048;
                byte[] buffer = new byte[BUFFER_SIZE];

                int readBytes = sftpFileStream.Read(buffer, 0, BUFFER_SIZE);

                while (readBytes > 0)
                {
                    fileStream.Write(buffer, 0, readBytes);
                    readBytes = sftpFileStream.Read(buffer, 0, BUFFER_SIZE);
                }
            }
        }

        client.Dispose();
    }
    catch (Exception e)
    {
        log.Error(e);
    }

    Thread.Sleep(1 hour);
}

In the line client.Connect() a first error was raised:

Renci.SshNet.Common.SftpPathNotFoundException: No such file at Renci.SshNet.Sftp.SubsystemSession.WaitHandle(WaitHandle waitHandle, TimeSpan operationTimeout) at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError) at Renci.SshNet.Sftp.SftpFileStream..ctor(SftpSession session, String path, FileMode mode, FileAccess access, Int32 bufferSize, Boolean useAsync) at Renci.SshNet.Sftp.SftpFileStream..ctor(SftpSession session, String path, FileMode mode, FileAccess access) at Renci.SshNet.SftpClient.OpenRead(String path) at Infraero.TINE3.STTColetor.Negocio.Drivers.DriverAbstratoArquivoFTP.BaixarArquivoPABXSFTP(ConfiguracaoColetor configuracaoColetor, String arquivoPABX) in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Drivers\DriverAbstratoArquivoFTP.cs:line 446 at Infraero.TINE3.STTColetor.Negocio.Drivers.DriverAbstratoArquivoFTP.ProcessarArquivoSFTP(SftpFile arquivo, ConfiguracaoColetor configuracaoColetor, List'1 listaBilhetes) in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Drivers\DriverAbstratoArquivoFTP.cs:line 658 at Infraero.TINE3.STTColetor.Negocio.Drivers.DriverAbstratoArquivoFTP.ColetarBilhetes(ConfiguracaoColetor configuracaoColetor, List'1 listaBilhetes, String ultimoArquivoTransmitido) in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Drivers\DriverAbstratoArquivoFTP.cs:line 73 at Infraero.TINE3.STTColetor.Negocio.Coletor.ColetarBilhetes() in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Coletor.cs:line 56 at Infraero.TINE3.STTColetor.WindowsService.ServicoColetor..ctor() in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.WindowsService\ServicoColetor.cs:line 85

After 1 hour the process try download the file again, but raises a new error in line client.Connect():

System.Threading.Tasks.TaskSchedulerException: An exception was thrown by a TaskScheduler. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at System.Threading.Thread.StartInternal(IPrincipal principal, StackCrawlMark& stackMark) at System.Threading.Thread.Start(StackCrawlMark& stackMark) at System.Threading.Thread.Start(Object parameter) at System.Threading.Tasks.ThreadPoolTaskScheduler.QueueTask(Task task) at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection) at System.Threading.Tasks.Task.InternalStartNew(Task creatingTask, Object action, Object state, CancellationToken cancellationToken, TaskScheduler scheduler, TaskCreationOptions options, InternalTaskOptions internalOptions, StackCrawlMark& stackMark) at System.Threading.Tasks.TaskFactory.StartNew(Action action, TaskCreationOptions creationOptions) at Renci.SshNet.Session.ExecuteThread(Action action) at Renci.SshNet.Session.Connect() at Renci.SshNet.BaseClient.Connect() at Infraero.TINE3.STTColetor.Negocio.Drivers.DriverAbstratoArquivoFTP.ConectarServidorSFTP(String servidor, Int32 porta, String login, String senha) in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Drivers\DriverAbstratoArquivoFTP.cs:line 551 at Infraero.TINE3.STTColetor.Negocio.Drivers.DriverAbstratoArquivoFTP.RequisitarArquivoServidorSFTP(ConfiguracaoColetor configuracaoColetor) in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Drivers\DriverAbstratoArquivoFTP.cs:line 519 at Infraero.TINE3.STTColetor.Negocio.Drivers.DriverAbstratoArquivoFTP.ColetarBilhetes(ConfiguracaoColetor configuracaoColetor, List'1 listaBilhetes, String ultimoArquivoTransmitido) in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Drivers\DriverAbstratoArquivoFTP.cs:line 67 at Infraero.TINE3.STTColetor.Negocio.Coletor.ColetarBilhetes() in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.Negocio\Coletor.cs:line 56 at Infraero.TINE3.STTColetor.WindowsService.ServicoColetor..ctor() in D:\SVN\STT\trunk\3-0_CodigoFonte_Coletor\3-4_SRC\Infraero.TINE3.STTColetor.WindowsService\ServicoColetor.cs:line 85

After 1 day, the second error continues, and the memory usage of the app increased from 60MB to 800MB.

Anyone knows what the problem can be?

Yes, ignore the line counter in the error, the code that i put i the question is a summary of all code. — Vinicius Ottoni, Aug 30 '12 at 14:11
Just to let you know that it wasn't me, although I think that you shouldn't post the complete stack trace. — Andre Calil, Aug 30 '12 at 16:58
Hi @ViniciusOttoni. Would you mind sharing the method you used to download. In your code you just have `// download the file`. I am trying to return a stream using your method — tinonetic, Apr 23 '15 at 11:24
Thanks @ViniciusOttoni. Sorry to take you back again. How do I return a stream without having to create a file? With FTP in .Net, you just say `response.GetResponseStream()` and you get it streaming back the file. I am downloading large files, so `MemoryStream` won't be optimum. How do I achieve the same using ssh.net? — tinonetic, Apr 24 '15 at 06:22
@user919426 the method client.OpenRead(path) returns a response stream, that I do what I want. For your case won't work? What you trying to do, it's only download the file? If yes, you can use the method client.DownloadFile(path, outputStream, actionCallback). — Vinicius Ottoni, Apr 24 '15 at 13:16
problem I'm having with client.DownloadFile() is that it causes the calling process to wait until the entire stream is ready to be returned. I would like it to stream it back as it reads. See my question here, as I know its out of topic for this one: http://stackoverflow.com/questions/29816812/downloading-and-returning-a-non-blocking-stream-of-data-using-ssh-net — tinonetic, Apr 26 '15 at 09:05

score 2 · Accepted Answer · edited Aug 30 '12 at 14:28

2

A quick glance through the code base of the library that you're using (http://sshnet.codeplex.com/SourceControl/changeset/view/18974) shows that SftpClient also implements IDisposable, but you're not cleaning it up or closing it when something fails. You should try something like:

try
{
    var connectionInfo = // something
    using(var client = new SftpClient(connectionInfo))
    {
        client.Connect();
        using(var sftpFileStream = client.OpenRead(path))
        {
            // download the file
        }
    }
}

It may also be a memory leak inside the library that you're using.

edited Aug 30 '12 at 14:28

Vinicius Ottoni

4,631
9
42
64

answered Aug 30 '12 at 14:09

Justin Niessner

242,243
40
408
536

@ViniciusOttoni - The way you have the code laid out above, you couldn't possibly call dispose from the catch block because the variable would be out of scope. I updated my answer with an example. – Justin Niessner Aug 30 '12 at 14:14
With a using block, if raises an error in the line `client.Connect()` of your answer, the object `client` will be disposed? – Vinicius Ottoni Aug 30 '12 at 14:20
@ViniciusOttoni - Yes. You may want to take a closer look at the code, though, to see if there's any other cleanup that needs done. – Justin Niessner Aug 30 '12 at 14:25

Exception of type 'System.OutOfMemoryException' was thrown in a SFTP connection

1 Answers1