1

Context: I have approx. 20K files to be transferred via SFTP.
I am skeptical that if I send them in single session then session might terminate midway.
I tried searching for timeout but what I found was timeout for idle session.

So I was trying to search if there is something like transfer in batches where I can transfer 500 files in a batch. And after each batch a new session will start.
I am using Ruby's Net::SFTP library

My current code is

      Net::SFTP.start(host, username, :password => password, :port => port) do |sftp|
        files.each do |file| # files is an array of filenames (with path)
          puts "Transferring '#{file}' to #{host}.."
          sftp.upload!(file, "/#{File.basename(file)}")
        end
      end

What I tried?
I tried to search in docs or in some blogs if there exists some option of batching.

If there is no such option then I am planning to handle this as below

files.each_slice(500) do |batch|
  Net::SFTP.start(host, username, :password => password, :port => port) do |sftp|
    batch.each do |file| # files is an array of filenames (with path)
      puts "Transferring '#{file}' to #{host}.."
      sftp.upload!(file, "/#{File.basename(file)}")
    end
  end
end
Mario Galic
  • 47,285
  • 6
  • 56
  • 98
Anand Bait
  • 331
  • 1
  • 12
  • 1
    What did you try so far? Please share your code. Are you just skeptical or did you actually see error? – spickermann Sep 03 '20 at 06:25
  • I am skeptical because those are 20K files and might take more than 2hours. That's why I was searching for timeout value. – Anand Bait Sep 03 '20 at 17:57

0 Answers0