Ruby and Net::SFTP: why can't I move this to a separate function?

Question

Writing a function that checks via SFTP if a file is present on the server.

I have written a function sftp_file_exists_1? that works. Now I want to split this function into two functions, and to my surprise, this does not work.

require "net/sftp"

def sftp_file_exists_1?(host, user, filename)
  Net::SFTP.start(host, user, verify_host_key: :always) do |sftp|
    sftp.stat(filename) do |response|
      return response.ok?
    end
  end
end

def sftp_stat_ok?(sftp, filename)
  sftp.stat(filename) do |response|
    return response.ok?
  end
end

def sftp_file_exists_2?(host, user, filename)
  Net::SFTP.start(host, user, verify_host_key: :always) do |sftp|
    return sftp_stat_ok?(sftp, filename)
  end
end

p sftp_file_exists_1?("localhost", "user", "repos")
p sftp_file_exists_2?("localhost", "user", "repos")

I expected:

true
true

since a file repos actually exists on the server. However, I get (abbreviated):

true
#<Net::SFTP::Request:0x000055f3b56732d0 @callback=#<Proc:0x000055f3b5673280@./test.rb:14>, ...

Addendum: this works:

def sftp_stat_ok?(sftp, filename)
  begin
    sftp.stat!(filename)
  rescue Net::SFTP::StatusException
    return false
  end
  return true
end

Eric Duminil · Accepted Answer · 2019-07-21T18:26:27.550

Interesting problem.

Old school debugging

Let's add some puts to see what happens:

require "net/sftp"

def sftp_file_exists_1?(host, user, filename)
  Net::SFTP.start(host, user, verify_host_key: :always) do |sftp|
    puts "  BEFORE STAT"
    sftp.stat(filename) do |response|
      puts "  REQUEST FINISHED"
      return response.ok?
    end
    puts "  AFTER STAT"
  end
  puts "  NOT EXECUTED"
end

def sftp_stat_ok?(sftp, filename)
  request = sftp.stat(filename) do |response|
    puts "  REQUEST FINISHED"
    return response.ok?
  end
  puts "  REQUEST SENT"
  request
end

def sftp_file_exists_2?(host, user, filename)
  Net::SFTP.start(host, user, verify_host_key: :always) do |sftp|
    puts "  CALL STAT_OK?"
    return sftp_stat_ok?(sftp, filename)
  end
  puts "  NOT EXECUTED"
end

sftp_file_exists_1? outputs:

  BEFORE STAT
  AFTER STAT
  REQUEST FINISHED
true

While sftp_file_exists_2? outputs:

  CALL STAT_OK?
  REQUEST SENT
#<Net::SFTP::Request:0x0000000001db2558>

" REQUEST FINISHED" doesn't appear.

Async logic

The block you pass to stat is a callback. It only gets called when the server responds. To make sure the block gets executed before sftp_stat_ok? returns, you need to wait for the request to be complete:

def sftp_stat_ok?(sftp, filename)
  request = sftp.stat(filename) do |response|
    return response.ok?
  end
  request.wait
end

def sftp_file_exists_2?(host, user, filename)
  Net::SFTP.start(host, user, verify_host_key: :always) do |sftp|
    return sftp_stat_ok?(sftp, filename)
  end
end

It isn't needed for the first version because start:

If a block is given, it will be passed to the SFTP session and will be called once the SFTP session is fully open and initialized. When the block terminates, the new SSH session will automatically be closed.

Thanks. I started with Ruby yesterday and it seems I misunderstood the function/block construction. I thought the functions is called, and then the block. Now I understand that this is not true. — Lasse Kliemann, Jul 22 '19 at 05:53
@LasseKliemann: This example was surprising to me too, even though I've been working with Ruby for a long time. It appears weird at first simply because it's asynchronous. A block can be executed any number of time (including 0) and at any time, depending on what the method does. Documentation helps a lot! — Eric Duminil, Jul 22 '19 at 06:33

Ruby and Net::SFTP: why can't I move this to a separate function?

1 Answers1

Old school debugging

Async logic