0

I would like to script a complex scenario of file transfers from Linux to a Windows FTP server (detecting new/changed files and then transfer) and I am unable to find a command-line ftp client which supports sending files over a persistent (one time login) session on non-interactive mode.

ssh/scp with password-less authentication is not an option, since I have no access to the Windows FTP server.

For example using lftp

`lftp -u user,pass -p 21 -e "mput newfile.txt;"` 10.0.0.100

will upload my file to the server, but requires authentication/handshaking to happen on every request, which takes a long time and severely slows down the transfers.

I am wondering whether there is a way to authenticate once, establish a persistent connection and then sending the files over this connection. lftp and other clients are able to do this in interactive mode (connect and then perform multiple commands until quitting).

Therefore:

  • Is there an FTP client on Linux which may operate like this in non-interactive mode?
  • Alternatively, is there a way to simulate this behaviour, scripting over the interactive mode? Maybe something utilizing expect with netcat/telnet and/or named pipes?
henfiber
  • 1,209
  • 9
  • 11
  • 1
    Aside from automating the password entry, lftp can be scripted already: https://web.archive.org/web/20111024014512/http://tutorials.papamike.ca/pub/lftp.html#scripts – glenn jackman Jun 19 '15 at 13:33
  • Many languages implement FTP clients, so you can create as arbitarily complex scripts as you like: [Tcl](http://core.tcl.tk/tcllib/doc/trunk/embedded/www/tcllib/files/modules/ftp/ftp.html), [Perl](https://metacpan.org/pod/Net::FTP), [Ruby](http://ruby-doc.org/stdlib-2.2.2/libdoc/net/ftp/rdoc/Net/FTP.html), [Python](https://docs.python.org/3/library/ftplib.html?highlight=ftp#module-ftplib) – glenn jackman Jun 19 '15 at 13:36
  • @glennjackman I know that `lftp` can read a set from commands from a file and that I can also pipe multiple commands into it, but that does not solve what I want to achieve. As I stated, I want to upload files when they are made available (detecting new/changed files). Thus I cannot prepopulate a script with commands, I have to initiate a new transfer when the files are detected. This is the reason I want the session to be kept open to avoid reconnecting all the time. – henfiber Jun 19 '15 at 19:17
  • Thank you @glennjackman. I know that, but I am interested in a command-line/shell script solution. I'm sure someone with more experience in using `expect` and `telnet/netcat` can come up with something interesting which may be applied to other use cases as well – henfiber Jun 19 '15 at 19:31
  • how often do new files come along? – glenn jackman Jun 19 '15 at 19:44
  • @glennjackman about 24 files per minute and I have to send them over a (sometimes unreliable) 3G connection, that's why the ping time is significant and the handshaking is slowing transfers considerably. – henfiber Jun 19 '15 at 19:50

1 Answers1

1

Untested, but you can do something like this Tcl:

package require ftp
set conn [ftp::Open server user pass]
while true {
    after 5000          ;# sleep 5 seconds
    foreach newfile [glob -nocomplain *] {
        puts "sending $newfile"
        ftp::Put $newfile
        file delete $newfile
    }
}

The expect would be similar. untested:

set timeout -1
spawn lftp -u user,pass server
expect "> "
while true {
    sleep 5
    foreach newfile [glob -nocomplain *] {
        puts "sending $newfile"
        send "put $newfile\r"
        expect "> "
        file delete $newfile
    }
    send "\r"          ;# "hit enter" as a keep-alive
}
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
  • Thank you Glenn. How could I make sure that the connection is kept alive? And if it is not, restart it in the loop then continue sending new files – henfiber Jun 20 '15 at 01:27
  • you'd have to check the output or result of the "put", if there's an error, you'd loop back to re-spawn – glenn jackman Jun 20 '15 at 13:57
  • Is there a command in `expect` that performs error handling and branching to the spawn command? In a shell script that would be for example `attempt_transfer || respawn` – henfiber Jun 20 '15 at 14:41
  • No, expect does not know what the command's exit status is on the spawned process. After `expect "> "`, you have to examine the contents of `$exect_out(0,string)` for error messages. – glenn jackman Jun 20 '15 at 18:21
  • Thanks, I'll make some tests and let you know. – henfiber Jun 20 '15 at 21:35