FTP Script to download a set of files

Question

I am after an FTP script to download all the files from an FTP server, then delete these files once complete, but leave files on the remote server if any were added during the download process, to be fetched during a later session.

Is it possible to achieve something like this using FTP scripts or do I need a different solution.

I was planning on using psftp.

I don't have anything yet, I was wondering if this sort of thing is possible with ftp scripting — benPearce, Jun 24 '10 at 22:33
i've never done ftp scripting but it depends on what you're running the ftp script from. it shouldn't be that hard to write a batch/bash command to delete all the files you just download, assuming appropriate permissions — wag2639, Jun 24 '10 at 22:36
This could get entertainingly Rube Goldberg-ian ... But it's do-able. I'll echo wag2639: what OS? — Adrien, Jun 24 '10 at 23:37
The script will be executed on Windows 2000 server...not my choice, the ftp client is not set in stone, the ftp server is from SSH Comms (ssh.com) — benPearce, Jun 24 '10 at 23:47

score 2 · Accepted Answer · edited Apr 13 '17 at 12:14

2

Based on Maxwell's answer to this question, I would probably do something along the lines of this pseudocode:

Use $favorite_scripting_language to gather the list of filenames to d/l then kill; write an output file named script.txt in the form:

cd /source/directory
lcd c:\target\directory
get foo.bar
delete foo.bar
<<lather, rinse, repeat>>

then wrap it up with:

psftp.exe username@server -be -pw user_password -b c:\script.txt

edited Apr 13 '17 at 12:14

Community

1

answered Jun 24 '10 at 23:51

Adrien

431
2
6

Thanks Adrien, I had already considered something like this. – benPearce Jun 25 '10 at 01:13

score 0 · Answer 2 · answered Jun 27 '10 at 18:04

How about python? As a plus, it should work more or less unmodified on any OS. And the even bigger plus is that it is more flexible that using the other, more brittle, methods.

It would go something like this:

from ftplib import FTP
import os

ftp = FTP("server")
ftp.login("id", "passwd")
ftp.cwd("/server/ftpdir/")

#copy every file as usual to local system
filelist = []
ftp.retrlines('LIST', filelist.append)
### the list filelist will have one element per file
### which you will have to cut up into the proper pieces
### among Python's installed files on your system is 
### the script Tools/scripts/ftpmirror.py which has such code
### it starts at around line 140 (for python 2.6.1) 
### --- not very long but this margin is too small to contain it
### assume the file names are in dictionary filedict
### keys are filenames and values are file sizes
for fname in filedict.keys():
   localfname = os.path.join(localdir, fname) # set localdir to cwd or whatever
   lf = open(localfname , "wb")
   ftp.retrbinary('RETR ' + fname, lf.write)
   lf.close()
#refresh remote directory listing into filedict2 as with previous filedict
for fname in filedict2.keys():
   if fname not in filedict:
       # new file since download started! ignore. it's for later
       pass
   else:
       if filedict[fname] == filedict2[fname]:
          # apparently we got a good copy so delete the file on ftp server
          ftp.delete(fname)
       else:
          # delete local file: remote was being uploaded or something
          os.unlink(os.path.join(localdir, fname))

You have to add error checking and other such stuff. The same could be accomplished with perl and it would probably look about the same.

FTP Script to download a set of files

2 Answers2