0

I've been trying to solve relatively small problem with moving some files across FTP servers but no luck so far.

In a nutshell this is what I'm doing, I have three servers:

SourceSFTP TargetSFTP Target_2_SFTP

The script is supposed to do the following

  1. Connect to SourceFTP
  2. Grab all files
  3. Loop through files
  4. Call function that takes file as parameter and does stuff to it, let's call it postfunc()
  5. Drop the files to TargetSFTP

The problem occurs when inside postfunc I put another call to lftp to transfer file to Target_2SFTP. The command is executed properly (I can see the file moved) but then the number 5 never happens.

This is the script I have:

function postfunc() {
  the_file=$1

  lftp<<END_SCRIPT2
open sftp://$Target2SFTP
user $USERNAME $PASSWORD
cd /root
put $the_file
bye
END_SCRIPT2
}


echo "Downloading files from $SOURCE_SFTP"
lftp -e "echo 'testing connection';exit" -u $SOURCE_USERNAME,$SOURCE_PASSWORD $SOURCE_SFTP
lftp -e "set xfer:clobber true;mget $SOURCE_DIR*.csv;exit" -u $SOURCE_USERNAME,$SOURCE_PASSWORD $SOURCE_SFTP || exit 0

files=(*.csv)
batch=10
for ((i=0; i < ${#files[@]}; i+=batch)); do
  commands=""
  # Do some stuff  
  for((j=0; j < batch; j+=1)); do
    commands=$commands"mv source_dir/${files[i+j} archivedir/${files[i+j]};"
    postfunc ${files[i]}
  done


  echo "Archiving batch..."
  lftp -e "$commands;exit" -u $SOURCE_USERNAME,$SOURCE_PASSWORD $SOURCE_SFTP

  lftp<<END_SCRIPT
open sftp://$TARGET_SFTP
user $TARGET_USERNAME $TARGET_PASSWORD
cd $TARGET_DIR
mput dirr/*
bye
END_SCRIPT

done

Hopefully I'm missing something obvious... At the moment even if I move one file "Archiving batch" never shows up, if I remove contents of postfunc() everything executes correctly

Moseleyi
  • 2,585
  • 1
  • 24
  • 46
  • I'd suggest fixing everything that http://shellcheck.net/ finds before asking questions here; there are some obvious problems that it'll be able to detect without needing human review. – Charles Duffy Mar 02 '16 at 19:04
  • On a different topic, consider avoiding all-uppercase names for your own variables -- these names are reserved for shell and system use; staying away from them avoids potential for collisions. See http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html, keeping in mind that shell variables and environment variables share a namespace. – Charles Duffy Mar 02 '16 at 19:05
  • I quickly wrote main parts of the script that's why but the main parts (LFTP) are written word by word as they are. Script in general works fine but doesn't finish only when I have the extra LFTP hence I think it's structural issue, something I don't know about LFTP, rather than syntax – Moseleyi Mar 02 '16 at 19:32
  • There's a mix of both -- some of your generic scripting bugs are things with potential to mess up lftp, if conditions are right (whitespace in passwords, &c). Fix them, and then there's no question as to whether they're at fault, and we don't need to waste our time nitpicking about things that aren't your actual problem. – Charles Duffy Mar 02 '16 at 19:36
  • (as another example: `postfunc ${files[i]}` will fail to correctly pass any file with whitespace in its name through as a single argument, putting the first part of the filename in $1 within the function, the next part in $2, etc). – Charles Duffy Mar 02 '16 at 19:40
  • I'd also suggest using the `--log` argument to lftp to write its commands to a file, and then inspecting that file. – Charles Duffy Mar 02 '16 at 19:43

0 Answers0