How to read a folder and move newly created files to two different network locations

Question

I have a folder on my server which get files from a ftp process running in another server.

What I need is to, whenever new file appears in the directory, move those files to another two network locations interchangeably. (In load-balancing manner - first file to Location1, Next file to location 2, Next to Location 1 likewise)

I use incrontab to run a bash script to do this.

What is the most suitable way to do this?

Here is the way I'm going to do it. Is it the best method to achieve this?

incrontab:

/myserver/monitored_folder IN_CREATE /root/scripts/move_files.sh $#

movefiles.sh bash script (pseudo):

start
read var from file1
if var is odd
move $# to NetworkLocation1
else
move $# to NetworkLocation2
end if
increment var
write var to file1
end

score 0 · Answer 1 · answered Jul 04 '14 at 10:09

0

You have a fairly simple way of doing it, that seems to be working fine. The only other method I would recommend would be to use a simple toggle (variable either 0 / 1):

let toggle=0

if test "$toggle" -eq 0; then
    move to NetworkLocation1
    ((toggle+=1))
else
    move to NetworkLocation2
    ((toggle-=1))
fi

That would just insure a NetworkLocation1 / NetworkLocation2 balance based on each file instead of an odd/even var within a file. Give it some thought.

answered Jul 04 '14 at 10:09

David C. Rankin

81,885
6
58
85

But as incorntab calls the fresh script every time it see a new file, this script always run with toggle=0 and move to files to NetworkLocation1 only. Isn't it? – U-map Jul 04 '14 at 11:00

score 0 · Accepted Answer · answered Jul 07 '14 at 08:23

I've implemented it. It's working fine.

incrontab:

/data/source_folder/ IN_CREATE /path_to_script/file_delivery.sh $#

Bash Script:

#!/bin/bash
#log file location
logfile=/var/log/file_delivery.log
exec > $logfile 2>&1

read var < var.dat
rem=$(( $var % 2 ))

#action here
sleep 30
if [ $rem -eq 0 ]; then
  scp /data/source_folder/$1 root@SERVER1:/path_on_server1/dest_folder
  rm -f /data/source_folder/$1
  echo "$(date +"%y-%m-%d-%H:%M:%S") - file $1 has been moved to server1"
else
  scp /data/source_folder/$1 root@SERVER2:/path_on_server2/dest_folder
  rm -f /data/source_folder/$1
  echo "$(date +"%y-%m-%d-%H:%M:%S") - file $1 has been moved to server2"
fi

var=$((var+1))
echo $var > var.dat

How to read a folder and move newly created files to two different network locations

2 Answers2