0

I would like to process a directory listing from a ftp server. At the end of the day I would like to delete old files on the ftp server, but I'm not there yet. I have problems to interpret the lines from the directory listing. I'm a newbie in shell scripts, so please be patient. It´s probably a very basic mistake I've made....

You find the piece of code below and the printout when I run this script.

    #!/bin/bash
    # LOCAL/FTP/SCP/MAIL PARAMETERS
    SERVER="xxx.yyy.zzz"  # IP of  NAS, used for ftp
    USERNAME="joe"         # FTP username of Network disk used for ftp
    PASSWORD="joe1234"         # FTP password of Network disk used for ftp
    DESTDIR="/opt/backup"   # used for temorarily storage
    DESTDIRNAS="/sdb1/domoticz_backup/A/" # Path to your  NAS backup folder
    DOMO_IP="192.168.0.243"   # Domoticz IP
    DOMO_PORT="8085"        # Domoticz port # get directory listing from remote source

putdir=$DESTDIRNAS
ftp -n $SERVER > /tmp/dirlist <<END_SCRIPT
quote USER  $USERNAME
quote PASS $PASSWORD
cd $putdir
ls -l
quit
END_SCRIPT


cat /tmp/dirlist | while read LINE; do
  echo $LINE
  awk '{print $9}' $(LINE)
done;

The output is as follows:

pi@raspberrypi:~ $ ./test.sh
-rwxrwxrwx 1 0 0 229632 Oct 09 07:27 domoticz_20171009092723.db.gz
./test.sh: line 23: LINE: command not found
domoticz_20171009105333.db.gz
domoticz_20171009111940.db.gz
domoticz_20171009113716.db.gz
domoticz_scripts_20171009105333.tar.gz
domoticz_scripts_20171009111940.tar.gz
domoticz_scripts_20171009113716.tar.gz
pi@raspberrypi:~ $

The file /tmp/dirlist holds 7 lines similar to the first printed when I run the script. However, the script can't parse the first line, but throws an error. Subsequent lines works fine but they are not printed by the echo command. I can't figure out why this doesn't work....

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
ubee
  • 27
  • 6
  • Have you considered a FUSE client such as [tfs](https://github.com/gonicus/fts)? That way you could just interact with your remote server the same way you would a regular (local) filesystem, and gets you out of the business of trying to parse `ls` output, which is [generally frowned on](http://mywiki.wooledge.org/ParsingLs). – Charles Duffy Oct 11 '17 at 18:18
  • That said, `$(LINE)` is your immediate problem. It's *not at all* the same as `${LINE}`, but rather tries to run a command called `LINE` (which you don't have). – Charles Duffy Oct 11 '17 at 18:19
  • (There are some other issues here -- filenames that start with or contain whitespace or glob characters will misbehave; likewise for backslashes, particularly with `read -r`; consider reviewing [BashFAQ #001](http://mywiki.wooledge.org/BashFAQ/001)). – Charles Duffy Oct 11 '17 at 18:21
  • I edited this to specify tftp in the title and tagging -- if tftp weren't part of your question, it would be a duplicate of many, *many* preexisting ones, for which the answer comes down to using shell built-in capabilities rather than trying to parse `ls`. – Charles Duffy Oct 11 '17 at 18:22
  • (By the way, as an aside -- note that [POSIX specifies](http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html) all-caps names be used for variables with meaning to the OS or shell, and guarantees that applications can use lower-case names without interfering with behavior of POSIX-defined tools. Generally speaking, it's a good idea to take advantage of this guarantee and stick to lowercase names for your own variables; keep in mind that setting even a regular shell variable will overwrite any like-named environment variable). – Charles Duffy Oct 11 '17 at 18:26
  • Thanks. I learn day by day.... Btw, found out the easiest way to split up the line in tokens is just to use bash built in support for this, not awk. . token=($line) and then file=${token[8]} etc. – ubee Oct 12 '17 at 08:10
  • `token=( $line )` is buggy (not in the sense that the shell has bugs -- it behaves exactly as documented -- but the documented behavior has caveats for the unwary). If your line contains `*`, it'll be replaced with a list of filenames in the directory where your script is being run. Use `while read -r -a tokens` instead. See [BashPitfalls #50](http://mywiki.wooledge.org/BashPitfalls#hosts.3D.28_.24.28aws_....29_.29) for more discussion on this point. – Charles Duffy Oct 12 '17 at 14:26
  • That said, even `read -r -a tokens` isn't safe with filenames, because part of a name with spaces would be in `token[9]`. Instead, in this case, you'd want something more like `while read -r mode nlinks owner group numbytes month day time filename; do` -- that way, if there are extra fields on the end (caused by spaces in the names), they go into `filename`. However, as the [ParsingLs](http://mywiki.wooledge.org/ParsingLs) page documents, date format is different between locales and platforms, so the above is inherently unsafe (so what worked for me above may not work for you). – Charles Duffy Oct 12 '17 at 14:34
  • (The above also doesn't work correctly with filenames that *start* with spaces or end with them; you're *really* better off trying to find a mechanism that doesn't involve splitting output of `ls -l` into fields in the first place). – Charles Duffy Oct 12 '17 at 14:36
  • Ok, I understand there are pitfalls. Thanks for pointing them out. But in my particular case the intention is to backup a predefined number of files (I know the file names...) and I have no intention to use this on different platforms. So I believe I can avoid troubles. Right now the backup and the clean-up procedure of old files works fine. – ubee Oct 12 '17 at 18:00

0 Answers0