2

I have an Ubuntu server which runs a web scraping cron job daily at 0901.

cron line: 01 09 * * 1-5 bash /root/FCPIDCheck.sh

The file runs a pid checker to see whether the process is already running from the day before this stops instances from running concurrently day after day.

PIDFILE=/root/FC.pid
if [ -f $PIDFILE ]
then
  PID=$(cat $PIDFILE)
  ps -p $PID > /dev/null 2>&1
  if [ $? -eq 0 ]
  then
    echo "Process already running"
    exit 1
  else
    ## Process not found assume not running
    echo $$ > $PIDFILE
    if [ $? -ne 0 ]
    then
      echo "Could not create PID file"
      exit 1
    fi
  fi
else
  echo $$ > $PIDFILE
  if [ $? -ne 0 ]
  then
    echo "Could not create PID file"
    exit 1
  fi
fi

/root/FCScript
rm $PIDFILE

Finally, the file FCScript calls the following Rscript instance:

cd /root/Program/
Rscript --no-save --no-restore --verbose doSomething.R >> ./logs/log_$(date "+%Y-%m-%d").Rout 2>> ./errors/err_$(date "+%Y-%m-%d").Rout

For whatever reason, some mornings the program does not start correctly. I went into syslog and found there was an error saying no MTA installed so I've installed postfix to output messages to local.

On the times that the server doesn't start with the cron job I can restart it manually but nearly always find that it will stop itself within 30mins so there's an underlying problem but I really don't know how to find what caused the bash to stop.

I tried running the FCScript file manually from bash alongside the code running from RStudio to see whether it was a change on the scraped web page that was causing R to hit an error but I found that the Rstudio instance runs fine whilst the instance in bash is terminated.

I'm clearly not an expert in Linux so help and suggestions for anything I can do better are much appreciated.

Freddie
  • 65
  • 4
  • Redirect the entry in crontab to a log file so you get a verbose impression of what's going on bash /root/FCPIDCheck.sh > somelogfile – Raman Sailopal Jul 26 '17 at 14:19

0 Answers0