2

I have a Digital Ocean Droplet (VPS) running Ubuntu 14.04. I have installed s3cmd and am able to run a sync successfully with this command:

s3cmd sync --recursive --preserve /srv s3://MY-BUCKET-NAME

And if I put that same command in to a .sh to run as a cron-task every minute, it works:

* * * * * sh /srv/backupToS3.sh >> /srv/backupToS3.log

But when I add some echos and dates to the .sh:

echo 'Started:'
date +'%a %b %e %H:%M:%S %Z %Y'
s3cmd sync --recursive --preserve /srv s3://MY-BUCKET-NAME
echo 'Finished:'
date +'%a %b %e %H:%M:%S %Z %Y'

the s3cmd sync does not work/run anymore. But the echos and dates print to the log file:

Started:
Mon Jun  8 17:45:01 PDT 2015
Finished:
Mon Jun  8 17:45:01 PDT 2015

Any help or push in the right direction would be very appreciated. I originally followed this article to get this far.

codeview
  • 209
  • 5
  • 15

2 Answers2

9

s3cmd uses a configuration file located at ~/.s3cfg. It's probably having trouble picking that up. Pass in --config=/home/username/.s3cfg and see if that helps.

In any case, s3cmd isn't consistently maintained. The official commandline client (aws-cli) is much better in many ways.

edit: use this as your .sh file, make sure it has the executable bit set (chmod 755 whatever.sh).

echo 'Started:'
date +'%a %b %e %H:%M:%S %Z %Y'
/usr/bin/s3cmd sync --recursive --preserve --config=/usr/bin/s3cmd --verbose /srv s3://MY-BUCKET-NAME
echo 'Finished:'
date +'%a %b %e %H:%M:%S %Z %Y'
tedder42
  • 23,519
  • 13
  • 86
  • 102
  • You're the 2nd person I've seen mention AWS-CLI over s3cmd (1st was in a similar question about cron-task), perhaps I will look in to that. I do not see a .s3cfg file in my home directory (I'm SFTPed in to droplet via Cyberduck, showing hidden files). /home is empty. – codeview Jun 09 '15 at 22:31
  • Also, the command runs fine alone (w/o the echo and date lines) wouldn't that mean that s3cmd is working fine? – codeview Jun 09 '15 at 22:32
  • If you don't have a .s3cfg, where is s3cmd reading your IAM keys from? – tedder42 Jun 09 '15 at 22:33
  • Ah, found it with a find command: `find / -name .s3cfg`, it is in `/root/.s3cfg`. What is your take on my other question/comment, though? Would it not be that the s3cmd is working fine since its runs w/o the echo and date lines? – codeview Jun 09 '15 at 23:53
  • To confirm...should I be putting `--config=/root/.s3cfg`? and is that to go in the sm3cmd line? ie: `s3cmd sync --recursive --preserve --config=/root/.s3cfg /srv s3://MY-BUCKET-NAME` – codeview Jun 10 '15 at 02:26
  • yeah, try that. you haven't given all the information necessary to confirm- for instance, is that cron in root's crontab? – tedder42 Jun 10 '15 at 02:28
  • Yes, the cron is in the root's crontab. – codeview Jun 10 '15 at 02:34
  • No change with `s3cmd sync --recursive --preserve --config=/root/.s3cfg /srv s3://MY-BUCKET-NAME`...still just the echos and dates in the log file and the sync does not run. – codeview Jun 10 '15 at 03:00
  • put the full path to s3cmd (use `which s3cmd` to find it), and add `--verbose`. – tedder42 Jun 10 '15 at 03:01
  • running `which s3cmd` returns `/usr/bin/s3cmd`. Do you mean for me to add that path in the config section from before? And is `--verbose` okay to add anywhere in the line? ie: `s3cmd sync --recursive --preserve --config=/usr/bin/s3cmd --verbose /srv s3://MY-BUCKET-NAME` – codeview Jun 10 '15 at 16:46
  • I think you mean for the `config=` to be `config=/root/.s3cfg`? It tried it how you have it and received ignore errors. Changing it produces same result: echo and date lines print to .log, but sync does not run. I did the `chmod` as well. – codeview Jun 10 '15 at 17:40
  • yeah, I meant for it to be .s3cfg, I just copied your line- oops. Anyhow, if you aren't getting anything, there's something more going on that is missing. – tedder42 Jun 10 '15 at 17:46
  • Ok. Thank you for your help. I will look in to learning aws-cli. – codeview Jun 10 '15 at 18:17
  • Thanks alot! Been struggeling for two days now getting s3cmd running from cron in Centos. ERROR: /.s3cfg: No such file or directory ERROR: Configuration file not available. ERROR: Consider using --configure parameter to create one --config=/path/to/.s3cfg worked like a charm – Espen Birk Sep 19 '15 at 10:11
1

i had the same problem and i tried to figure out why it is not running. My solution was to add the following lines to the top of my script:

export HOME=/home/$USERNAME cd /home/$USERNAME

Also you should call the s3cmd with:

/usr/bin/s3cmd

OR put the following line to the top of your script:

PATH=$PATH:/usr/bin
gcommit
  • 11
  • 1