0

I have a zsh script that uses rsync to move files from an EC2 instance to my local machine, then copies those files to a Box drive. I have a Mac, so I am trying to use launchd to automate this sync to happen everyday. This is my .plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.username.pipelines.syncdata</string>
    <key>EnvironmentVariables</key>
    <dict>
        <key>PATH</key>
        <string>/Users/username/.pyenv/versions/anaconda3-2020.11/bin:/Users/username/.pyenv/versions/anaconda3-2020.11/condabin:/Users/username/.pyenv/shims:/Users/username/.pyenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/username/Desktop/geckodriver:/opt/X11/bin</string>
    </dict>
    <key>ProgramArguments</key>
    <array>
        <string>/bin/zsh</string>
        <string>/Users/username/Desktop/repos/china/zzz_pipelines/sync_data.sh</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>11</integer>
        <key>Minute</key>
        <integer>15</integer>
    </dict>
    <key>StandardErrorPath</key>
    <string>/Users/username/Desktop/syncdata.err</string>
    <key>StandardOutPath</key>
    <string>/Users/username/Desktop/syncdata.out</string>
</dict>
</plist>

This is my zsh script, which runs just fine if I paste the file path for the zsh script directly into my terminal:

#!/bin/zsh

########################################
## SCRIPT TO SYNC DATA
## /Users/username/Desktop/repos/china/zzz_pipelines/sync_data.sh
##
## (1) Syncs data from EC2 to Local
## (2) Syncs data from Local to Box
########################################

## PARAMS ----------------------------------------------------------------
# these should be within the repo
# should be subdirs with data we want to move from ec2 to local
SYNCFROMAWS=(
    "csrc/raw_data/csrc_ss2015/"  
)

# these are files we want to sync from local to Box
# before the | is subdir of `china` repo locally
# after the | is subdir of `china_data` in Box
SYNCTOBOX=(
    "csrc/raw_data|csrc"
)

## Logging
BASHLOG_DIR="/Users/username/Desktop/repos/china/zzz_pipelines/sync_data_logs"
BASHLOG="${BASHLOG_DIR}/$(date +'%Y%m%d_%H%M%S')_syncdata.log"
printf "Log File - " > $BASHLOG
date >> $BASHLOG
echo "---------------------------------------------------------------" >> $BASHLOG

EC2INFO="ubuntu@12.123.123.1231"
EC2HOME="/home/ubuntu"
REPONAME="Desktop/repos/china"
BOXDATA="/Users/username/Box/china_data"


## sync from ec2 to local
echo "----------------- SYNCING FROM EC2 to LOCAL -----------------" >> $BASHLOG
for x in ${SYNCFROMAWS[@]}; do
    # get the name of the directory for the news source with raw data
    FROMDIR="${EC2INFO}:${EC2HOME}/${REPONAME}/${x}"
    TODIR="/Users/username/${REPONAME}/${x}"

    COMMAND="/usr/bin/rsync -hvrPt -e \"ssh -i /Users/username/.ssh/pemkey.pem\" ${FROMDIR} ${TODIR}"
    echo "---------------------------------------------------------------" >> $BASHLOG
    echo $COMMAND >> $BASHLOG  # to run the script
    eval $COMMAND >> $BASHLOG
done


## sync from local to Box
echo "----------------- SYNCING FROM LOCAL to BOX -----------------" >> $BASHLOG
for y in ${SYNCTOBOX[@]}; do
    FROM="$(echo ${y} | cut -d'|' -f1)"
    TO="$(echo ${y} | cut -d'|' -f2)"
    # get the name of the directory for the news source with raw data
    FROMDIR="/Users/username/${REPONAME}/${FROM}"
    TODIR="${BOXDATA}/${TO}"

    COMMAND="/usr/bin/rsync -hvrPt ${FROMDIR} ${TODIR}"
    echo "---------------------------------------------------------------" >> $BASHLOG
    echo $COMMAND >> $BASHLOG 
    eval $COMMAND >> $BASHLOG
done

I put this file in ~/Library/LaunchAgents, and loaded it using launchctl load -w [PLISTFILENAME].

This is the error I get in the output syncdata.err file when the launchd job tries to run at 11:15am:

/bin/zsh: can't open input file: /Users/username/Desktop/repos/china/zzz_pipelines/sync_data.sh

How can I give the launchd job permission to run my zsh script? These are the permissions on the script when I run ls -lh:

-rwxr-xr-x   1 username  1534107694   3.2K Feb 15 10:17 sync_data.sh

It has execute permissions for me as the user, and my understanding is that launchd scripts run as the user, so I don't understand why it's not working. Any help is appreciated!

seagullnutkin
  • 311
  • 1
  • 10
  • I would be inclined to put an appropriate `zsh` shebang as the first line of the script and remove the first element of the `ProgramArguments` array. – Mark Setchell Feb 17 '21 at 00:36
  • That error is saying the file was not found, not permission denied (and you don't need execute permission for that invocation). Check that you have the exact filename, maybe by copying and pasting the string from the plist into an ls command. – Gairfowl Feb 17 '21 at 01:16
  • Thanks @Gairfowl! The file definitely exists though, when I paste that file path into my terminal and hit enter it runs the zsh script. I also have `!/bin/zsh` at the beginning of the script – seagullnutkin Feb 17 '21 at 01:30
  • I oversimplified a bit - that error could also indicate that you don't have read permission. I suspect that you're running into a security measure that restricts where launchd can access files from, but I'm having trouble finding the link atm, so I don't have more details. – Gairfowl Feb 17 '21 at 01:58
  • 1
    See if this helps: https://stackoverflow.com/a/58936196/9307265 . Note that setting the permissions wide open like that answer suggests is a bit of a security risk, but you can at least try it and see if it's the right track. There's a more detailed guide out there somewhere, but apparently I've lost the link. – Gairfowl Feb 17 '21 at 04:41
  • I would hope you actually have a shebang with a **hash** at the start `#!/bin/zsh` – Mark Setchell Feb 17 '21 at 07:15
  • @MarkSetchell : The #! line is not the problem and unnecessary in this case. You can see from the question that zsh does not find the script itself. If it would find it, it would ignore the #! line. – user1934428 Feb 17 '21 at 09:17
  • @seagullnutkin : Your permissions are fine. You would not even need executable permission on the script if you run it that way. Dull question: What does the command `file /Users/username/Desktop/repos/protocol-china/zzz_pipelines/sync_data.sh` tell you? Just copy and paste it into your command line. – user1934428 Feb 17 '21 at 09:20
  • @user1934428 OP stated in the comments that his script begins with `!/bin/zsh` - which will cause an error - rather than `#!/bin/zsh` – Mark Setchell Feb 17 '21 at 09:29
  • Ah, I now see the comment. This type of information should really go into the question, not in a comment. Hence it is just a typo in the script. I will vote for closing the question. – user1934428 Feb 17 '21 at 10:49
  • @MarkSetchell My zsh script does start with `#!/bin/zsh`, I just forgot to include the hash in my comment. – seagullnutkin Feb 17 '21 at 16:21
  • I've updated my question to include the zsh script in case that's helpful. The script runs just fine if I paste it into my terminal, I want it to just run automatically though with launchd so I don't have to manually paste it everyday – seagullnutkin Feb 17 '21 at 16:29
  • 2
    I think you now need to explicitly allow programs (such as `launchd`) to access your Desktop and Documents... https://apple.stackexchange.com/a/396396 – Mark Setchell Feb 17 '21 at 16:41

0 Answers0