how to add crontab scheduler through user-data script in aws ec2?

Question

I am trying to add a crontab so that I can get the cloudwatch metrics of diskspace used and disk space utilization every 5 minutes through a user data script. Below is my user-data script:

   #!/bin/bash
sudo yum install -y perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64
curl https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip -O
unzip CloudWatchMonitoringScripts-1.2.2.zip && rm CloudWatchMonitoringScripts-1.2.2.zip && cd aws-scripts-mon

crontab<<EOF
*/1 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl -mem-util --mem-used --mem-avail --swap-util --disk-space-util --disk-space-used --disk-space-avail --memory-units=megabytes --disk-path=/dev/xvda1 --from-cron
EOF

./mon-put-instance-data.pl -mem-util --mem-used --mem-avail --swap-util --disk-space-util --disk-space-used --disk-space-avail --memory-units=megabytes --disk-path=/dev/xvda1

All these steps are working properly when running from aws-terminal, also no steps are failing in cloud-init-logs. The first time I am able to get the cloud watch metrics but after that, they don't come through, so it means crontab is not working, how can this be fixed?

Answer 1

Don't use -e as that is edit mode, you want something like:

  (crontab -l 2>/dev/null || echo ""; echo "*/5 * * * * /path/to/job -with args") | crontab -

Update: Added empty echo pipe, to avoid user data stopping due to failure in missing crontab list for root (default running user for userdata), which would return non-zero and fail the script.

Note: Also be aware that Userdata by default runs only once, not every time the instance is rebooted. So changing userdata when the instance is stopped doesn't let you test different ways unless you modify as per this document. If set to run each time, the script above would also concatenate the same rule over and over to crontab!

Answer 2

You can also use heredoc to achieve this in a cleaner way

crontab<<EOF
* * * * * script.sh
EOF

And if you want to append to the existing crontab, do the below

crontab<<EOF
$(crontab -l)
* * * * * script2.sh
EOF

Now list the crontab using

crontab -l

Also, the man page says that each user can have their own crontab, and though these are files in /var/spool/cron, they are not intended to be edited directly.

e.g. if you are creating cron as root user, the corresponding user's cron file would be

/var/spool/cron/root

Please see below in detail

[root@localhost ~]# crontab -l
no crontab for root

[root@localhost ~]# crontab<<EOF
*/5 * * * * script1.sh
EOF
[root@localhost ~]# crontab -l
*/5 * * * * script1.sh

[root@localhost ~]# crontab<<EOF
*/10 * * * * script2.sh
EOF
[root@localhost ~]# crontab -l
*/10 * * * * script2.sh

[root@localhost ~]# crontab<<EOF
$(crontab -l)
* * * * * script3.sh
EOF
[root@localhost ~]# crontab -l
*/10 * * * * script2.sh
* * * * * script3.sh

[root@localhost ~]# crontab<<EOF
$(crontab -l)
* * * * * script4.sh
EOF
[root@localhost ~]# crontab -l
*/10 * * * * script2.sh
* * * * * script3.sh
* * * * * script4.sh

[root@localhost ~]# cat /var/spool/cron/root
*/10 * * * * script2.sh
* * * * * script3.sh
* * * * * script4.sh
[root@localhost ~]#

In your case, it would look like

#!/bin/bash
sudo yum install -y perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64
curl https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip -O
unzip CloudWatchMonitoringScripts-1.2.2.zip && rm CloudWatchMonitoringScripts-1.2.2.zip && cd aws-scripts-mon

crontab<<EOF
echo $'i*/5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl -mem-util --mem-used --mem-avail --swap-util --disk-space-util --disk-space-used --disk-space-avail --memory-units=megabytes --disk-path=/dev/xvda1 --from-cron\E:x\n'
EOF

./mon-put-instance-data.pl -mem-util --mem-used --mem-avail --swap-util --disk-space-util --disk-space-used --disk-space-avail --memory-units=megabytes --disk-path=/dev/xvda1

Answer 3

This post is old but i spent way too long finding this solution.

I was trying to automate adding a cronjob to an ec2 so that i didnt have to manually schedule the job or add the script. all of this was done through user data in the cloudformation. my solution required that i used Systems manager. U may need to change system manager to ec2-user.

To do this I used:

crontab -u ssm-user /path/to/script

I tried like 5 other solutions that worked in the console but didnt work in userdata. I really hope this helps others automate their deployments in the future

Answer 4

for adding lines in the crontab i used these lines in my user-data script and it worked fine for me

sudo su

crontab -u ec2-user -l

echo -e "*/5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl -mem-util --mem-used --mem-avail --swap-util --disk-space-util --disk-space-used --disk-space-avail --memory-units=megabytes --disk-path=/dev/xvda1 --from-cron " | crontab -u ec2-user -

exit

the linked which help me to find the solution is https://serverfault.com/questions/296949/programmatically-add-entry-to-users-crontab

Answer 5

I had similar issue like others are saying here that these commands work fine from CLI but when added through userdata not working. After several hours of head scratching search finally realized I had to add line "#!/bin/bash" before crontab setup commands. This fixed my problem, some very well experienced people might know this already but if you are bit new to this realm may not know this. My script look like below.

# !/bin/bash
sudo su 
echo '*/1 * * * * <path-to-python> <path-to-python-script>' > /tmp/mycrontab.txt
sudo -u ubuntu bash -c 'crontab /tmp/mycrontab.txt'