Why is EC2 not pulling updates from bitbucket/github via post-receive hook?

Question

Before moving to EC2, I would push commits to a bitbucket repo and have a post-receive hook pull them to the server: after a commit, the url http://mywebsite.com/update.php is called, where update.php is:

    <?php `git pull`; ?>

That used to work like a charm. However, it's not working on EC2 and I'm not sure why.

I tried the following:

1. sudo chmod +x update.php This should make update.php executable.

2. Changed update.php to

    <?php `sudo git pull`; ?>
   

3. Changed update.php to

    <?php `sudo -s`; `git pull`; ?>
   

4. Changed update.php to

    <?php `sudo -s git pull`; ?>
   

I think it has something to do with permissions, because when I'm on my instance via ssh, I can run "git pull" as ec2-user. Also, "git pull" works when I'm the root user.

How can I get it to work? I think it was something to do with permissions.

Update

I did some troubleshooting (thanks for the tips @cyberx86) and found the following:

I was able to execute the update hook on the command line by running php update.php and it worked why I was root, but not when I was ec2-user. The error log showed

error: cannot open .git/FETCH_HEAD: Permission denied

so I ran the command chmod -R 777 .git as the root user.

Now I'm able to pull updates via git pull and php update.php on the command line, but it's not working when I do it through the post-receive hook or when I point my browser to url/update.php. The error log shows

Host key verification failed.^M
fatal: The remote end hung up unexpectedly

I think this means that whichever user runs the command when it's accessed via a browser doesn't have an ssh key set up, so it looks like the shell command is running, but the remote repository is not letting that user pull.

As per @cyberx86's advice, I checked out the users in /etc/passwd and while there is no PHP user, there is an user named apache: apache:x:48:48:Apache:/var/www:/sbin/nologin and then I ran sudo -u apache php -q update.php and got the following message

Could not create directory '/var/www/.ssh'.

The authenticity of host 'bitbucket.org (207.223.240.181)' can't be established.

RSA key fingerprint is REDACTED

Are you sure you want to continue connecting (yes/no)? yes

Failed to add the host to the list of known hosts (/var/www/.ssh/known_hosts).

Permission denied (publickey).

fatal: The remote end hung up unexpectedly

So it seems like it's a matter of setting up a ssh key for the user (presumably apache) that runs the shell script via browser.

Answer 1

1. PHP files do not normally need to be executable to run
2. Try to run your PHP script directly (instead of just running git pull) - more importantly, try running your PHP script as the user PHP runs as (possibly www-data, apache, etc. - unlikely to be ec2-user) something like (as root): sudo -u php_user php -q /path/to/update.php.
3. Increase the verbosity of your error_reporting and display_errors - also check your log files.
4. Finally sudo (what you have in your script) only works if it is setup in sudoers - which it probably isn't for the php user.

The user php is running as will be:

1. In Apache's httpd.conf (User, Group directives)
2. In your VirtualHost setup (e.g. if using suExec - look for the SuexecUserGroup directive)
3. In your config for your PHP-FPM pool (/etc/php-fpm.d/) if you use php-fpm as your php process manager (the user and group directives for the pool).

Instead of changing the permissions (i.e. chmod) , you should change the ownership (chown) of the files to match that of your php user. PHP files should usually have 644 permissions and directories 755 permissions. 777 should be avoided for anything but a test (or sometimes a temporary directory) - if you fix the ownership, you won't need such lax permissions.

Finally, there appear to be two problems: 1. that you don't have a key setup for your php user to connect to the git server and 2. that you don't have a copy of the git server's key (i.e. to authenticate that you are connecting to the right server).

To get the key that will let you login to the git server (I am not familiar with bitbucket) I presume one of the following will be necessary (only one, not all):

• You already have an SSH key (for the user root) that you can make available to your PHP user
• You will generate a new key on the server for the PHP user, and upload it to bitbucket
• You will generate a new key on bitbucket, and download it to your server, adding it to your PHP user's keys.

To get the git server's key should be simpler:

1. Create the directory /var/www/.ssh with the correct ownership and permissions (so that php can write to it).
2. Try SSH'ing from your server to the git repo (or running git clone - it is possible that git pull may work as well) as php's user - and accepting the key the git server provides.

I presume you did something similar to the above when you initially setup your server to use bitbucket (if not, see the documentation)

Answer 2

(Posted on behalf of OP)

Solved it. Here's what I did. To troubleshoot, I added

echo `whoami`;

to update.php and when I ran it through the browser, it showed the username (it wasn't the user apache, it was another user that was set in the virtual hosts file). I also noticed that in /etc/passwd that username as access to /bin/bash (unlike the user apache, which is restricted to /sbin/nologin), but that username's access to the shell is limited in the sudoers file. That file contained !SHELL at the end of the sudo rights for the user, so I just removed that using visudo and now it works.

I found a few posts on Stack Overflow and Server Fault that are related: one, two and three.

I used a simplified version of the method in the third link, but I used a shell command instead of pointing to a shell script.

I'm a bit worried about whether I've left any security holes because of all the fiddling I've been doing with file permissions and /etc/sudoers but at this point I'm just glad to get it working. Ideally, I'd like to get the command to run without sudo because it seems like that would be more secure, but that's likely going to be one of those things that one never gets around to once one finds a good enough solution to the problem.