20

I am trying to run git in AWS lambda to make a checkout of a repository.

This is my setup:

  • I am using nodejs 4.3
  • I am not using nodegit because I want to use the "--depth=1" parameter, which is not supported by nodegit.
  • I have copied the git and ssh executable from the correct AWS AMI and placed then in a "bin" folder in the zip I upload.
  • I added them to PATH with this:

->

process.env['PATH'] = process.env['LAMBDA_TASK_ROOT'] + "/bin:" + process.env['PATH'];

The input variables are set like this:

"checkout_url": "git@...",
"branch":"master

Now I do this (for brevity, I mixed some pseudo-code in):

downloadDeploymentKeyFromS3Sync('/tmp/ssh_key');
fs.chmodSync("/tmp/ssh_key",0600);
process.env['GIT_SSH_COMMAND'] = 'ssh -o StrictHostKeyChecking=no -i /tmp/ssh_key';
execSync("git clone --depth=1 " + checkout_url + " --branch " + branch + " /tmp/checkout");

Running this in my local computer using lambda-local everything works fine! But when I test it in lambda, I get:

warning: templates not found /usr/share/git-core/templates
PRIV_END: seteuid: Operation not permitted\r
fatal: Could not read from remote repository.
  • The "warning" is of course, because I did not install git but just copied the binary. Is that a reason why this should not work?
  • Why is git needing "setuid"? I read that in some shells, that is disabled for security reasons. So it makes sense that it does not work in lambda. Can git somehow be instructed to not "need" this command?
danilopopeye
  • 9,610
  • 1
  • 23
  • 31
Nathan
  • 7,099
  • 14
  • 61
  • 125

3 Answers3

23

Yep, this is definitely possible, I've created a Lambda Layer that achieves just this. No need to mess with any env variables, should work out of the box:

https://github.com/lambci/git-lambda-layer

As stated in the README, all you need to do is add a layer with the following ARN:

arn:aws:lambda:<region>:553035198032:layer:git:<version>

(replace <region> and <version>, check README for latest version)

Michael Hart
  • 5,109
  • 3
  • 22
  • 21
  • So what did you do in LambCI that got around the seteuid() call? I do not see anything other than the open issue yet? – Jared Chmielecki Dec 20 '16 at 13:22
  • 1
    When using git-2.4.3.tar from LambCI with ssh, I get the error message `ssh: error while loading shared libraries: libfipscheck.so.1: cannot open shared object file: No such file or directory`. I believe you also have to set `LD_LIBRARY_PATH=/tmp/myDir/usr/lib64`. – Shaun Jackman Jan 24 '17 at 07:51
  • Want to leave this here: https://blog.enki.com/using-git-on-aws-lambda-f365a2db706b – Shinigami Aug 09 '18 at 10:43
  • I am tring to add the aws bin binary to your layer , so i am unzip the layer.zip folder and add the aws into the bin folder and then zip all the file but after uploaded it i am getting git command not found error – Mickael Zana Jul 26 '21 at 13:32
  • I needed to set LD_LIBARY_PATH=/opt/lib, since lamda layers are installed in /opt and /opt/lib contains all library files – Jonny Shanahan Jun 13 '23 at 23:54
1

The issue is that you cannot copy just the git binary. You need a portable version of git and even with that you're going to have a bad time because you cannot guarantee that the os the lambda function runs on is going to be compatible with the binary.

Stepping back, I would just walk away from this approach completely. I would clone and build a package that I would just download pretty much the same way you do downloadDeploymentKeyFromS3Sync.

Mircea
  • 10,216
  • 2
  • 30
  • 46
  • 3
    The OS is guaranteed to be Amazon Linux. – Mark B May 17 '16 at 21:04
  • where in the documentation do you see that the OS is guaranteed to be Amazon Linux? It's Amazon Linux right now, w/ the patches being applied, but that's not a guarantee it's always going to be the same os and building something that makes that assumption seems like asking for trouble to me – Mircea May 18 '16 at 01:25
  • 3
    It's right there in the documentation: http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html And then there's this: https://aws.amazon.com/blogs/compute/running-executables-in-aws-lambda/ Which states "If you compile your own binaries, ensure that they’re either statically linked or built for the matching version of Amazon Linux. The current version of Amazon Linux in use within AWS Lambda can always be found on the Supported Versions page of the Lambda docs." – Mark B May 18 '16 at 01:55
  • 1
    that's different than saying the OS will not change (either major changes or minor changes). So the OS is not guaranteed to be something. It's done this way to unlock some use cases, but it still seems hacky to me and I would not build something along these lines. – Mircea May 18 '16 at 02:57
  • Agree with Mircea. Certainly does not deserve a -1. Equalized that. – Wanderer Jun 28 '16 at 17:23
-1

You might consider this a non-answer, but I've found the easiest way to run arbitrary binaries from Lambda is... not to. If I cannot do the work from within a platform-independent, non-binary approach, I integrate Docker into the workflow, managing Docker containers from the Lambda function.

On AWS one way to do this is to use the Elastic Container Service (ECS) to spawn a task that runs git.

If you stand up a Docker Swarm instance or integrate another Docker-API compatible service such as Rackspace Carina or Joyent's Triton, then you could use a project I personally put together specifically for integrating AWS Lambda with Docker: "Dockaless".

Good luck!

Eric Windisch
  • 42
  • 1