Running Git in AWS Lambda

Last updated: January 10, 2019

The AWS Lambda execution environment is based on Amazon Linux AMI release 2017.03. Git client is not pre-installed in the AWS Lambda execution environment.

You can compile the latest sources of Git and include the binaries into the deployment package of your Lambda function. You must build your binaries for the matching version of Amazon Linux, or link them statically.

Alternatively, you can download Git RPM package from the official Amazon Linux repository of Amazon Linux AMI release 2017.03. You can do it in the Lambda function itself during its execution. The following procedure describes this approach.

How to Run Git in AWS Lambda

The repositories of Amazon Linux AMI release 2017.03 are defined in the configuration files in the /etc/yum.repos.d directory. You can query the repositories using repoquery utility to get the download URLs of RPM packages.

The download URL of the latest available version of Git RPM package for Amazon Linux AMI release 2017.03 is http://packages.$awsregion.amazonaws.com/2017.03/updates/ba2b87ec77c7/x86_64/Packages/git-2.13.5-1.53.amzn1.x86_64.rpm, where $awsregion can have one of the following values:

  • ap-northeast-1
  • ap-northeast-2
  • ap-south-1
  • ap-southeast-1
  • ap-southeast-2
  • ca-central-1
  • eu-central-1
  • eu-north-1
  • eu-west-1
  • eu-west-2
  • eu-west-3
  • sa-east-1
  • us-east-1
  • us-east-2
  • us-west-1
  • us-west-2
  • In this procedure you do not install the missing dependencies of Git RPM package. Thus, some features of Git client may not work in your Lambda function.

To download and run Git client in your Lambda function during its execution:

  1. Download git-2.13.5-1.53.amzn1.x86_64.rpm package, verify its signature and extract the files from this package to /tmp/git-2.13.5 directory.
    For example, if you invoke your Lambda function in US East (N. Virginia) region, execute the following commands:
    rm -fr /tmp/git-2.13.5 && \
    mkdir /tmp/git-2.13.5 && \
    cd /tmp/git-2.13.5 && \
    curl -s -O http://packages.us-east-1.amazonaws.com/2017.03/updates/ba2b87ec77c7/x86_64/Packages/git-2.13.5-1.53.amzn1.x86_64.rpm && \
    rpm -K git-2.13.5-1.53.amzn1.x86_64.rpm && \
    rpm2cpio git-2.13.5-1.53.amzn1.x86_64.rpm | cpio -id && \
    rm git-2.13.5-1.53.amzn1.x86_64.rpm
    
  2. Run Git client as follows:
    cd /tmp/git-2.13.5 && \
    HOME=/var/task \
    GIT_TEMPLATE_DIR=/tmp/git-2.13.5/usr/share/git-core/templates \
    GIT_EXEC_PATH=/tmp/git-2.13.5/usr/libexec/git-core \
    /tmp/git-2.13.5/usr/bin/git [options] <command> [<args>]
    

For a Python implementation, see Appendix B, How to Clone a Git Repository in AWS Lambda (Python).

Appendix A, How to Connect to an AWS CodeCommit Repository Using the AWS Lambda IAM Role Credentials

To connect to AWS CodeCommit repositories over HTTPS using the AWS Lambda IAM role credentials (without a username and password), perform the following steps:

  1. Include the AWS CLI into the deployment package of your Lambda function.
  2. Set up the AWS CLI credential helper for Git.
    1. Create Git configuration file .gitconfig with the following contents:
      [credential]
          helper = /var/task/aws codecommit credential-helper $@
          UseHttpPath = true
      
    2. Add .gitconfig to the root directory of the deployment package ~/lambda.zip of your Lambda function.
      zip -g ~/lambda.zip .gitconfig
      
      • Git will search the configuration file at /var/task/.gitconfig, because you set HOME=/var/task when you run Git client.
  3. In the IAM role of your Lambda function, grant permissions for the required CodeCommit actions.
    For example, if you want to clone your CodeCommit repository, grant permissions for the codecommit:GitPull action.
  4. Download and run Git client from your Lambda function. See How to Run Git in AWS Lambda.

Appendix B, How to Clone a Git Repository in AWS Lambda (Python)

The following code snippet shows how you can clone the public repository of Hugo static website generator in your Lambda function. You create a shallow clone with a depth of 1 to the /tmp/git-2.13.5 directory.

The execution duration of this Lambda function depends on the size of the configured memory and can be as less as 3 seconds.

from __future__ import print_function
import os
import subprocess

def lambda_handler(event, context):
    remote_repository = 'https://github.com/gohugoio/hugo.git'
    git_command = 'clone --depth 1'

    git_rpm_download_url = '/'.join([
        'http://packages.%s.amazonaws.com' % os.environ['AWS_REGION'],
        '2017.03/updates/ba2b87ec77c7/x86_64/Packages',
        'git-2.13.5-1.53.amzn1.x86_64.rpm'])

    git_rpm = git_rpm_download_url.split('/')[-1]  # git-2.13.5-1.53.amzn1.x86_64.rpm
    tmp_dir = '/tmp/%s' % '-'.join(git_rpm.split('-')[0:2])  # /tmp/git-2.13.5

    # Download, verify and unpack Git RPM package
    print(subprocess.check_output(
        ' && '.join([
            'rm -fr %s' % tmp_dir,
            'mkdir %s' % tmp_dir,
            'cd %s' % tmp_dir,
            'curl -s -O %s' % git_rpm_download_url,
            'rpm -K %s' % git_rpm,  # Check the GnuPG signature
            'rpm2cpio %s | cpio -id' % git_rpm,
            'rm %s' % git_rpm
        ]),
        stderr=subprocess.STDOUT,
        shell=True))

    # Clone the remote repository
    print(subprocess.check_output(
        ' && '.join([
            'cd %s' % tmp_dir,
            ' '.join([
                'HOME=/var/task',
                'GIT_TEMPLATE_DIR=%s/usr/share/git-core/templates' % tmp_dir,
                'GIT_EXEC_PATH=%s/usr/libexec/git-core' % tmp_dir,
                '%s/usr/bin/git %s %s' % (tmp_dir, git_command, remote_repository)
            ])
        ]),
        stderr=subprocess.STDOUT,
        shell=True))