Using AWS CloudFormation Custom Resources to Deploy AWS Lambda Functions

Figure showing the CloudFormation service that deploys a Lambda function.

To create a CloudFormation stack containing a Lambda function, you usually need to prepare a deployment package that contains the function code and dependencies, and upload it to an S3 bucket located in the same AWS region where you want to create the stack. In the CloudFormation template, you specify the corresponding S3 bucket name and key.

For simple Lambda functions written in Python or Node.js, you can put the function code inline in your CloudFormation template. In this case, the deployment package is not needed, but the function code can only contain up to 4096 characters.

If your Lambda function requires a deployment package, you can automate the creation of this package using a custom resource in your CloudFormation template. In this case, the deployment package can be created at the time of the creation of your CloudFormation stack, for example, by an existing Lambda function, on an existing EC2 instance, or even by some external system/service.

This blog post describes how to declare the source code of a Lambda function inline in a CloudFormation template and how to use a custom resource to store the function code in a CodeCommit repository and inject it dynamically into a CloudFormation stack.

How to Declare the Source Code of a Lambda Function Inline in a CloudFormation Template

Simple Lambda functions that are written in Python or Node.js can be declared inline in a CloudFormation template. In this case, you include the source code with the ZipFile parameter of the Code property. The source code can only contain up to 4096 characters. Also, for the Handler property, the first part of the handler identifier must be index.

If you use JSON to author your CloudFormation template, you have to declare each line of the function code separately, and join all lines using the CloudFormation intrinsic function Fn::Join. Also, you must escape quotes and special characters.

...
"Resources": {
  "HelloWorldLambdaFunction": {
    "Type": "AWS::Lambda::Function",
    "Properties": {
      "Runtime": "python3.7",
      "Handler": "index.lambda_handler",
      "Code": {
        "ZipFile" : {
          "Fn::Join" : [
            "\n", [
              "import json",
              "def lambda_handler(event, context):",
              "    return {",
              "        'statusCode': 200,",
              "        'body': json.dumps('Hello World!')",
              "    }"
            ]
          ]
        }
      },
...

If you use YAML to author your CloudFormation template, you can include the function code without any modifications.

...
Resources:
  HelloWorldLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: python3.7
      Handler: index.lambda_handler
      Code:
        ZipFile: |
          import json
          def lambda_handler(event, context):
              return {
                  'statusCode': 200,
                  'body': json.dumps('Hello World!')
              }
...

Declaring the source code of a Lambda function inline in a CloudFormation template is useful for simple Lambda functions that you do not change often.

CloudFormation custom resources allow you to deploy Lambda functions in more complex cases. For further details and examples, see the following sections of this blog post.

CloudFormation Custom Resources Overview

CloudFormation custom resources allow you to add custom logic to your CloudFormation templates and do additional provisioning tasks.

When a custom resource is created, updated, or deleted, CloudFormation sends a request to a service token. A service token can be an Amazon SNS topic or AWS Lambda function Amazon Resource Name (ARN) from the same AWS region in which you are creating your CloudFormation stack. In the JSON request to the service token, CloudFormation sends StackId, RequestId, LogicalResourceId, ResponseURL, custom resource properties, and some other information.

The ResponseURL field of the request is an S3 pre-signed URL. The custom resource must send a response (upload a JSON-formatted file) to this callback URL. Otherwise, the stack operation fails. The response to a callback URL can be sent, for example, using the cfn-request module, or the curl utility.

If you use a Lambda-backed custom resource, you are not constrained to send the response from this Lambda function. You can pass the CloudFormation request further to another AWS service or even to an external system/service, which creates the corresponding physical resource and sends the response to the callback URL.

Similarly, if you use an SNS-backed custom resource, you can, for example, queue the CloudFormation requests in an SQS queue. Then, a job running on an EC2 instance polls the messages from the queue. For a create or an update request, the job creates an artifact, stores it in an S3 bucket, and sends the S3 bucket name and key of the artifact along with the response status (SUCCESS or FAILED) and some other information to the callback URL (ResponseURL). The following diagram shows the call flow.

Flow diagram showing creation of a CloudFormation stack with an SNS-backed custom resource. CloudFormation requests are queued in an SQS queue and, then, processed on an EC2 instance.

How to Use a CloudFormation Custom Resource to Inject the Source Code of a Lambda Function Dynamically into a CloudFormation Stack

In this procedure, you use a Lambda-backed custom resource that can get a file from a CodeCommit repository. The following diagram shows the call flow.

Flow diagram showing creation of a CloudFormation stack with an Lambda-backed custom resource. The Lambda functions gets the requested file from the specified CodeCommit repository.

In the properties of the custom resource, besides the service token, you specify the AWS region of the CodeCommit repository, the name of the repository, commit specifier, and the path to the file that you want to get. For example, the custom resource can be declared in a CloudFormation template as follows:

...
  HelloWorldLambdaFunctionCode:
    Type: AWS::CloudFormation::CustomResource
    Properties:
      ServiceToken: !Join
        - ':'
        - - 'arn:aws:lambda'
          - !Ref 'AWS::Region'
          - !Ref 'AWS::AccountId'
          - 'function:get-file-from-codecommit-repository'
      RepositoryRegion: 'us-east-1'
      RepositoryName: 'example-repository'
      CommitSpecifier: '5d21d12e10167a4f3f048cce8415be36f5345cb6'
      FilePath: 'lambda/hello_world.py'
...

This custom resource represents the source code of the Lambda function hello-world that you deploy with your CloudFormation stack.

When the custom resource is created, updated, or deleted, CloudFormation sends a request to the Lambda function get-file-from-codecommit-repository specified by the ServiceToken property. For a create or an update request, this Lambda function gets the requested file from the specified CodeCommit repository and sends the source code from this file along with the response status (SUCCESS or FAILED) and some other information to the callback URL (ResponseURL).

Then, you use the CloudFormation intrinsic function Fn::GetAtt to inject this source code into your CloudFormation stack during the creation or update of the Lambda function hello-world.

...
  HelloWorldLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: hello-world
      Runtime: python3.7
      Handler: index.lambda_handler
      Code:
        ZipFile: !GetAtt 'HelloWorldLambdaFunctionCode.FileContent'
...

For the source code of the Lambda function get-file-from-codecommit-repository, see Appendix A, Lambda Function That Gets a File from a CodeCommit Repository (Python). To create this Lambda function, use the CloudFormation template below. You may want to adjust the IAM role of this Lambda function and grant permissions only to a few CodeCommit repositories to follow the standard security advice of granting the least privilege.

The download URL of the get-file-from-codecommit-repository.yml CloudFormation template.

Appendix A, Lambda Function That Gets a File from a CodeCommit Repository (Python)

# CloudBriefly.com

import json
import boto3
import cfnresponse

def lambda_handler(event, context):
    print('Received request:\n%s' % json.dumps(event, indent=4))

    resource_properties = event['ResourceProperties']

    if event['RequestType'] in ['Create', 'Update']:
        try:
            codecommit_client = boto3.client(
                'codecommit', region_name=resource_properties['RepositoryRegion']
            )
            response = codecommit_client.get_file(
                repositoryName=resource_properties['RepositoryName'],
                commitSpecifier=resource_properties['CommitSpecifier'],
                filePath=resource_properties['FilePath']
            )
        except:
            cfnresponse.send(event, context, cfnresponse.FAILED, {})
            raise
        else:
            cfnresponse.send(event, context, cfnresponse.SUCCESS,
                             {'FileContent': response['fileContent'].decode('utf-8')})
    elif event['RequestType'] == 'Delete':
        cfnresponse.send(event, context, cfnresponse.SUCCESS, {})