Adding Custom HTTP Headers to Amazon S3 Origin Responses Using Amazon Lambda@Edge

Figure showing users who send requests to Amazon CloudFront. A Lambda@Edge function adds X-Robots-Tag header to Amazon S3 origin responses.

Last updated: August 24, 2019

Amazon CloudFront is a content delivery network (CDN) that speeds up the delivery of content to users. The origin of content can be, for example, an HTTP web server or an S3 bucket configured as a website endpoint.

If you use an S3 bucket configured as a website endpoint, you do not need to configure and maintain an HTTP web server, but you have less control over HTTP headers that are sent from your S3 origin to CloudFront cache, and, thus, also to viewers. CloudFront forwards the HTTP headers received from its origin, but S3 only allows to add a limited list of headers, for example, Cache-Control, Content-Encoding, Content-Language, and Content-Type.

You can add other custom headers (metadata), but they must start with the x-amz-meta- prefix. Hence, using only CloudFront and S3 it is not possible to add to responses arbitrary HTTP headers, for example, X-Robots-Tag, X-Content-Type-Options or security headers.

You can customize content and headers that CloudFront delivers to your users using Amazon Lambda@Edge. This blog post describes how to use a Lambda@Edge function to add custom HTTP headers to S3 origin responses. The headers are stored as optional user-defined metadata of S3 objects in your S3 bucket.

How to Add Custom HTTP Headers to S3 Origin Responses Using a Lambda@Edge Function

In this procedure, you use a Lambda@Edge function to add custom HTTP headers to S3 origin responses. CloudFront then forwards these headers to your users. All custom headers are stored as optional user-defined metadata of S3 objects in your S3 bucket. Hence, you do not need to adjust the code of the Lambda function each time when you want to add a new HTTP header to a response.

Your Lambda@Edge function is invoked only when CloudFront sends requests to your S3 bucket configured as a website endpoint. The number of such requests is limited and predictable. Thus, the charges are also low and predictable.

Flow diagram showing users who send requests to Amazon CloudFront. A Lambda@Edge function customizes Amazon S3 origin responses. All HTTP headers are stored as optional user-defined metadata of S3 objects.

To add custom HTTP headers to S3 origin responses, perform the following steps:

  1. Store custom HTTP headers as user-defined metadata of S3 objects in your S3 bucket.
    S3 metadata keys have by default the x-amz-meta- prefix. For your Lambda@Edge function to distinguish between ordinary metadata keys and custom HTTP headers, you additionally add header-.
    For example, to add X-Robots-Tag and X-Content-Type-Options to the existing files object1.json and object2.html in the bucket example-bucket, copy these files to themselves as follows:
    aws s3 cp s3://example-bucket/object1.json s3://example-bucket/object1.json \
    --metadata Header-X-Robots-Tag=noindex;
    aws s3 cp s3://example-bucket/object2.html s3://example-bucket/object2.html \
    --metadata Header-X-Content-Type-Options=nosniff;
    
    Your S3 objects will have x-amz-meta-header-x-robots-tag and x-amz-meta-header-x-content-type-options metadata keys.
  2. Open the Lambda Management Console of the US East (N. Virginia) region at https://console.aws.amazon.com/lambda/home?region=us-east-1. Then, create a new Lambda function with the configuration below.
    • Make sure that you create the Lambda function in the US East (N. Virginia) region. You can add CloudFront triggers only for Lambda functions created in this AWS region.
    Runtime: Python 3.7
    IAM Role: create a new Lambda IAM role that has the inline and trust relationships policies below. Alternatively, you can create a new role from the AWS policy template Basic Lambda@Edge permissions (for CloudFront trigger).
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Action": [
            "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"
          ],
          "Resource": "arn:aws:logs:*:*:*",
          "Effect": "Allow"
        }
      ]
    }
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Action": "sts:AssumeRole",
          "Principal": {
            "Service": [
              "lambda.amazonaws.com", "edgelambda.amazonaws.com"
            ]
          },
          "Effect": "Allow"
        }
      ]
    }
    • Make sure that in the trust relationships policy of your role, you have edgelambda.amazonaws.com service.
    Function code: use the following snippet.
    This function removes headers (metadata keys) that start with x-amz-meta-header- prefix and adds corresponding headers without this prefix. As a result, for example, x-amz-meta-header-x-robots-tag will be replaced with x-robots-tag.
    # CloudBriefly.com
    
    def lambda_handler(event, context):
        response = event['Records'][0]['cf']['response']
        headers = response['headers']
    
        custom_header_prefix = 'x-amz-meta-header-'
    
        for old_header in list(headers):
            if old_header.lower().startswith(custom_header_prefix):
                new_header = old_header.lower().replace(custom_header_prefix, '')
                headers[new_header] = [
                    {'key': new_header.title(), 'value': headers[old_header][0]['value']}
                ]
                del headers[old_header]
                print('Header %s was replaced with %s.' % (old_header, new_header))
    
        return response
    Memory (MB): 128 MB
    Timeout: 3 sec
  3. Publish a new version of your Lambda function. Then, copy from the top of the page the function ARN (including the version suffix).
  4. Add a CloudFront trigger to run your Lambda function whenever your S3 origin responds to a request of the CloudFront cache.
    1. Open CloudFront Management Console at https://console.aws.amazon.com/cloudfront/home.
    2. In the Behaviors tab of your CloudFront distribution, for the cache behavior that you want to use with the trigger, add a new Lambda function association that has the Origin Response event type and the ARN of your Lambda function from Step 3.
      Keep Include Body unchecked.
    3. Wait until the status of your CloudFront distribution is changed from In Progress to Deployed.
      It can take up to 20 minutes until your Lambda function is replicated across different AWS regions.
    4. In the Invalidations tab of your CloudFront distribution, create a new invalidation for the paths to the objects from Step 1. Alternatively, you may want to invalidate your complete distribution by specifying /* path.
  5. Verify that your Lambda@Edge function adds custom HTTP headers to viewer responses.
    For example, execute the following commands:
    curl -I https://abcdefjhij1234.cloudfront.net/object1.json
    curl -I https://abcdefjhij1234.cloudfront.net/object2.html
    
    • Alternatively, you can inspect the returned HTTP headers in the Network tab of the Developer tools of your favorite browser.
  • CloudWatch logs of your Lambda@Edge function are stored in the AWS region closest to the location where this function is executed. The log group name has the /aws/lambda/us-east-1. prefix.