Running SSM Agent in an AWS Lambda Function

AWS Systems Manager Agent (SSM Agent) is lightweight Amazon software that can be installed and configured on an Amazon EC2 instance or an on-premises server/system. SSM Agent securely communicates with the Systems Manager service and gives this AWS service visibility and control of the managed servers.

Systems Manager has different capabilities that allow you, for example, to run a command on a managed instance, or forward a local network port on a client machine to any port inside a managed instance. SSM Agent runs on servers using root permissions (Linux) or SYSTEM permissions (Windows), which gives you full remote control of these servers using the Systems Manager service.

Depending on your use case, you may want to install and run SSM Agent on your server/system as a non-privileged user. The source code of SSM Agent is available on GitHub so you can adapt the agent to meet your needs.

This blog post describes how to do simple modifications of the SSM Agent source code to make it possible to run SSM Agent on Unix systems as a non-privileged user. AWS Lambda is chosen as a target platform for the SSM Agent deployment.

How to Run SSM Agent in a Lambda Function

Using the Systems Manager service and SSM Agent running in your Lambda function, you can be able, for example, to remotely run shell commands in your Lambda function or start a port forwarding session to this Lambda function (container).

Running SSM Agent in a Lambda function can not be considered as a practical use case, but it illustrates the power of the Systems Manager service. The open-source nature of SSM Agent, in turn, gives even more power and flexibility.

Flow diagram showing a Lambda function with running SSM Agent. An administrator can use the AWS CLI and Management console to execute commands in the Lambda function or create a port forwarding tunnel.

  • Some Systems Manager capabilities may not work with customized SSM Agent running in a Lambda function as a non-privileged user.

To run SSM Agent in a Lambda function, perform the following steps:

  1. Create a managed-instance activation for a hybrid environment.
    To register any server/system outside of Amazon EC2, you must create a Systems Manager activation.
    1. Open the AWS Systems Manager console at https://console.aws.amazon.com/systems-manager/home and switch to the AWS region of your choice.
    2. In the navigation pane, expand Instances & Nodes and choose Hybrid Activations.
    3. Click Create activation and use the following parameters for the new hybrid activation:
      • Activation description: AWS Lambda functions
      • Instance limit: 10
        Specify the total number of servers/systems that you want to register as a part of this activation.
        • In this procedure, every cold start of your Lambda function results in a new registration with Systems Manager.
      • IAM role: Choose Use the default role created by the system (AmazonEC2RunCommandRoleForManagedInstances)
    4. Click Create activation on the bottom of the page.
      Systems Manager returns the Activation Code and Activation ID to the console.
      Activation Code and Activation ID are strings that look something like a1B2c3D4e5F6g7H8i9J0 and abcd1234-56ef-aabb-7890-a1b2c3d4e5f6. Store them securely to use in the following steps.
  2. Create a Lambda layer that contains customized SSM Agent in the same AWS region where you created your Systems Manager activation.
  3. Create a Lambda function.
    Open the Lambda Management console at https://console.aws.amazon.com/lambda/home and switch to the same AWS region where you created your Systems Manager activation and Lambda layer. Then, create a new Lambda function with the configuration below.
    Runtime: Python 3.7
    Execution role: choose Create a new role with basic Lambda permissions.
    Function code: download lambda_function.py and use the Python code from this file.
    For further information about the function source code, see Appendix B, Lambda Function That Runs SSM Agent and a Simple HTTP Server (Python).
    Memory (MB): 2048 MB
    Timeout: 15 min 0 sec
    Layers: Add the Lambda layer that you created in Step 2.
    Environment variables: set the ACTIVATION_CODE and ACTIVATION_ID environment variables of your Lambda function (use your values from Step 1).
    The following screenshot shows an example configuration. Screen capture showing a sample configuration of the environment variables of the Lambda function.
  4. Invoke your Lambda function.
    You can provide any test event, for example, {}.
    • It takes a few seconds for SSM Agent to register with Systems Manager.
      You can invoke the Lambda function many times. If, in the subsequent runs, the container of the Lambda function is reused (warm start of the Lambda function), then SSM Agent uses the existing registration saved in vault (/tmp/ssm-agent/var/lib/amazon/ssm/Vault) and starts immediately.
  5. Observe the output of SSM Agent in the logs of the Lambda function.
    1. Open the CloudWatch management console at https://console.aws.amazon.com/cloudwatch/home and switch to the AWS region where you are running your Lambda function.
    2. In the navigation pane, choose Logs. Then, expand the log group that corresponds to the name of your Lambda function.
      In the logs of your Lambda function, you can find your Instance ID.
      Instance ID is a string that for on-premises servers/systems starts with the prefix mi-, for example, mi-123456789aabbccdd.
  6. Test different Systems Manager capabilities, while your Lambda function is running.
    For example, execute the following tasks:
  7. When you complete the tests, deregister your Lambda function container.
    In the Managed Instances view of the AWS Systems Manager console, select your Instance ID. Then, in the Actions menu, choose Deregister this managed instance. Screen capture showing how to deregister a managed instance.

Appendix A, How to Create a Lambda Layer That Contains Customized SSM Agent

The following procedure describes how to modify the SSM Agent source code to make it possible to run SSM Agent on Unix systems (and also inside a running Lambda function container) as a non-privileged user.

  • Do not use the resulting Lambda layer in production.
    The modifications of the SSM Agent source code used in this procedure are given only as a simplified working example.

To create the SSM Agent Lambda layer, perform the following steps:

  1. Install Docker on your PC/system.
  2. Create a new directory, for example, ssm-agent directory, and change to this directory.
    mkdir ssm-agent; cd ssm-agent
    
    Then, download to your current directory Dockerfile that contains the SSM Agent build instructions.
    curl -s -O https://cloudbriefly.com/file/post/0016/Dockerfile
    
  3. Build a new docker image that contains the Lambda layer with SSM Agent.
    docker build --rm --file Dockerfile ./ --tag ssm-agent-dev-image:latest
    
    The source code of the SSM Agent version 2.3.722.0 is modified and, then, compiled. The zip file with the Lambda layer content is saved at /root/amazon-ssm-agent-layer.zip inside the built docker image.
  4. Copy the zip file with the Lambda layer content to your current directory.
    From the built docker image create a dummy container without running it, copy the zip file from this container to your current directory, and, then, remove the container as follows:
    docker container create --name dummy-container ssm-agent-dev-image:latest
    docker cp dummy-container:/root/amazon-ssm-agent-layer.zip ./
    docker container rm dummy-container
    
  5. Create a new Lambda layer from the zip file.
    • The layer is compatible with all Lambda runtimes, except the Nodejs 10.x runtime.
    To create a new Lambda layer you can use the AWS Lambda management console or the AWS CLI, for example:
    aws lambda publish-layer-version --layer-name amazon-ssm-agent-layer \
    --zip-file fileb://amazon-ssm-agent-layer.zip \
    --compatible-runtimes nodejs8.10 java8 python3.7 dotnetcore2.1 go1.x \
    --region '<AWS Region>'
    

Appendix B, Lambda Function That Runs SSM Agent and a Simple HTTP Server (Python)

The following Python code snippet shows how you can run SSM Agent and a simple HTTP server in a Lambda function. The code is compatible with Python 3.6 and Python 3.7.

To use this source code in a Lambda function, you have to attach the SSM Agent Lambda layer to the execution environment of this function.

# CloudBriefly.com

import os
import subprocess
import signal
import time

os.environ['HOME'] = '/tmp/ssm-agent'
SSM_AGENT_LOG_DIR = '/tmp/ssm-agent/var/log/amazon/ssm'
SSM_AGENT_SHUTDOWN_WAITING_TIME = 30
HTTP_SERVER_PORT = 8080

try:
    # Register SSM Agent
    print(subprocess.check_output(
        ['/opt/bin/amazon-ssm-agent', '-register', '-y', '-code',
         os.environ['ACTIVATION_CODE'], '-id', os.environ['ACTIVATION_ID'],
         '-region', os.environ['AWS_REGION']],
        stderr=subprocess.STDOUT).decode('utf-8'))
except subprocess.CalledProcessError as e:
    print(e.output.decode('utf-8'))
    raise

def lambda_handler(event, context):
    # Run a simple HTTP server in background
    http_server = subprocess.Popen(['/var/lang/bin/python3', '-m',
                                    'http.server', str(HTTP_SERVER_PORT)],
                                   cwd=SSM_AGENT_LOG_DIR)

    # Run SSM Agent
    ssm_agent = subprocess.Popen(['/opt/bin/amazon-ssm-agent'])
    # Shut down SSM Agent gracefully and kill the HTTP server before
    # the Lambda function is timed out
    try:
        ssm_agent.wait(timeout=max(context.get_remaining_time_in_millis() / 1000
                                   - SSM_AGENT_SHUTDOWN_WAITING_TIME, 0))
    except subprocess.TimeoutExpired:
        # Send SIGINT signal to SSM Agent to shut down it gracefully
        ssm_agent.send_signal(signal.SIGINT)
        http_server.kill()
        time.sleep(SSM_AGENT_SHUTDOWN_WAITING_TIME * 3 / 4)

    ssm_agent.poll()
    print('Return code:', ssm_agent.returncode)

    if ssm_agent.returncode is None or \
        (ssm_agent.returncode != 0 and ssm_agent.returncode != -signal.SIGINT):
        raise subprocess.CalledProcessError(
            -99 if ssm_agent.returncode is None else ssm_agent.returncode,
            ssm_agent.args)

    return ssm_agent.returncode