Deploying AWS Lambda with CloudFormation

For an application to run on AWS, you need an AWS cloud environment to be built. For example, a serverless application would need API Gateway, Lambda, DynamoDB and many other services. Building it once is ok but if you need upgrading these services using AWS console or AWS CLI, will be a manual effort that can be cumbersome and error-prone.

So, AWS introduced the CloudFormation service, which enables infrastructure management as a service using templates. AWS CloudFormation takes care of provisioning and configuring the services for you. You don’t need to individually create and configure AWS services and resources. You don’t need to worry about dependencies.

When you apply the Lambda CloudFormation template, it creates an AWS CloudFormation stack. AWS CloudFormation provisions the API Gateway, Lambda function, and database for you. Once the stack has been successfully created, all AWS resources in the template are up and running. If you want to delete all the resources created by the CloudFormation template, you can just delete the stack. AWS CloudFormation can easily manage a collection of resources as a single unit.

Image Source: AWS

Though AWS CloudFormation can provision any resource in AWS, in this article, we will be focusing on the Lambda CloudFormation template and the resources used in it.

How Lambda CloudFormation Works

As described above, when you use CloudFormation, you use a template and a stack. A template consists of one or more resources and their properties. When you create a stack, AWS CloudFormation provisions the resources that are described in the template.

An AWS CloudFormation template is a JSON or YAML formatted text file created for Lambda functions. You can save these files with any extension, such as .json, .yaml, .template, or .txt. AWS CloudFormation uses these templates as blueprints for building Lambda and other AWS resources.

For Lambda CloudFormation, you will be using these resources:

AWS::Lambda::Function

The function is the main resource required for the Lambda service. To create a function, you need a deployment package and an execution role.

• The deployment package is the function code that you zip along with the dependencies. You need to set Code, Handler, and Runtime properties for it.
• The execution role grants the function permission to use other AWS services, such as DynamoDB, SNS, Amazon CloudWatch Logs for log streaming, and so on. You need to set the Role property for that.

Here is an example of Lambda CloudFormation:

{
  "Type" : "AWS::Lambda::Function",
  "Properties" : {
      "Code" : Code,
      "DeadLetterConfig" : DeadLetterConfig,
      "Description" : String,
      "Environment" : Environment,
      "FileSystemConfigs" : [ FileSystemConfig, ... ],
      "FunctionName" : String,
      "Handler" : String,
      "KmsKeyArn" : String,
      "Layers" : [ String, ... ],
      "MemorySize" : Integer,
      "ReservedConcurrentExecutions" : Integer,
      "Role" : String,
      "Runtime" : String,
      "Tags" : [ Tag, ... ],
      "Timeout" : Integer,
      "TracingConfig" : TracingConfig,
      "VpcConfig" : VpcConfig
    }
}

Other properties are optional and can be set as needed based on the use case.

AWS::Lambda::Function Code Property

The Code property is used to provide the path for the deployment package for a Lambda function. For all runtimes, you can upload the code to an S3 bucket and specify the location of an S3 object. For Node.js and Python functions, you can use the inline function code in the template itself.

If you need to change to a deployment package in Amazon S3, update the function code, change the object key or version in the template. If you don’t change the key or version, it won’t identify the change automatically with a stack update.

Here is a CloudFormation Lambda function example with a code resource:

{
  "S3Bucket" : String,
  "S3Key" : String,
  "S3ObjectVersion" : String,
  "ZipFile" : String
}

What is a Stack?

A stack is a single unit to manage all the related resources together. If you need to create, update, and delete a collection of resources, you do that by creating, updating, and deleting stacks. All the resources in a stack are defined by the AWS CloudFormation template discussed above.

Image Source: AWS

Let’s say that for your serverless application, you need AWS API Gateway, a Lambda function, DynamoDB, and an S3 bucket resource. For these resources, you have to create a stack by creating and submitting the CloudFormation template, and AWS CloudFormation will provision these resources in the AWS region you specified.

You can work on stacks with the AWS Console or the AWS CLI.

What are Change Sets?

Change Sets is a very important concept to understand to make changes to existing running resources in a stack. To make changes to your running resources in a stack, you need to update the stack and for that, you have to generate a Change Set. It is a summary of your proposed changes to the resources.

Change Sets enable you to see how these changes might impact your running resources, before implementing them.

Image Source: AWS

For example, if you change the name of a DynamoDB database instance, AWS CloudFormation assumes that you want to create a new database and delete the old one. This may impact your system as you may lose the data in the old database. To avoid it, you can generate a Change Set. You will see that your change will cause your database to be replaced. This will help you to plan upfront for any impact before you update your stack.

Quick Tutorial: Deploying a Lambda Function with CloudFormation

This is an abbreviated version of the excellent tutorial by Leah Erb. Before you begin, create an AWS account and ensure you install the AWS CLI on your local machine.

1. Create the CloudFormation Template

If you’re not familiar with the structure of a CloudFormation template, take a look at the “template anatomy” provided by Amazon. To automatically deploy a Lambda function, we will add information under the Resources section.

Create a YAML file for your new template (this tutorial will use YAML, but CloudFormation also supports JSON format).

Under the Resources section, we’ll start by defining the Identity and Access Management (IAM) Role. This creates an IAM role named HelloLambdaRole, and tells the Amazon Lambda service it is allowed to assume this role.

Keep in mind that if you needed your function to interact with other resources, you would need to assign roles for those resources as well.

Next, we define the Lambda Function. We name the function HelloLambaFunction, and specify that the HelloLambdaRole is allowed to run it. The code uses the!GetAtt function to get the ARN for the role name we defined above.

The Handler directive specifies which Python function should be run when the Lambda function is invoked.

In this case, the full code of the function is provided inline within the YAML file, and the ZipFile directive tells CloudFormation to dynamically create a ZIP file from the code. In a real life scenario, you would typically upload a ZIP file to S3 and reference it in your YAML.

2. Create the Stack

To create a stack from the template we defined above, run this command, substituting {TEMPLATE-FILE} with the path and filename of the template you defined in step 1.

aws cloudformation create-stack --stack-name hello-lambda-stack \
  --template-body {TEMPLATE-FILE} \
  --capabilities CAPABILITY_NAMED_IAM

You can check the status using this command list-stacks command. Ensure the status is CREATE_COMPLETE before moving forward.

3. Invoke Lambda Function

Your Lambda function has been automatically deployed by CloudFormation, and you can try to involve it. Run this command:

aws lambda invoke \
--invocation-type RequestResponse \
--function-name HelloLambdaFunction \
--log-type Tail outputfile.txt;  more outputfile.txt

You should see output like the below:

What is AWS SAM?

CloudFormation is a great service for automating the provisioning of AWS resources. As serverless applications are becoming more popular, AWS launched the Serverless Application Model (SAM), intended to make it easier to build and test serverless applications. SAM is built on top of CloudFormation and reduces the lines of code you need to write YAML or JSON files. It also provides a CLI to create the local Lambda environment to debug and test the function locally before deploying to the AWS cloud.

Additional CloudFormation Lambda Entities

When you define your CloudFormation Lambda template, you can use these additional entities to fine tune the behavior of the resulting Lambda deployment.

AWS::Lambda::Function DeadLetterConfig

DeadLetterConfig is used for handling asynchronous Lambda call failures.

The syntax is below for this property:

{
  "TargetArn" : String
}

AWS::Lambda::Function Environment

The environment is used to set properties, a configuration in the Lambda instance environment variables.

{
  "Variables" : {Key : Value, ...}
}

AWS::Lambda::Function VpcConfig

VpcConfig is required when you want to configure Lambda to access VPC resources. To connect a function to a VPC, Lambda creates an elastic network interface (ENI) for each combination of Security Group and subnet in the function’s VPC configuration. The function can only access resources and the Internet through that VPC.

{
  "SecurityGroupIds" : [ String, ... ],
  "SubnetIds" : [ String, ... ]
}

AWS::Lambda::Alias

Lambda alias is created to abstract the version of a Lambda function. It provides clients with a function identifier that you can update to invoke any version of a function.

You can also map an alias to implement canary deployment by splitting the invocation requests between two versions. You can use the RoutingConfig parameter to specify a second version and the percentage of invocation requests that it receives.

{
  "Type" : "AWS::Lambda::Alias",
  "Properties" : {
      "Description" : String,
      "FunctionName" : String,
      "FunctionVersion" : String,
      "Name" : String,
      "ProvisionedConcurrencyConfig" : ProvisionedConcurrencyConfiguration,
      "RoutingConfig" : AliasRoutingConfiguration
    }
}

Below is an example of a CloudFormation Lambda function with alias:

{
   "Resources": {
      "function": {
         "Type": "AWS::Lambda::Function",
         "Properties": {
            "Handler": "welcome.handler",
            "Role": "arn:aws:iam::214616739002:role/lambdaalias-role",
            "Code": {
               "ZipFile": "exports.handler = function(event){\n    console.log(JSON.stringify(event, null, 2))\n    const response = {\n        statusCode: 200,\n        body: JSON.stringify('Hello from Lambda Alias and versioning!')\n    }\n    return response\n};\n"
            },
            "Runtime": "nodejs12.x",
            "TracingConfig": {
               "Mode": "Active"
            }
         }
      },
      "version": {
         "Type": "AWS::Lambda::Version",
         "Properties": {
            "FunctionName": null,
            "Description": "v1"
         }
      },
      "alias": {
         "Type": "AWS::Lambda::Alias",
         "Properties": {
            "FunctionName": null,
            "FunctionVersion": null,
            "Name": "BLUE"
         }
      }
   }
}

Here is a sample YAML CloudFormation Lambda function example for ProvisionedConcurrencyConfig that is used to launch multiple instances in Lambda execution environments.

alias:
    Type: AWS::Lambda::Alias
    Properties:
      FunctionName: !Ref function
      FunctionVersion: !GetAtt newVersion.Version
      Name: BLUE
      ProvisionedConcurrencyConfig:
        ProvisionedConcurrentExecutions: 20

AWS::Lambda::EventInvokeConfig

The EventInvokeConfig resource is used for a CloudFormation Lambda trigger. It configures options for asynchronous invocation.

By default, if the asynchronous invocation of a function fails and returns an error, Lambda retries twice by retaining events in a queue for up to six hours. When Lambda fails to invoke it with all processing attempts and stays in the queue for too long, Lambda discards it.

Here is a template for Lambda CloudFormation EventInvokeConfig:

{
  "Type" : "AWS::Lambda::EventInvokeConfig",
  "Properties" : {
      "DestinationConfig" : DestinationConfig,
      "FunctionName" : String,
      "MaximumEventAgeInSeconds" : Integer,
      "MaximumRetryAttempts" : Integer,
      "Qualifier" : String
    }
}

The AWS::Lambda::EventSourceMapping

The EventSourceMapping resource creates a mapping between an event source and a Lambda function. It configures which event source triggers a Lambda function and defines all the properties to control the behavior to trigger the function.

"KinesisSourceMapping": {
    "Type": "AWS::Lambda::EventSourceMapping",
    "Properties": {
        "EventSourceArn": {
            "Fn::Join": [
                "",
                [
                    "arn:aws:kinesis:",
                    {
                        "Ref": "AWS::Region"
                    },
                    ":",
                    {
                        "Ref": "AWS::AccountId"
                    },
                    ":stream/",
                    {
                        "Ref": "KinesisStream"
                    }
                ]
            ]
        },
        "FunctionName": {
            "Fn::GetAtt": [
                "LambdaFunction",
                "Arn"
            ]
        },
        "StartingPosition": "TRIM_HORIZON"
    }
}

AWS::Lambda::LayerVersion

The LayerVersion resource is used to manage the dependencies externally and reduce the size of a deployment package.

Here is an example of a Lambda layer version:

"PythonLayer": {
    "Type": "AWS::Lambda::LayerVersion",
    "Properties": {
        "CompatibleRuntimes": [
            "python3.6",
            "python3.7"
        ],
        "Content": {
            "S3Bucket": "python-bucket-ap-east-2-214616739002",
            "S3Key": "pythonlayer.zip"
        },
        "Description": "Python dependency layer",
        "LayerName": "python-layer",
        "LicenseInfo": "MIT"
    }
}

AWS::Lambda::Permission

The CloudFormation Lambda permission resource is used to grant an AWS service, or another account, permission to use a function. A policy can be applied at the function level or specific to a version or alias. To grant permission to another account, you need to specify the account ID as the Principal in the policy.

Below is an example of a CloudFormation Lambda permission for an S3 resource to call the function and process a configured bucket:

"s3triggerPermission": {
    "Type": "AWS::Lambda::Permission",
    "Properties": {
        "FunctionName": {
            "Fn::GetAtt": [
                "function",
                "Arn"
            ]
        },
        "Action": "lambda:InvokeFunction",
        "Principal": "s3.amazonaws.com",
        "SourceAccount": {
            "Ref": "AWS::AccountId"
        },
        "SourceArn": {
            "Fn::GetAtt": [
                "bucket",
                "Arn"
            ]
        }
    }
}

Now that we discussed the template in-depth, let’s briefly talk about the stack that is created using this template.

Conclusion

In this article, we reviewed AWS Lambda Cloudformation, one of the most talked-about services in the AWS serverless developer community, and its significance to improving the continuous deployment and delivery process. In addition, we covered how AWS SAM was introduced to extend the CloudFormation features specific to Lambda services, event sources, and other AWS services to manage them more effectively through one specification.