A while back, I wrote about how you can shave latency off every AWS SDK operation by enabling HTTP keep-alive, like this:
It had the desired effect and I saw lots of people apply this technique in their projects. But it also resulted in the same 10 lines of code being copied and pasted everywhere! I began thinking about ways to distribute an optimized version of AWS SDK so everyone can benefit.
Do I publish and maintain a separate NPM package, i.e. @theburningmonk/aws-sdk? As an approach, it will cause more friction than I’d like:
- Every project would need to explicitly require my NPM package. That’s an easy thing to forget! Not to mention you have to go back and patch all your older projects.
- It doesn’t work with shared libraries that already depend on the official aws-sdk. Many companies have developed their own tooling layer consisting of many shared libraries. They would have to rewrite their tools.
- It doesn’t work with other open source projects that already depend on the official aws-sdk.
Then it struck me that Lambda Layer is the perfect distribution channel in this case.
Introducing the optimized-aws-sdk layer
With the support of Lumigo, I created a Lambda layer for an optimized AWS-SDK! The layer is available at the following ARN:
arn:aws:lambda:us-east-1:374852340823:layer:optimized-aws-sdk:<latest version>
You can include it in your existing projects using the Serverless framework:
Or with SAM:
Or add it via the console manually:
At runtime, dependencies that are packaged with your project would take priority over libraries injected via layers. To make sure the optimized AWS SDK is required at runtime, you should not package the AWS SDK with your deployment. Instead, add the AWS SDK as a dev dependency if you still need it to run tests locally.
Introducing the autodeploy-layer app
Even then, it’s still easy to forget to add the layer to your functions. Not to mention the maintenance headache of updating all the functions to a new version of this layer later. This is one of the challenges with using layers, which we discussed in a previous post. To help you manage the rollout and update of layers, we have also published a new app to the Serverless Application Repository (SAR):
The source code is available here and you can install the app directly from the page above. However, doing so introduces a manual process and breaks infrastructure-as-code (IaC). Instead, the prefered way to deploy a SAR app is through existing deployment tools such as SAM or the Serverless framework.
You can configure the deploy of the autodeploy-layer app with SAM by adding an AWS::Serverless::Application resource, like this:
To make this work for the Serverless framework, you can use this simple trick to add support for the AWS::Serverless::Application resource type (which is not native to CloudFormation).
Once you have deployed the autodeploy-layer app, it’ll automatically add the layer to every function in the region. If you only want to add the layer to some of the functions, then the app also supports filtering by tags. You can use the include and exclude tags to control which functions would receive the specified layer.
We will publish new versions of the optimized-aws-sdk layer when the AWS SDK is updated. You can find the ARN for the corresponding version of the AWS SDK here. When a new version of the layer is available, you can roll it out to all opted-in functions by redeploying the autodeploy-layer app with the updated ARN.
To use the autodeploy-layer app to manage multiple layers, you have to deploy multiple instances of the app, one for every layer. If you are using the SAM or Serverless framework, then you can do this by adding another AWS::Serverless::Application resource to the stack.
So there you have it, two free tools to help make all your Lambda functions faster!
- optimized-aws-sdk layer: a version of the AWS SDK with HTTP keep-alive enabled
- autodeploy-layer SAR app: to manage the rollout and update of layers
As always, please let us know your thoughts and feedback on these tools, and please submit any feature requests or bugs against the relevant repositories.