When used properly, serverless technologies like AWS Lambda can lower the cost of running a system. This is because you only pay for these services when you’re using them, so you don’t waste any money.
Serverless technologies also have other benefits. They can provide better security, built-in redundancy and scalability. The biggest plus is that they let you do more with less time and effort. You can focus on the things that directly add value to your business. By leveraging the cloud to its full potential and letting the cloud handle the infrastructure for you, you can get a lot more done.
But, this doesn’t mean that serverless is the right choice for every situation. I’m not saying serverless solutions will always cost less than traditional server-based ones. Actually, I’ve seen many people make mistakes when trying to figure out how much their serverless systems will cost.
So, let’s talk about the 7 most common cost-related mistakes people make with serverless systems, and how you can avoid them!
1. Overlooking monitoring and logging costs
Monitoring and logging are critical for maintaining the health of your application. But, they come at a cost. For example, AWS CloudWatch charges you for both ingesting and storing data.
It’s not surprising to see CloudWatch costing a lot more than Lambda and API Gateway on an AWS bill. Sometimes, by an order of magnitude!
Here are some ways to keep your CloudWatch costs under control:
- Log at
INFOorERRORlevel in production to reduce the ingestion cost. - Sample
DEBUGlogs in production based on the volume of traffic. Ideally, you should have just enough debug logs to cover all parts of your code. This will help you to solve problems when they pop up. - Set log retention to 30 days. This stops storage costs from building up over time. At the same time, it gives you enough data to investigate problems that happened recently.
If you are using a third-party log aggregation platform, you can skip CloudWatch Logs altogether. To do this:
- Make sure your Lambda function can’t send logs to CloudWatch Logs by not giving it
logs:PutLogEventsIAM permission. - Use Lambda Extensions to send logs to the third-party service. For more details, please check out my post on Lambda Telemetry API.
2. Not reading the documentation
A common mistake people make is using an AWS service in production before they fully understand how it’s priced. This “wait-and-see” approach can often lead to unpleasant billing surprises.
Each AWS service has its own pricing page. For instance, the Lambda service’s pricing page shows all the different ways you can be charged (as shown below). Some of these charges only apply if you use certain features, like Provisioned Concurrency.
It’s always smart to understand the pricing of a service or feature before using it in anger. Most AWS billing surprises I’ve seen could have been avoided if people had a better understanding of the pricing for the services they were using.
Don’t skip steps. Take the time to learn how a service works and how it’s priced before jumping in. This initial time investment can save you a lot in the long run.
3. No billing alarms
Another mistake people frequently make is not setting up billing alarms in AWS. These alarms are crucial to keep track of your expenses, and not using them can lead to unexpected high costs.
Billing alarms in AWS work like a heads-up system. They alert you when your costs reach a certain threshold that you set. Without these alarms, you might not notice increasing costs until you receive a larger than expected bill at the end of the month.
Setting up these alarms should be one of the first things you do when using AWS. With these alarms, you’ll be alerted when your costs are climbing, giving you a chance to investigate and address the issue before it gets out of hand. This simple step can help you keep your AWS costs under control and avoid unpleasant surprises in the future.
Billing alarms may not be perfect as the billing data in AWS usually lags a few hours behind. However, it’s still better to get notified about a potential cost issue a bit late, rather than discovering it at the end of the month when the costs have piled up significantly!
4. Synchronous Lambda-to-Lambda invocations
Another common mistake is when people make one Lambda function call another synchronously. Meaning the first one waits for the second one to finish before it can continue. This approach can significantly increase your Lambda costs because you end up paying for the execution time of both functions.
Let’s clarify this with an example. Suppose you have Lambda Function A that calls Lambda Function B. If Function A calls Function B synchronously, Function A has to wait until Function B finishes running before it can continue. During this wait time, you’re being billed for both Function A’s and Function B’s execution time, even though Function A isn’t doing anything useful.
To avoid this unnecessary cost, consider making asynchronous invocation whenever possible. With an asynchronous invocation, Function A could send the request to Function B and then continue with its own work, not waiting for Function B to finish. This way, you are not paying for Function A to just sit and wait.
I’ve talked more about Lambda-to-Lambda calls in a different post. The main point is that it’s usually not a good idea to do this! But there’s one exception. You can use this method to give secondary tasks to another function by calling it asynchronously. This allows the first function to finish earlier, which is useful for reducing user-facing latency in an API function.
To invoke a function asynchronously, you can use Lambda’s Invoke API and set the InvocationType parameter to Event.
5. Missing Caching
Caching is a critical, yet often overlooked, mechanism to efficiently manage system resources and reduce operational costs.
Not using caching can lead to more requests reaching your backend API and Lambda functions. This results in your systems doing more work, which costs you more money. By using caching, you can reduce the workload of your systems and lower costs.
In another post, I’ve delved deeper into the subject of caching, examining various places within your system where caching could be beneficial. For instance, CloudFront enables caching at the edge, closest to your users. Alternatively, API Gateway’s built-in caching mechanism provides another level of efficiency. Beyond these, application-level caching is possible by using a service like Amazon ElastiCache or Momento, storing previously fetched results for quicker access.
Another option is DynamoDB Accelerator (DAX), which offers a fully managed, transparent caching layer to DynamoDB, further improving system efficiency.
There are multiple ways to implement caching in your system. And you can combine these in different ways to meet your specific needs. When utilized effectively, caching can substantially reduce your AWS costs. In addition, it can also enhance the performance and scalability of your system, resulting in a win-win scenario for performance and cost efficiency.
6. Over-Provisioning
Over-provisioning memory for Lambda functions is another cost-related mistake that many make. It can increase your expenses without necessarily enhancing the performance of your system.
Lambda functions in AWS are charged based on two factors: the amount of memory allocated and the execution time of the function. When you allocate more memory than your function needs, you increase the cost per execution. The catch here is that increasing memory doesn’t always result in a proportional increase in function performance. You might be paying more for resources that your function doesn’t use or need.
To avoid over-provisioning, it’s vital to understand the memory requirements of your Lambda functions. Tools such as AWS Lambda Power Tuning can help you find the optimal memory configuration that balances cost and performance. Continually monitoring and adjusting memory allocation based on your function’s needs can lead to sizable cost savings.
But you shouldn’t blindly power-tune all of your functions. Because for the vast majority of your Lambda functions, there are likely no meaningful cost savings to be had.
If you use Lumigo, then a good way to identify good targets for power tuning is to go to the Functions tab and sort your functions by cost in descending order.
7. Poor Architecture Design
Finally, poor architecture design is another common cost mistake in serverless architectures. Having superfluous components in your architecture or selecting unsuitable services for your scale can inflate costs without contributing to your system’s performance or efficiency.
One facet of poor design is having too many unnecessary moving parts. The more moving parts your architecture has, the more you’ll likely have to pay. And not just in terms of AWS charges, but also in maintenance and monitoring efforts. Each additional component brings with it an overhead cost in managing, running, and securing that component.
I discussed this in the context of whether you still need SQS between SNS and Lambda, which you can read about here. In general, I like to start with the simplest solution, and if it ticks all my boxes then I stop there. There should be a good reason to add a moving part to your architecture.
Conversely, under-engineering or selecting an unsuitable service can also be costly. If the service selected cannot scale efficiently to handle your workload, it can lead to performance degradation, or worse, service disruptions. Selecting a service without considering its scalability in relation to your specific needs can result in higher costs in the long run due to scaling inefficiencies.
For example, AWS offers many messaging services. There’s SNS, SQS, EventBridge and Kinesis just to name a few. Each of these services has different trade-offs and capabilities.
Despite their differences, they can all perform the basic function of capturing messages and delivering them to target consumers. However, the cost of using these services can differ significantly depending on the scale of your application.
Similarly, API Gateway can cost significantly more than ALB at scale but it also offers more capabilities.
AWS services are the new data structures and they are the building blocks of cloud-native applications. Choosing the right service for the job can make a huge difference to your system’s performance, scalability and AWS spending. It’s important to take the time to learn the services you use. Understand how they work, their trade-offs and how they’re priced.
And use the AWS Well-Architected framework and the serverless lens as guidelines to help you design better serverless architectures.
Conclusion
In conclusion, successfully navigating the landscape of serverless requires a good understanding of AWS services, thoughtful application design, and careful resource management. Making smart choices about when and how to use AWS services can significantly optimize your costs and improve your system’s efficiency and performance.
One key point to remember is that cost optimization isn’t just about choosing the cheapest service. It’s about striking a balance between performance, scalability, and cost.
The seven common cost-related mistakes we outlined above are pitfalls you can avoid by taking a deliberate and informed approach. Avoiding these mistakes can lead to substantial savings and make your serverless journey more effective and enjoyable.
Remember that the goal of serverless is to empower you to do more with less — less time, less effort, and less money. So, take advantage of these technologies and the scalability, redundancy, and security they offer. Leverage them to your benefit and focus your efforts on what matters most: adding value to your business.
Every mistake is a learning opportunity. So, let’s learn from these common mistakes and let serverless be an accelerator for your success. With careful planning, monitoring, and continual learning, your journey with AWS can be cost-effective, efficient, and rewarding.
Don’t be afraid of making mistakes. Embrace them, learn from them, and always strive to optimize. The world of serverless is yours to explore and master. Happy exploring!