re:Invent 2023 day 0 recap

In typical re:Invent fashion, AWS has made a series of big announcements before the main event even kicks off!

Here are some groundbreaking serverless updates that you should know about.

Lambda scales 12x faster

Here’s the official announcement.

Lambda has completely revamped how its burst concurrency limit works. Previously there was a region-wide burst limit between 500–3000 and a refill rate of 500 per minute. So if you have a full account concurrency of 10,000 then it will take you 14 mins to reach this max throughput while getting throttled along the way.

With the new scaling behaviour, every function can burst to 1000 concurrent executions instantly. And the limit rises by another 1000 every 10 seconds. Importantly, each function can scale independently.

This unblocks the use of Lambda in situations where you might experience sudden spikes of traffic, such as flash sales.

Assuming an average request time of 100ms, a single concurrent execution can handle 10 requests per second (RPS). Assuming you have one function per API endpoint, that translates to being able to burst to 10,000 RPS per endpoint. And an additional 10,000 RPS per endpoint every 10 seconds until you reach your full account-level concurrency limit.

With this much scalability, you’re much more likely to hit throttling limits elsewhere in your system. Such as API Gateway, which has a default region-wide limit of 10,000 RPS.

Step Functions gets public HTTP endpoints

Here’s the official announcement.

Previously, Step Functions only had direct integration with API Gateway endpoints. So you still had to use Lambda to call 3rd party APIs, or set up your own API Gateway proxy.

No more.

Now you can have Step Functions call any public APIs.

One thing to note is that Step Functions uses the existing HTTP connections from EventBridge. So whatever connection you’ve set up in EventBridge would show up here, and vice versa.

Step Functions lets you test individual states

Here’s the official announcement.

You can now test individual states without executing your state machine, This is super useful for testing those hard-to-reach conditional branches.

Previously, the best way to test these was to run Step Functions Local and use mock responses to drive the execution to where you want to test.

With this change, you can do it with a simple API call to Step Functions. It can be done programmatically without using the console. There’s a new TestState endpoint you can call with the definition of your state to test it.

I’m very excited to test this out and update my Testing Serverless Architectures course accordingly.

Step Functions adds optimized integration with Bedrock

Here’s the official announcement.

Of course, they did. Everyone’s building AI apps and this just made that easier.

There are still cases where you want to use Lambda instead though. Because you can stream responses with Lambda and Function URLs.

If the request is coming from a frontend application, then you can use Lambda with response streaming to improve user experience.

CloudWatch gets AI assistant

Here’s the official announcement.

You can now use English to query CloudWatch logs and metrics. Very cool!

BUT… you still need to know your system. The example below generates a valid query, but it doesn’t work.

The AI doesn’t get it right a lot of the time, but it gives you a useful starting point to work from.

CloudWatch adds Infrequent Access log class

Here’s the official announcement.

If you don’t use any of these features, then you can save 50% on CloudWatch Logs ingestion cost by switching to the new Infrequent Access log class.

CloudWatch Logs adds anomaly detection

Here’s the official announcement.

The anomaly detection looks great, and I like the fact that you can compare your log patterns with a previous period to see how they have changed.

EventBridge adds partner integration with Adobe and Stripe

Here’s the official announcement.

Back in Aug 2022, EventBridge announced integration with Stripe via webhooks using Quick Starts. Now it’s a fully managed feature. Nice!

CloudFormation syncs with Git

Here’s the official announcement.

This is an odd one. CloudFormation can now synch with a template file in your Git repository and automatically deploy your updates. In a way, you can do without a CI/CD pipeline for very simple stacks.

It’s not for everyone, but as Ben Kehoe pointed out, there are lots of small admin/ops/security stacks that can benefit from this.

For your application stack, where you want to run tests, bundle Lambda functions, etc. You’re better off doing them in your existing CI/CD pipeline. Rather than splitting out the CloudFormation deployment step into a separate branch in order for this to work.

However, this is a pretty good argument for doing exactly that. This is relevant in many enterprise environments where there are strict access controls in place. Compared to the gymnastics people have to do to get their application deployed, this is actually the lesser evil.