HTTP API goes GA today!

As you read this, API Gateway HTTP API is now Generally Available. This is an important milestone, and kudos to the AWS API Gateway team for turning this around so quickly. As part of the GA announcement, they also introduced a number of features, including:

• stage and route level throttling
• mix and match REST and HTTP APIs in the same custom domain name
• payload format version 2.0
• integration to private resources (private ALB, NLB and CloudMap)

As with other AWS Serverless announcements, Lumigo platform supports HTTP APIs with the availability of the feature in AWS.

Let’s get started with a quick walkthrough of the new features revealed in today’s announcement, and why you should care about them.

Payload Format Version 2.0

You can now change the format of the request and response to and from the Lambda function. To enable this, change the PayloadFormatVersion to 2.0. This maps to the PayloadFormatVersion attribute on the AWS::ApiGatewayV2::Integration resource in CloudFormation. It can also be changed in the console.

Once switched to v2.0, the event payload HTTP API sends to your Lambda function changes to the following.

{
  "version": "2.0",
  "routeKey": "ANY /",
  "rawPath": "/",
  "rawQueryString": "q=value1&q=value2",
  "queryStringParameters": {
    "q": "value1,value2"
  },
  "cookies": [
    "cookie1",
    "cookie2"
  ],
  "headers": {
    "accept": "...",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "...",
    "cache-control": "no-cache",
    "content-length": "0",
    "host": "lwvbht635a.execute-api.us-west-1.amazonaws.com",
    "pragma": "no-cache",
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "none",
    "sec-fetch-user": "?1",
    "upgrade-insecure-requests": "1",
    "user-agent": "...",
    "x-amzn-trace-id": "Root=1-5e696e47-8950b7f5de7bd300b6b5eb4f",
    "x-forwarded-for": "77.173.243.53",
    "x-forwarded-port": "443",
    "x-forwarded-proto": "https"
  },
  "requestContext": {
    "accountId": "775708144865",
    "apiId": "lwvbht635a",
    "domainName": "lwvbht635a.execute-api.us-west-1.amazonaws.com",
    "domainPrefix": "lwvbht635a",
    "http": {
      "method": "GET",
      "path": "/",
      "protocol": "HTTP/1.1",
      "sourceIp": "77.173.243.53",
      "userAgent": "..."
    },
    "requestId": "JP4rNhmXSK4EMGw=",
    "routeId": null,
    "routeKey": "ANY /",
    "stage": "$default",
    "time": "11/Mar/2020:23:03:35 +0000",
    "timeEpoch": 1583967815603
  },
  "isBase64Encoded": true
}

Notice that it’s a lot lighter than the v1.0 schema (below).

"version": "1.0",
  "resource": "/",
  "path": "/",
  "httpMethod": "GET",
  "headers": {
    "Content-Length": "0",
    "Host": "lwvbht635a.execute-api.us-west-1.amazonaws.com",
    "User-Agent": "...",
    "X-Amzn-Trace-Id": "Root=1-5e696d5f-bf736624f6fcefb2150f9609",
    "X-Forwarded-For": "77.173.243.53",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https",
    "accept": "...",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "...",
    "cache-control": "max-age=0",
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "none",
    "sec-fetch-user": "?1",
    "upgrade-insecure-requests": "1"
  },
  "multiValueHeaders": {
    "Content-Length": [
      "0"
    ],
    "Host": [
      "lwvbht635a.execute-api.us-west-1.amazonaws.com"
    ],
    "User-Agent": [
      "..."
    ],
    "X-Amzn-Trace-Id": [
      "Root=1-5e696d5f-bf736624f6fcefb2150f9609"
    ],
    "X-Forwarded-For": [
      "77.173.243.53"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ],
    "accept": [
      "..."
    ],
    "accept-encoding": [
      "gzip, deflate, br"
    ],
    "accept-language": [
      "en-US,en;q=0.9,zh-TW;q=0.8,zh;q=0.7,zh-CN;q=0.6,it;q=0.5"
    ],
    "cache-control": [
      "max-age=0"
    ],
    "sec-fetch-dest": [
      "document"
    ],
    "sec-fetch-mode": [
      "navigate"
    ],
    "sec-fetch-site": [
      "none"
    ],
    "sec-fetch-user": [
      "?1"
    ],
    "upgrade-insecure-requests": [
      "1"
    ]
  },
  "queryStringParameters": {
    "q": "value2"
  },
  "multiValueQueryStringParameters": {
    "q": [
      "value1",
      "value2"
    ]
  },
  "requestContext": {
    "accountId": "775708144865",
    "apiId": "lwvbht635a",
    "domainName": "lwvbht635a.execute-api.us-west-1.amazonaws.com",
    "domainPrefix": "lwvbht635a",
    "extendedRequestId": "JP4G8i0nSK4EMIw=",
    "httpMethod": "GET",
    "identity": {
      "accessKey": null,
      "accountId": null,
      "caller": null,
      "cognitoAuthenticationProvider": null,
      "cognitoAuthenticationType": null,
      "cognitoIdentityId": null,
      "cognitoIdentityPoolId": null,
      "principalOrgId": null,
      "sourceIp": "77.173.243.53",
      "user": null,
      "userAgent": "...",
      "userArn": null
    },
    "path": "/",
    "protocol": "HTTP/1.1",
    "requestId": "JP4G8i0nSK4EMIw=",
    "requestTime": "11/Mar/2020:22:59:43 +0000",
    "requestTimeEpoch": 1583967583593,
    "resourceId": null,
    "resourcePath": "/",
    "stage": "$default"
  },
  "pathParameters": null,
  "stageVariables": null,
  "body": null,
  "isBase64Encoded": true
}

Here are a few of the key differences:

• multiValueHeaders are removed
• multiValueQueryStringParameters are removed
• duplicated query string parameters are merged and comma-separated in the queryStringParameters field (whereas in v1.0, only one value is kept and the rest are dropped and must be fetched from multiValueQueryStringParameters)
• similarly, duplicated headers are merged and comma-separated in the headers field
• there is a new cookies field

These provide better handling for multi-value headers and query string parameters. It’s useful for cases where you need to pass an array of values into an endpoint. The HTTP API parser would now merge them into a comma-separated string for you automatically.

Simplified Lambda response

With v1.0, your function has to return a payload like so:

{
  "isBase64Encoded": false,
  "statusCode": 200,
  "headers": { 
    "headerName": "headerValue", ...
  },
  "multiValueHeaders": {
    "headerName": ["headerValue", "headerValue2", ...], 
    ... 
  },
  "body": "..."
}

Some of these fields can be omitted, but you must return the statusCode at minimum. However, most of the time, we see functions that return a JSON payload like this:

{
  "statusCode": 200,
  "body": "{ ... }"
}

With v2.0 payload format, you can simply return an object and it will be automatically JSON serialized and returned as a HTTP 200 response.

This would simplify the success path for Lambda functions behind HTTP API. Many teams are using middleware and wrappers to provide similar shortcuts. This new feature would remove the need for these custom-built solutions.

Custom Domain Cross Compatibility

Another important new feature is that you’re now able to mix REST and HTTP APIs under the same custom domain name.

This is especially useful for those of us who are stitching multiple APIs together using custom domain names. In this context, when you migrate these APIs from REST to HTTP you will need to be able to mix different APIs under the same custom domain name. Otherwise, you would have to find another way (e.g. using CloudFront) to stitch these APIs under the same domain name.

This is a very welcomed change indeed!

Private Integrations

If you have private ALBs or NLBs, you can now integrate them with HTTP APIs too. To do that, start by creating a VPC link for HTTP APIs.

You would need to configure the VPC Id, subnets, and security groups. Please keep in mind that once the VPC link is created, you will not be able to change the security groups.

This process can take a minute or two to complete. The ENIs are provided during this creation process so there is no runtime overhead (think Lambda cold start) for accessing private resources through a VPC link.

Now that you have a VPC link, it’s time to create an integration. For ALBs and NLBs, choose ALB/NLB and find the load balancer and listener you’d like to integrate with. And finally, select the VPC link you have just created.

All that’s left then, is to attach the integration to a route.

Congratulations, you can now access private ALB/NLB resources through HTTP API!

Why is this important?

Personally, I prefer zero-trust networking where every request through the system needs to be authenticated and authorized. It’s much more secure than the traditional model of hiding behind private networks where you are still vulnerable to attacks from within. For example, an attacker can gain access to your private network by compromising one of your application dependencies and gain the ability to execute code from within the network. This allows the attacker access to everything in the network.

You can implement a zero-trust security model by dedicating authentication and authorization to API Gateway. You can use a combination of AWS_IAM, Cognito User Pool and custom Lambda authorizer with API Gateway. As I explained above, this provides much stronger security to your system than hiding them behind VPCs.

With this new private integration, it allows you to expose these private resources to the outside world in a zero-trust fashion. However, keep in mind that at the time of writing HTTP API only supports Cognito User Pool and OAuth 2.0 as authentication methods. Support for AWS_IAM and custom Lambda authorizer is likely to be forthcoming in the near future.

Furthermore, this private integration also works with CloudMap, which provides service discovery capability to ECS-based applications. You can integrate with a service that is registered with CloupMap by specifying its Namespace and Service name.

As before, you can attach this CloudMap integration to a route in the HTTP API to provide access to the CloudMap-registered service.

Throttling

Another useful addition is the ability to specify stage and route-level throttling.

This is another sorely missed security feature that REST APIs has, but was missing in HTTP APIs during the preview.

Lumigo support for HTTP APIs is live!

Lumigo has been working with AWS during the beta in order to add the ability to monitor HTTP APIs via Lumigo platform.

So, we’re very pleased to announce Lumigo platform support for HTTP APIs available to all Lumigo users from today.

In the System Map view, you can now filter by service to see all of your HTTP APIs (or REST APIs), along with their first-degree connections.

Since a call from a REST API to a Lambda function costs more than from the new HTTP APIs, this new feature makes it a breeze to identify where you should move to the new HTTP APIs to take advantage of the cost savings. 

Start taking advantage of the new HTTP API straight away using the new Lumigo filter tool. See where you could be making savings in your serverless environment.

Wrap up

I’m really excited with today’s announcement, and very happy to see these new features being added to HTTP API. With them, the feature disparity between REST and HTTP APIs are getting much smaller.

The Serverless framework has also added support for HTTP API recently, so you can more easily take advantage of this significantly cheaper (~70%!) alternative to API Gateway REST APIs. Please, give HTTP API a try and let us know what you think of the new features, especially the new payload format.