Serverless DynamoDB

  • Topics

Serverless DynamoDB: How to Use It with Serverless Framework

DynamoDB is one of the best known NoSQL databases and is widely used by AWS developers. It is built to deliver single-digit millisecond performance at any scale — and is one of the favored database choices by serverless developers.

It is a key-value and document-based database that is fully managed, available in multiple regions, with built-in security, backup, and security. DynamoDB can handle 10 trillion requests per day. No other database claims throughput that high.

DynamoDB stores its data in tables in the form of items. Each item has a set of fields or attributes. Each table has a primary key applicable to all the items within the table. This primary key is either a single field or a combination of two fields: a partition key and a sort key. The primary key is used to reference specific items in a table, or you can create your own indices and use the keys from those indices.

DynamoDB exposes an HTTP API to access the tables for read and write. The reason for having this HTTP API is to provide the scalability behind the proxy where DynamoDB tables are stored in multiple hosts and replicas are created. You can access these APIs using AWS SDK or AWS CLI.

When you build a serverless architecture application, the key thing to remember is that all the components of this architecture should be able to scale based on the application’s needs. With the benefit of auto-scaling, pay-per-use, self-provisioning, and no server management, DynamoDB becomes the easy choice for many serverless applications. DynamoDB also provides easy integration with other AWS services, enabling a serverless architecture. Some of the use cases in which you can use DynamoDB include:

  • Using DynamoDB Streams as an event source for triggering Lambda functions
  • A REST API triggered through AWS API Gateway, with Lambda functions and DynamoDB (a serverless backend for a web application)
  • A serverless chat application with AppSync and a DynamoDB database.

Serverless Framework and DynamoDB

Serverless Framework is an open-source project that supports building serverless applications on AWS and other clouds. It is built on top of CloudFormation scripts supported by AWS. Using Serverless Framework makes it easy to integrate DynamoDB and other serverless components.

For Serverless Framework, you need to write a serverless.yml file. In this yml file, you need to configure DynamoDB in the resource section.

In Serverless Framework, when you use AWS as a provider, all the resources are the other AWS services that are called by the AWS Lambda function mentioned in the service section.

# serverless.yml

service: customersCrud
provider: aws
functions:

resources: # CloudFormation template syntax
  Resources:
    usersTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: customersTable
        AttributeDefinitions:
          - AttributeName: name
          AttributeType: S
        KeySchema:
          - AttributeName: name
          KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1

To create this resource in AWS, you need to run the serverless deploy command in your serverless project.

Let’s take a look at several plugins provided by serverless frameworks to manage several aspects of DynamoDB.

Serverless DynamoDB Local

Serverless DynamoDB local plugin is used to run DynamoDB in your local system for serverless flow.

Run the command below to install Serverless DynamoDB local plugin:

npm install --save serverless-dynamodb-local@0.2.10

Once installed, add this plugin entry in the plugins array of the project JSON file:

"plugins": ["serverless-dynamodb-local"]

It supports the following:

  • Install DynamoDB locally – sls dynamodb install
  • Starts DynamoDB locally with all the supported parameters (e.g., InMemory, sharedDb) – sls dynamodb start
  • Create, manage and execute migration scripts required to populate data in local DB – sls dynamodb create -n <filename> sls dynamodb execute -n <filename>

To start DynamoDB in local, run this command:

sls dynamodb start

It will launch a web portal at  http://localhost:8000/shell

You need to use the code below in your NodeJS function to call the local DynamoDB:

var AWS = require('aws-sdk');

new AWS.DynamoDB.DocumentClient({
    region: 'localhost',
    endpoint: 'http://localhost:8000'
})

Serverless DynamoDB Stream

The Serverless DynamoDB stream plugin is used to emulate the DynamoDB stream triggering the Lambda function offline.

Run the command below to install the Serverless DynamoDB stream plugin:

npm install --save serverless-plugin-offline-dynamodb-stream

Once installed, add the plugin in the serverless.yml file or project.json file:

plugins:
  - serverless-plugin-offline-dynamodb-stream

Do the following configuration in the serverless.yml file to create a stream locally:

custom:
  dynamodbStream:
    host: {LOCAL_DYNAMODB_HOST}
    port: {LOCAL_DYNAMODB_PORT}
    pollForever: boolean
    streams:
      - table: {TABLE_NAME}
      functions:
        - {FUNCTION_NAME}

The pollForever attribute can be set to true to continue to poll for dynamodbstreams events indefinitly. The default value for pollForever is false. If it’s set as false, it will stop polling for events once the end of the stream is reached.

Serverless Framework DynamoDB Autoscaling

The Serverless Framework DynamoDB autoscaling plugin automatically generates auto-scaling configurations for DynamoDB tables. It automatically discovers preconfigured tables in the database and adds dedicated scaling resources to them.

It reuses the existing project’s IAM role for handling scaling target resources. It’s only mandatory ScalableTarget and ScalingPolicy resources that are added to a CloudFormation stack

Run the command below to install the Serverless Framework DynamoDB autoscaling plugin:

npm install serverless-plugin-dynamodb-autoscaling

Once installed, add the plugin in in the serverless.yml file:

plugins:

- serverless-plugin-dynamodb-autoscaling

This plugin will automatically generate an auto-scaling configuration for all the preconfigured tables and global secondary indexes. However, you may exclude a specific table or tweak the configurations:

Here’s an example:

resources:
  Resources:
    Table1:
      Properties:
        TableName: foo
    Table2:
      Properties:
        TableName: bar
    Table3:
      Properties:
        TableName:
          Fn::Sub: ${AWS::Region}-dynamodbtest
custom:
  dynamodbAutoscaling:
    tablesConfig:
      # Disable auto scaling for table "Table1"
      Table1: false
      # Disable auto scaling just for indexes of "Table2" table
      Table2:
        indexes: false
      # Tweak minCapacity setting for all tables of which resource names start with ‘Table’
      Table*:
        minCapacity: 10

ScalingPolicy resources are deployed sequentially and chained via the DependsOn property.

custom:
  dynamodbAutoscaling:
    chainScalingPolicies: false

There are many other Serverless Framework DynamoDB plugins, such as serverless create global DynamoDB table, serverless plugin DynamoDB Backup, etc., available to enhance the experience of using DynamoDB with the Serverless Framework.

Summary

Serverless Framework is a very popular framework to build serverless applications on AWS, GCP, and other cloud platforms. Its huge stock of plugin libraries makes it very attractive compared to AWS SAM. Its integration with DynamoDB and offering several DynamoDB plugins made it easier for developers to move to the Serverless Framework.