Advanced Serverless CICD - Part 2: CircleCI

In the second part of his series comparing CICD services for serverless applications, Erez Rokah, developer of the open source aws-testing-library, focuses on the popular commercial tool, CircleCI.

---

CircleCI has several main advantages:

• It’s free for private repositories
• It’s a mature, well-documented tool with plenty of examples to follow
• Integrates very well with other development tools (e.g. Slack)
• Relevant notifications for build jobs are available out of the box
• It has a REST API to manage build jobs

Visit part 1 to read about the advantages and disadvantage of using AWS CodeBuild

---

Introduction

In this post I’ll show you how to configure an advanced CICD process using CircleCI.

If you don’t have a CircleCI account you should sign up here.

Our goal is to have a CICD setup that will:

• Have different deployment stages
• Be able to deploy all/specific services with their dependencies
• Include automated tests for deploying with confidence
• Allow adding new services with ease
• Be reproducible so we can set it up again automatically

In part 1 we used CodeBuild to set up our CICD.

There are a few notable differences between CircleCI and CodeBuild that will affect our setup:

• We can’t use CloudFormation to describe our deployment setup
• There are no regions in CircleCI so we can’t have a CICD setup per region per stage as in part 1
• We will need to give CircleCI access to our AWS account
• We don’t need to setup email notifications or GitHub access tokens as CircleCI handles it for us

---

Let’s Get Started

As in part 1 we’ll be using the following repository as a baseline for setting up the deployment process, so you should fork and clone it.

In order to create a reproducible CICD setup, I wrote a small CLI that utilizes the CircleCI API to follow a GitHub project, connect it to AWS, set up all the required environment variables and relevant project settings.

In order to use the CLI you’ll need to create a Personal API Token.

Since I’ve already described our tool stack and prerequisites I won’t repeat them here, but refer back to my previous post for all the details

---

CICD Setup

Run the following command from the cicd directory to set up the process (replace with the correct values):

yarn setup:circleci --token *********** --stagingAdminEmail 
admin@email.com --prodAdminEmail admin@email.com

The setup command will:

1. Follow a GitHub project (it takes the project information based on git metadata)
2. Create an IAM user and API key with admin access and set the relevant environment variables to access your AWS account
3. Set the relevant environment variables required for our Serverless application
4. Update CircleCI to only build on pull requests, not to build on forks and not to pass secrets to forks (I found theses settings better than the defaults)

Here is a snippet from the setup command:

You can see the full command in cicd/package.json:

Note the remove:circleci and trigger:circleci commands to unfollow a project and to trigger a manual build.

 

After running the setup command Circle you can visit you CircleCI dashboard to verify the project has been added.

---

Build Setup

CircleCI config file is under .circleci/config.yml .

The file is very similar to our buildspec.yml from part 1 with the notable difference of the need to handle different deploy stages in the file:

Note that I’m using yaml aliases to avoid code repetition

The build filters are defined as follows:

The test job will run on pull requests, a staging deploy will run on merge to master and a prod deploy will run on tag push (as in part 1).

Build phases, environment variables, build scripts and end to end tests are similar to the ones described in part 1.

The deployment magic happens in the cicd/scripts/deploy.js script. You can find the code here (I strongly recommend going over it).

---

Summary

Using CircleCI we’ve set up an advanced CICD process for our Serverless application, in a very similar way as we did using CodeBuild in part 1.

While losing the option to use CloudFormation to describe our CICD process in a reproducible way, the CircleCI API is a decent replacement, and by using the AWS SDK we can easily set the proper permissions for our CICD jobs.

The usefulness of a CICD process comes from the ability to monitor errors (e.g. relevant notifications), integrate with other development tools (e.g. Slack), the ability to make changes and resolve issues quickly.

These are all missing from CodeBuild (or require development time and effort) and are included in CircleCI.

If you have questions about replicating this setup, or simply want to talk serverless CICD, get in touch on Twitter.