Serverless Web Apps with AWS Lambda and Kotlin

This article takes a look at Kotlin, a programming language that runs on the JVM, and explores how it can be used to build serverless web apps with AWS Lambda and API Gateway.

In programming language terms Kotlin is a relative newcomer. It was first announced in 2011 by JetBrains, the makers of IntelliJ IDEA, and was designed as a modern successor to Java. It is a statically typed OO language with functional features and it fixes many of Java’s best-known problems – excessive boilerplate, lack of expressiveness and seemingly unnecessary repetition. It has proved particularly popular in the Android community and has been Google’s recommended language for Android development since May 2019.

AWS Lambda and the JVM

I work for the fintech startup OpenGamma. Over the last two years we have built a serverless SaaS platform on AWS, mostly using Java. This makes us unusual among Lambda users. According to a 2018 survey by the Serverless framework, only about 6% of Lambda functions use the Java runtime.

There are good reasons for this. Java is notoriously memory-hungry, and it is easy to hit the 3GB RAM limit in a lambda function, particularly when doing financial calculations that require large amounts of data. The JVM is relatively slow to start, so lambda functions written in Java suffer from worse cold-start problems than functions using the other runtimes.

Another drawback of using Java on Lambda is the lack of libraries and tooling for building serverless web apps. The other runtimes, and JavaScript in particular, have plenty of options for building web applications using Lambda and API Gateway. Java has far fewer, and in many cases where it is supported it is a second-class citizen compared to JavaScript.

Despite these disadvantages Java was the obvious choice for us. Java is one of the most popular languages used in banks and financial institutions. This means it has a wide range of relevant libraries and a large pool of developers with finance knowledge.

When we started developing the platform in 2017 we evaluated the libraries and frameworks available and found there was nothing that met our requirements. So we created our own. It is heavily inspired by JAX-RS, using annotations to link REST endpoints to the methods that handle requests.

This library solved our immediate problem, but is limited in scope and doesn’t handle setting up the resources in AWS or redeploying the code when the application changes. These problems were tackled separately using Terraform to define the infrastructure and custom code for deployment.

My experience with this library made me think about what I would want from an open source library to solve the problem. And the more I thought about it, the more I imagined how I would write that library myself. And in the end the temptation was too much and I wrote it.

Osiris

The library is called Osiris, it is written in Kotlin and it runs on the Java 8 runtime in AWS Lambda. My aim when I started it was to make building serverless, cloud-native web apps as simple as writing something that runs in a traditional server.

It has been in development since May 2017 and is used in production by tech recruitment sites like CodeScreen. It is open source and is released under the Apache 2.0 license.

Why Kotlin?

There were a number of reasons I chose Kotlin for Osiris.

The JVM

As I outlined above, Java is the natural choice for someone working in the financial sector. And Kotlin’s excellent interoperability with the JVM and Java libraries make it a low-risk choice.

Modern Language Features

Kotlin was specifically designed to fix the problems with Java and it features many of the advances in language design that have become mainstream in the 25 years since the creation of Java

• Reduction in boilerplate – type inference, properties, data classes, top-level functions
• A more advanced type system – null-safe types, no primitives, easier generics
• Greater expressiveness – extension functions, destructuring, better sequences

Tool Support

Kotlin is unusual in that its IDE integration is developed alongside the language. Immature tooling is often a problem when adopting a new language but the support for Kotlin in IntelliJ is on par with Java.

Support for writing DSLs

Kotlin’s syntax and features make it easy to write DSLs. Using a DSL for defining the REST endpoints provides a familiar programming model for the user and also makes life easier for the library developer (i.e. me). Building a DSL is simpler than building a library based on annotations or using an external format such as YAML.

Overview

The tools that Osiris provides can be divided into 3 main categories – generating a new project, defining the application logic and deploying to AWS.

Generating a Project

Osiris can generate a skeleton project using either Maven or Gradle. The generated project includes some example REST endpoints and can be deployed straight to AWS.

Defining the Application

The application logic is defined in Kotlin using a DSL. The design of the DSL is deliberately familiar and is inspired by the many REST libraries that exist for other languages, e.g. SparkJava, Sinatra.

Deploying to AWS

Osiris includes plugins for Maven and Gradle that generate the AWS configuration files and deploy the application to AWS.

Hello, World!

The simplest application you can build with Osiris looks something like this:

val api = api {

    get("/helloworld") { req ->
        "hello, world!"
    }
}

This defines a REST API with a single endpoint at the path /helloworld. It returns a response with status 200 and a body containing the string “hello, world!” Learn more about AWS Lambda in our guide: What is Lambda Edge?

Project Walkthrough

The easiest way to start a project with Osiris is to generate one. Osiris includes a Maven archetype and a Gradle plugin that generates a “hello, world!” project that can be deployed straight to AWS.

This walkthrough uses Maven. For Gradle instructions see here.

Requirements

• Maven
• An AWS account with admin privileges
• AWS credentials (see here)

Generate the Project

Open a terminal and enter the following command:

mvn archetype:generate -DarchetypeGroupId=ws.osiris -DarchetypeArtifactId=osiris-archetype -DarchetypeVersion=1.0.0

You will be prompted four times:

1. Enter the group ID (namespace) for the project, e.g. com.example
2. Enter the artifact ID (name) for the project, e.g. example-app
3. Press enter to accept the default suggestion for the package name
4. Type “y” and press enter to generate the project

Maven will generate a project in a directory with the same name as the artifact ID you entered above.

The generated project contains two modules: core (the code that’s deployed to AWS) and local-server (the code for running the application locally during development). The most interesting file is ApiDefinition.kt. This contains the definition of the REST endpoints.

Deploy to AWS

The generated project contains some example endpoints (that return “hello, world!” of course) and it can be deployed straight to AWS. Open a terminal, change to the project directory and enter the following command:

mvn deploy

This builds the project and deploys it to AWS. The output in the terminal should end something like this:

[INFO] example-app ........................................ SUCCESS [  1.042 s]
[INFO] example-app-core ................................... SUCCESS [01:03 min]
[INFO] example-app-local-server ........................... SUCCESS [  1.860 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:06 min
[INFO] Finished at: 2018-03-24T11:22:49Z
[INFO] Final Memory: 46M/439M
[INFO] ------------------------------------------------------------------------

The project is now deployed to AWS and available via a public URL. Enter the following command to open a browser pointing at the application:

mvn osiris:open -f core -Dstage=dev -Dendpoint=/helloworld

You should see a JSON object { “message”: “hello, world!” }

Any time you make changes to the project you can run mvn deploy to build it and deploy to AWS.

Run in an IDE

Osiris includes an HTTP server so an application can be run locally during development and debugging. The generated project contains a file Main.kt with a main function that starts the server. When run with no arguments this serves the application on port 8080. Learn more about AWS Lambda in our guide: Complete Guide to AWS Lambda Cost

Other Features

Integration with CloudFormation

The Osiris plugins generate the configuration needed to create the REST API and lambda function in AWS. The configuration files are YAML files in CloudFormation format.

CloudFormation is the native AWS format used to declaratively define infrastructure. If an application built with Osiris needs any other AWS infrastructure (e.g. a database, message queues etc) then it can be defined in hand-written CloudFormation files and included in the project. Osiris will deploy them along with the generated files as part of the same CloudFormation stack.

There is an example project here demonstrating how to do this.

CI / CD Integration

Using the Maven or Gradle plugin to deploy directly to AWS from the development machine is fine for prototypes, personal projects and dev environments, but (hopefully) not for production environment or in organizations with a more formal release process.

In order to integrate with a CI / CD pipeline the Osiris build can be set up to generate the project artifacts and CloudFormation files but not to do the deployment. The artifacts and files it produces are completely standard and can be deployed by any CD tool that integrates with AWS and CloudFormation.

Keep-Alive

Lambda functions using the Java runtime are notorious for suffering from so-called “cold start” problems. These happen when a request arrives and there are no function instances available to handle it. The request is blocked while AWS creates a new instance. This requires the creation of a new JVM which can take a few seconds and can be noticeable to a user in a web app.

The standard way to work around this problem is by creating a mechanism that keeps a number of function instances available at all times. This takes advantage of an implementation detail of AWS Lambda – when a function has handled a request it is not immediately destroyed. Function instances can be kept alive for some time after handling a request, sometimes for up to an hour. The trick is to periodically send a request to the functions that does but prevents them from being cleaned up.

These are generally known as “keep-alive” mechanisms, and Osiris has one built in. The application configuration allows the user to specify the minimum number of function instances that should be available and Osiris schedules events needed to keep them alive.

Example Projects

The Osiris Example GitHub repo (https://github.com/cjkent/osiris-examples) contains projects showing how to handle some common scenarios with Osiris. These include:

• Authentication
• Using DynamoDB
• Defining other AWS infrastructure in the project
• Handling binary data
• Defining the API in Kotlin and writing the application logic in Java
• Integrating with the Lumigo monitoring platform

These are working applications and can mostly be cloned and deployed straight to AWS.

Conclusion

Despite the drawbacks of the Java runtime, Kotlin has proved to be an excellent choice for building serverless apps with Lambda and API Gateway. The language is a big step forward compared to Java and its expressiveness makes it particularly suitable for building and using libraries like Osiris. And its excellent interoperability with Java libraries and the JVM allow you to take advantage of the huge Java ecosystem.

The Osiris source is available on GitHub (https://github.com/cjkent/osiris/) and full documentation is available on the GitHub wiki (https://github.com/cjkent/osiris/wiki).

Learn how easy AWS Lambda monitoring can be with Lumigo