In today’s world, with Large tech giants and businesses looking forward to moving toward serverless architecture, there has been a significant demand for scaling the applications. It’s therefore no surprise that millions of companies worldwide have adopted, or are planning on migrating to a Kubernetes and AWS Lambda solution to take their serverless applications to the next level. 

In this article, We will explore different possibilities and best practices for getting started with Kubernetes Lambda.

What is Kubernetes?

Kubernetes is an impressive tool that helps you manage your containerized workloads that helps to automate the deployment, scaling, and management processes, making it a great choice for businesses looking to streamline their operations. Kubernetes was created by Google and donated to the Cloud Native Computing Foundation (CNCF), currently maintained by CNCF as well as open source contributors. 

Let’s think of Kubernetes as the ultimate tool for developers. Imagine having multiple servers (like a diverse set of toys) but limited infrastructure. Kubernetes acts as a sophisticated manager, efficiently allocating server resources, tracking tasks, and ensuring optimal performance. It’s like having an expert organizer for your tech stack, making everything run seamlessly. 

What is AWS Lambda?

AWS provides a serverless computing service named AWS Lambda, AWS Lambda allows you to run the code and install packages without provisioning any servers and you don’t have to set up any high-level systems. AWS Lambda automatically scales your workload depending on incoming requests and the best part is that you only have to pay for the compute time consumed for using the service. AWS Lambda can be easily integrated with other AWS services like AWS Cloudformation, API gateway, DynamoDB, and S3.

Let’s explore, How we can leverage the power of Kubernetes and AWS Lambda working together! Get ready to dive into this amazing journey with us.

There are several benefits of using Kubernetes Lambda together, let us discuss them below:

Scalability:

One of the main advantages of using Kubernetes with Lambda is scalability because Kubernetes comes with automatic horizontal scaling that helps your applications to scale in any direction depending on the incoming traffic. Hence, Next time you see any sudden spike in your serverless application Kubernetes will be there for you to ensure that you pay only for what you use.

Continuous Development and Integration:

Another benefit of using Kubernetes with Lambda is Continuous Integration(CI) and Continuous development(CD) which enable a great environment for testing, staging, and production environments. Kubernetes and AWS lambda make your development faster by automating the deployments process allowing developers to push code faster and review changes frequently, resulting in faster delivery of new features and improvements. Furthermore, This integration also helps in consistent testing and validation as well as regular scanning for better security and minimizing threats.

High Availability

Imagine running your application on multiple instances/computer machines simultaneously, there is a high chance that something could go wrong or the system gets shut down due to the limited number of servers available. Fortunately, Kubernetes provides automatic failover and load balancing for your instances, Hence if any one instance goes down your application will still be always available.

Integrated Developer Environment

Using an Integrated Developer Environment becomes crucial when you’re dealing with serverless applications, Kubernetes provides a bundle of tools and plugins that help the developer raise their bar during the process. Furthermore, by integrating a joint developer atmosphere for Kubernetes with AWS Lambda, you can improve resource management, promote seamless reusability of code, accelerate troubleshooting and testing procedures, and ultimately deliver reliable, high-performance applications.

Okay! We understood that using Kubernetes Lambda together has a lot of benefits. But, How can we make the most out of them? We have to focus on best practices in leveraging the use of Kubernetes with AWS Lambda to make powerful serverless applications.

Below are some best practices for Using Kubernetes With Lambda:

Plan your Architecture

If we were to create a serverless application using Kubernetes, We will plan everything beforehand as it’s essential. This includes deciding on if we need a containerization platform or not, a networking model, and we need to plan deployment strategies thoroughly. We will consider every possible factor such as scalability, high availability, and security when designing the architecture.

Manage Resources Efficiently

Managing resources efficiently comes with the tendency to optimize the applications, Kubernetes provides automatic horizontal scaling. We need to check if we want to consider using a Horizontal Pod Autoscaler (HPA) or Vertical Pod Autoscaler (VPA). This helps when any resource goes down, Kubernetes automatically adjusts the services and you need to pay as per what you used. Hence, plan billings properly to track that you’re not spending unnecessary amounts on resources.

Secure your applications

While building applications using Kubernetes, Implement authentication and authorization mechanisms like OAuth, OpenID Connect, and role-based access control (RBAC) to protect your applications from unauthorized attacks and threats. Ensure that all communication between containers and services is encrypted using tools like SSL/TLS certificates and mutual TLS authentication.

Implement CI/CD Pipelines

Automate your development lifecycle with tools like Jenkins, CircleCI, Gitlab, or CodePipeline for implementing Continuous Integration and delivery pipeline. Your developers can work on machines or version control systems like Git and push changes into a pipeline for automatic build, test, and deploy code updates.

Optimize Costs:

Kubernetes is marked as a costlier to use if not managed properly by implementing necessary precautions beforehand. To reduce costs, consider using spot instances, pay-per-use pricing models, reserved instances, and billing mechanisms. Monitor and observe resources time-to-time to identify areas where more optimization is needed. Furthermore, Use specific instance types depending on your workloads rather than using generic ones that help to reduce cost on unnecessary expenses.

We learned about some benefits and best practices for using Kubernetes with AWS Lambda. Now, Let’s deep dive into a simple example of how you can setup the power of both to create your serverless applications.

Prerequisites

Ensure that you have these tools already installed on your system for smooth usage, I’ll attach their links so you can download them:

• kubectl
• eksctl
• AWS Command Line Interface (AWS CLI)
• Helm

Note: You should have an AWS account and make sure you keep authentication ready configured to your local system. Use was authorization for using eksctl from the command line.

Setting up the Amazon EKS cluster

1. Save the following YAML as cluster.yml

apiVersion: eksctl.io/v1alpha5

kind: ClusterConfig

metadata:

  name: lumigo-cluster

  region: ap-south-1

nodeGroups:

   - name: ng1

     desiredCapacity: 2

     instanceType: t2.micro

     ssh:

        publicKeyName: rgkey1

   - name: ng-mixed

     minSize: 1

     maxSize: 3

     instancesDistribution:

       maxPrice: 0.010

       instanceTypes: ["t2.micro"] 

       onDemandBaseCapacity: 0

       onDemandPercentageAboveBaseCapacity: 50

       spotInstancePools: 2     

     ssh:

         publicKeyName: rgkey1 `//create a public key from EC2 Dashboard //`

Then run the following: 

eksctl create cluster -f cluster.yml

2. Wait for 10-15 mins, You will find your cluster ready. Check whether your pods are running or not.

kubectl get nodes

3. Get your EKS cluster name by running this command and exporting it.

eksctl get cluster
export EKS_CLUSTER_NAME=<output of previous command>

We’re going to use the AWS ACK controller for Lambda for integration with Kubernetes

Setting up the ACK Controller for Lambda

1. Install an ACK Controller using Helm by following these instructions

Change ‘export SERVICE=s3’ to ‘export SERVICE=lambda’
Change ‘export AWS_REGION=us-west-2’ to set your region

2. Configure IAM permissions for accessing lambda through pods, follow these instructions.

Replace ‘SERVICE=”s3”’ with ‘SERVICE=”lambda”’

3. Check whether ACK Lambda is running or not:

kubectl get pods -n ack-system

Provisioning a Lambda function from the Kubernetes cluster

We will perform the simple task to deploy the sample “Hello Kubernetes Lambda” Lambda function:

Step 1: Create a zip file for any language code and upload the zip file to the S3 bucket. For this demo we are going to use python for the demo app.

Step 2: Deploy the zip file to a Lambda function using the ACK controller from the previous EKS controller that we created.

1. Run the following commands to create the sample “Hello Kubernetes Lambda” Lambda function code, and then zip it up:

mkdir hello-k8slambda-function
cd hello-k8slambda-function
cat << EOF > hello-k8slambda-function.py
import json
def lambda_handler(event, context):
    # TODO implement
    return {
        ‘statusCode’: 200,
        ‘body’: json.dumps(‘Hello Kubernetes Lambda !!’)
    }
EOF
zip lumigo-package.zip hello-k8slambda-function.py

2. Create a sample S3 bucket, see the AWS s3 docs here to find out more.
3. Run the below commands to upload the zip file into the S3 bucket, using your bucket name from the previous step:

export BUCKET_NAME=<S3 bucket from step 2>
aws s3 cp lumigo-package.zip s3://${BUCKET_NAME}

4. You should expect output like this:

upload: ./lumigo-package.zip to s3://<BUCKET_NAME>/lumigo-package.zip

We’re creating an ACK controller now for the lambda function:

1. Name your function:
   

export FUNCTION_NAME=hello-k8slambda-s3-ack

2. Create and deploy the Kubernetes manifest file with basic execution role with limited access:

export AWS_ACCOUNT_ID=$(aws sts get-caller-identity –query “Account” –output text)export LAMBDA_ROLE=”arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda_basic_execution”
cat << EOF > lambdamanifest.yaml
apiVersion: lambda.services.k8s.aws/v1alpha1
kind: Function
metadata:
 name: $FUNCTION_NAME
 annotations:
   services.k8s.aws/region: $AWS_REGION
spec:
 name: $FUNCTION_NAME
 code:
   s3Bucket: $BUCKET_NAME
   s3Key: lumigo-package.zip
 role: $LAMBDA_ROLE
 runtime: python3.9
 handler: lambda_function.lambda_handler
 description: function created by ACK lambda-controller e2e tests
EOF
kubectl create -f lambdamanifest.yaml

This should output something like:  

function.lambda.services.k8s.aws/< FUNCTION_NAME> created

3. Fetching the function details using the kubectl command:

kubectl describe function/$FUNCTION_NAME

4. To get our expected output from the lambda function, We’re using invoke now:
   

aws lambda invoke –function-name $FUNCTION_NAME  response.json

Then run

cat response.json

5. You should expect this output from your above command
   

{“statusCode”: 200, “body”: “\\”Hello Kubernetes Lambda !!”\\”}

Congratulations!🔥 You successfully created your own Lambda function from your Kubernetes cluster.

Scaling Versatility with Kubernetes and AWS Lambda

The fusion of Kubernetes and AWS Lambda offers unparalleled scalability for modern serverless applications. As enterprises pivot to serverless architectures, this dynamic partnership addresses the growing need for efficient scalability. By combining Kubernetes’ automatic scaling with AWS Lambda’s event-driven approach, businesses can craft agile applications that dynamically adapt to demand. This convergence streamlines resource management, development, and security, opening doors to innovative and resilient cloud solutions.

Sign up for Lumigo today to not only gain valuable insight into your cloud applications but also gain unparalleled debugging and troubleshooting capabilities. Leap into a world where scalability meets efficiency, and harness the synergy of Kubernetes and Lambda with the power of speedy issue resolution and tracing.

Cleaning Up the Demo Deployment

Don’t forget to delete your cluster and setup you did today, This will help you to save on unnecessary bills at the end of the month. Deleting the cluster:

eksctl delete  cluster -f cluster.yml

Thanks for Reading, Happy learning ✨