Monitoring and Debugging Python Apps on AWS Lambda

As a developer, Python for me is a heavy-lifting and versatile language. I’ve used it for building APIs, internet of things projects, file and data conversions, machine learning and (of course) web development. 

Like with any modern, commonly used language, the functionality behind the application is only as good as the infrastructure that it is deployed onto. Combined with the flexibility and scalability of serverless, and you’ve got a winning combination that can scale apps in a single bound. 

This rapid scale however, can also scale issues and bugs within your code. This is why observability is so important to deployed applications. Here’s how to get started turning your “oh no’s” into whoa’s by learning some ways to monitor and debug Python applications on AWS Lambda.

Getting set up to charm Python Lambda Functions

Before we get started we are going to need a few things. Foremost is a Lumigo account connected to AWS. It’s quick and easy to set up and get connected. For more on how to do that check out our getting started with Lumigo guide. 

The other important element in this setup is Python. For this, we are going to set up a rudimentary API gateway with a Python Lambda function querying a random record from DynamoDB.

The code we are starting with is as follows:

import json

import random

import boto3
dynamodb = boto3.resource(‘dynamodb’)table = dynamodb.Table(‘lmb1’)
def lambda_handler(event, context):
  data = table.scan(AttributesToGet=[‘productName’])
  res = random.choice(data[‘Items’])
  response = {
      ‘statusCode’: 200,
      # ‘body’:json.dumps(res), 
      ‘body’: json.dumps(res),
      ‘headers’: {
        ‘Content-Type’: ‘application/json’,
        ‘Access-Control-Allow-Origin’: ‘*’
      },

 }
  return response

The DynamoDB that is queried by the Lambda script contains an index along with a text value stored under `productName`. The code is pulling back a list of items, then choosing one at random.

Running an API test appears to bring back the error- free result I’m looking for. Sifting through logs is one of my least favorite things, but everything seems to be ok.

Checking it on Lumigo, I can see auto tracing for the Lambda Function has already been picked up and the transaction is being traced.

Additionally API Gateway has also been identified along with the API route.

When building out any application, it’s always important to have a strong starting point. Integrating tracing in from the start will help us level up and add in some additional functionality to our Python Lambda API. 

Extending the Lambda Python Function

The above example utilized a simple GET API to return a random value from a DynamoDB. Let’s now extend that to support a POST request so that we can also add some additional data in. While this could easily be set as a separate Lambda and API route, for this blog I’m just going to include it into the existing API Route by reading the httpMethod. 

First, let’s add the code to input into DynamoDB. Having used the boto3 python library this becomes as easy as `table.put_item`. In the updated code below, I’ve commented out some parts for testing the database insert.

import json
import random
import boto3
 
dynamodb = boto3.resource(‘dynamodb’)
table = dynamodb.Table(‘lmb1’)
 
def lambda_handler(event, context):
  
  # data = table.scan(AttributesToGet=[‘productName’])
 
  # res = random.choice(data[‘Items’])
 
  inp = table.put_item(Item= {‘productName’:  ‘test’})
  print(inp);
  
  response = {
      ‘statusCode’: 200,
      ‘body’: “complete”,
      ‘headers’: {
        ‘Content-Type’: ‘application/json’,
        ‘Access-Control-Allow-Origin’: ‘*’
      },
  }
  
  return response

Running the test in API Gateway seems to return the result I’m looking for, at least it throws no errors. However, I am yet again faced with the unfavorable task of sifting through log output. 

Using Lumigo and looking at the traced invocation for the function, things become a lot clearer and I can see that database `table.put_item` easier.

Now for the final part, and this is where end-to-end traceability is really going to come in handy. As part of our route we need to pick up the `httpMethod` value from API Gateway to put some logic around the database insert. 

Essentially, if the httpMethod coming in is a POST request then a database insert needs to happen, else we are going to use a GET type response and randomly choose an option from DynamoDB. 

Let’s first test the httpMethod to make sure we are able to see that API Gateway value being passed through to our Python Lambda. According to the API Gateway documentation it should be coming in on the `lambda_handler` event variable.

The updated code now looks like this:

import json
import random
import boto3
 
dynamodb = boto3.resource(‘dynamodb’)
table = dynamodb.Table(‘lmb1’)
 
def lambda_handler(event, context):
 
  print(“This is a ” + event[‘httpMethod’] + ” request” )
  
  response = {
      ‘statusCode’: 200,
      ‘body’: “complete”,
      ‘headers’: {
        ‘Content-Type’: ‘application/json’,
        ‘Access-Control-Allow-Origin’: ‘*’
      },
  }
  
  return response

After deploying the updated Lambda and running a test via API Gateway method test, I am now greeted by an error and associated logs. 

Switching tabs to the connected Lumigo account, I’m able to hone in on the failed API really easily from the transaction tab. Clicking to get further detail, I can then clearly see the error as part of traced logging.

From this we can see that the `httpMethod` variable is throwing an error. From further `print()` exploration on the `event` variable coming in from the lambda_handler, I can see that API Gateway is not sending it through to the Lambda function. 

This led me to realize that the ‘Use Lambda Proxy Integration’ on the API Gateway was not ticked. Easily missed and overlooked, especially when building things out amidst the pressures of delivery schedules and the many moving parts of a complex modern application.

Lumigo to the rescue!Running the method API test again with the Lambda proxy set returns the httpMethod request variable.

See how Lumigo can help with your Python Lambda Functions

If you are looking to go to the next level with your Python functions also check out Lumigo execution tags. These can be extremely handy in not only gathering a lot more information from your Python Lambda but doing so at specific breakpoints with additional trace data. 

Sign up for a Lumigo account and get started today.