Lambda Logging

Lambda Logging: 3 Ways to Access Logs, Logging in Python, and More

What Is AWS Lambda Logging? 

AWS Lambda is a cloud service offered by Amazon Web Services (AWS) that lets developers run their code in response to a trigger, such as an HTTP request or a message in a message queue. It is the primary component of the AWS serverless computing platform. The code is executed in a serverless environment, which means that the developer does not have to worry about the underlying infrastructure, such as servers or virtual machines.

AWS Lambda automatically logs the events and the results of function execution, including any errors or exceptions that may have occurred. These logs can be accessed through the AWS Management Console, the AWS CLI, or the AWS CloudWatch Logs API.

By default, AWS Lambda logs all function invocations and the results to Amazon CloudWatch Logs. CloudWatch Logs is a cloud service that collects, monitors, and stores logs from various sources, including AWS Lambda. It provides a way to view, search, and set alarms for the logs, as well as to export the logs to other destinations.

In addition to the logs generated by AWS Lambda, developers can also add their own logs to the function execution by using the logging libraries or functions provided by their programming language of choice. These custom log messages can also be accessed through CloudWatch Logs and can be used to debug and troubleshoot issues with the function.

This is part of a series of articles about serverless debugging. 

3 Ways to Access Amazon CloudWatch Logs for AWS Lambda

AWS Lambda automatically reports metrics for your Lambda functions, logging all requests handled by the function and storing the logs in Amazon CloudWatch. You can insert logging statements into function code, and these custom logs will also be stored in CloudWatch. All the logs are stored to a CloudWatch Logs group associated with the Lambda function, named /aws/lambda/<function name>.

You can access the logs for a Lambda function in the following ways:

Using Amazon CloudWatch

To access Lambda logs using CloudWatch, follow these steps:

1. Sign in to the AWS Management Console and navigate to the CloudWatch service.
2. In the left-hand menu, under Logs, select Log Groups.
3. Select the log group for your function (the log group will be named /aws/lambda/<function-name>).
4. Select the log stream for the specific execution of the function that you want to view.

The logs for the function execution will be displayed in the log stream. You can use the search and filter options to find specific log messages.

Using the AWS CLI

In order to use the AWS CLI, you will need to create an AWS Access Key and Key Secret in Amazon IAM User Management. If you do not properly configure access, you will not be able to run the code below.

To access Lambda logs using the AWS CLI, follow these steps:

1. Install and configure the AWS CLI on your local machine.
2. Run the following command to retrieve the logs for a specific function execution:
   Copy Code Copied Use a different Browser

   
   aws logs filter-log-events --log-group-name "/aws/lambda/<function-name>" --start-time <start-time> --end-time <end-time>
   

3. Replace <function-name> with the name of your function, and <start-time> and <end-time> with the time range for the logs you want to retrieve, in Unix timestamp format.

Using the AWS Lambda Console

To access Lambda logs using the AWS Lambda console, follow these steps:

1. Sign in to the AWS Management Console and navigate to the AWS Lambda service.
2. Select the function you want to view the logs for.
3. In the Monitoring tab, you will see a list of the recent invocations of the function.
4. Select an invocation to view the details of the execution, including the logs.
5. You can use the search and filter options to find specific log messages in the logs for the function execution.

Related content: Read our guide to Lambda tracing

Lambda Function Logging in Python 

When you invoke a Lambda function for the first time, a log group will automatically be created. There can be multiple versions of a Lambda function, allowing you to choose the version you want to run. A single log group stores all the logs for the invocations of a Lambda function. 

A logging library is useful for formatting and categorizing log messages. For instance, you might use the Python logging module for Lambda functions written in Python—this will make it easier to control your output format and log messages. Another tip is to log messages in the JSON format to make querying with CloudWatch Logs Insights easier. This also makes filtering and exporting easier.

Another good practice is using a variable to set and adjust the logging level and based on your requirements and environment. The Lambda function’s code can generate smaller or larger volumes of log data at different logging levels, impacting logging performance and costs. 

With Lambda, you can specify environment variables for the function’s runtime environment without updating the code. For instance, you can start by creating a LAMBDA_LOG_LEVEL environment variable that defines the logging level for your code. Here is an example of how to import and use the LAMBDA_LOG_LEVEL variable and to define a “logger”: 

import os
import logging

# Get the value of the LAMBDA_LOG_LEVEL environment variable
log_level = os.environ.get('LAMBDA_LOG_LEVEL', 'INFO')

# Configure the logger
logger = logging.getLogger()
logger.setLevel(log_level)

# Your code goes here
def my_function(event, context):
    logger.debug('This is a debug message')
    logger.info('This is an info message')
    logger.warning('This is a warning message')
    logger.error('This is an error message')

In this example, the value of the LAMBDA_LOG_LEVEL environment variable is used to set the log level for the logger. If the environment variable is not set, the default log level of INFO is used. The log level determines the type of messages that will be logged. 

Sending Logs to Other Destinations from CloudWatch

To send Amazon Lambda logs to other destinations from CloudWatch, you can use the CloudWatch Logs Subscription Filter feature. This allows you to set up a real-time feed of your logs to other services, such as Amazon Elasticsearch Service, Amazon Kinesis Data Streams, or a third-party logging service, using a custom AWS Lambda function as the delivery stream.

To set up a CloudWatch Logs Subscription Filter:

1. Create a new Lambda function or use an existing one as the delivery stream.
2. In the CloudWatch console, go to the Logs section and select the log group that you want to send to the delivery stream.
3. Click the Create export task button and select the Export to a new Lambda function option.
4. Select the Lambda function that you created as the delivery stream and specify any other required options.
5. Click the Create export task button to create the subscription filter.

Once the subscription filter is set up, CloudWatch will automatically send new log events to the delivery stream in real-time, where they can be processed and forwarded to the destination service.

You can also use the CloudWatch Logs API or the CloudWatch Logs Agent to set up a subscription filter programmatically.

AWS Lambda Logging Best Practices 

There are several best practices for logging in AWS Lambda functions:

• Include identifying information in log events: It can be helpful to include information in your log events that can help you identify the source of the log message, such as the function name, the request ID, or the user ID. This can make it easier to track down issues and understand the context of the log message.
• Use consistent formatting for log events: Using a consistent format for your log events can make it easier to search and analyze your logs. You can use a logging library or framework that provides formatting capabilities, such as the built-in logging module in Python or the Loguru library.
• Use CloudWatch Logs Insights: CloudWatch Logs Insights is a powerful tool for searching, filtering, and analyzing your logs. You can use it to identify trends, find errors, and debug issues in your functions.
• Monitor Lambda function memory use: Lambda functions are allocated a certain amount of memory, and the amount of memory available can affect the performance and cost of your functions. It is a good idea to monitor your functions’ memory usage and optimize it as needed. 

By following these best practices, you can improve the quality and usefulness of your AWS Lambda logs and better understand the behavior and performance of your serverless applications.

Log-free troubleshooting with Lumigo

Lumigo is a distributed tracing platform purpose-built for troubleshooting microservices in production. Developers building serverless apps with  AWS Lambda and other serverless services use Lumigo to monitor, trace and troubleshoot their serverless applications. Deployed with no changes and automated in one-click, Lumigo stitches together every interaction between micro and managed services into end-to-end stack traces, giving complete visibility into serverless environments. Using Lumigo to monitor and troubleshoot their applications, developers get:

• End-to-end virtual stack traces across every micro and managed service that makes up a serverless application, in context
• API visibility that makes all the data passed between services available and accessible, making it possible to perform root cause analysis without digging through logs 
• Distributed tracing that is deployed with no code and automated in one click 
• Unified platform to explore and query across microservices, see a real-time view of applications, and optimize performance

Learn more about Lumigo