OpenTelemetry and Python: Benefits and a Quick Tutorial

What Is OpenTelemetry? 

OpenTelemetry is an open-source, vendor-neutral project that provides a collection of tools, APIs, libraries, and instrumentation to generate, collect, and process telemetry data from distributed systems. It is designed to facilitate observability for modern applications by capturing metrics, logs, and traces, which are crucial for understanding system performance and behavior. 

However, it is important to note that OpenTelemetry is not an observability backend. Instead, it serves as a unified framework for collecting and exporting telemetry data to various observability platforms and backends, such as Prometheus, Jaeger, and others, where the actual data analysis, visualization, and alerting take place.

5 Benefits of Using OpenTelemetry with Python 

Using Python with OpenTelemetry offers several benefits that stem from the language’s characteristics and its compatibility with the OpenTelemetry framework:

1. Ease of use: Python is known for its readability and simplicity, which makes it easier for developers to instrument their applications using the OpenTelemetry Python SDK. The intuitive syntax and clear structure of Python code allow for quick and efficient implementation of instrumentation and integration with monitoring tools.
2. Extensive library support: Python has a vast ecosystem of libraries and packages, which simplifies the process of integrating OpenTelemetry with various components of your application. The OpenTelemetry Python SDK provides auto-instrumentation packages for popular Python frameworks and libraries, such as Flask, Django, Redis, and SQLAlchemy, enabling seamless monitoring and observability.
3. Flexibility: Python’s dynamic nature and extensibility make it easy to customize and extend OpenTelemetry to meet specific requirements. You can create custom spans, metrics, and logs, as well as implement custom exporters and processors to handle your telemetry data in a way that suits your needs.
4. Integration with data analysis tools: Python is a popular language for data analysis and visualization, with numerous libraries like NumPy, pandas, and matplotlib available. By using Python with OpenTelemetry, you can more easily analyze and visualize the collected telemetry data, enabling you to gain insights into your application’s performance and behavior.
5. Scalability: Python’s support for concurrency and parallelism, using libraries like asyncio or multiprocessing, makes it a suitable choice for building scalable applications. OpenTelemetry’s Python SDK takes advantage of these capabilities to efficiently collect and process telemetry data from your application without significantly impacting its performance.

Related content: Read our guide to OpenTelemetry architecture

Tutorial: Getting Started with OpenTelemetry in Python 

This tutorial shows how to set up and use telemetry data from an HTTP server using Flask. It is based on the OpenTelemetry documentation.

Step 1: Installation

Start by setting up a new environment in a directory using venv:

mkdir otel-example
cd otel-example
python3 -m venv .
source ./bin/activate

Next,  install OpenTelemetry and Flask:

pip install opentelemetry-distro
pip install flask

The opentelemetry-distro installation package includes the SDK, API, and useful tools like the opentelemetry-instrument and opentelemetry-bootstrap</code >

Step 2: Creating an HTTP Server

Use the following to create an app.py file:

from random import randint
from flask import Flask, request

app = Flask(__name__)

@app.route("/rolldice")
def roll_dice():
    return str(do_roll())

def do_roll():
    return randint(1, 6)

This script should launch a new HTTP server with a “/rolldice” route. You will need to issue the following command: 

flask run

This will start a Web Server under localhost, as shown in the following screenshot:

Step 3: Enabling Auto-Instrumentation

Automatic instrumentation enables Open Telemetry to generate data for you. In this example, the opentelemetry-instrument agent is used. You can add Flask auto-instrumentation by running the following command:

opentelemetry-bootstrap -a install

Running your instrumented application

The instrumented app is now ready to run using opentelemetry-instrument, where it should be displayed on the console: 

opentelemetry-instrument \
    --traces_exporter console \
    --metrics_exporter console \
    flask run --host=0.0.0.0

Note: Adding the argument –host=0.0.0.0 allows Flask to use the server’s IP address and makes it possible to access /rolldice URL from the internet.

For any request you send to the server, you should receive a trace with the result in a single span displayed on the console. The generated span will track the request’s lifetime.

You can send several more requests to the endpoint and wait for the metrics to be displayed. Alternatively, terminate the application and the metrics should be printed to the console.

Step 4: Adding Manual Instrumentation 

While auto-instrumentation can capture telemetry at the system edge (i.e., outbound and inbound HTTP requests), it cannot capture the internal workings of the application. Therefore, you will also need to provide manual instrumentation, which you can link with the auto-instrumentation. 

Start by importing the following:

from random import randint
from flask import Flask, request
from opentelemetry import trace

You will then be able to modify the app.py code to initialize a tracer and use it to create a child trace of an auto-generated trace:

tracer = trace.get_tracer(__name__)

app = Flask(__name__)

@app.route("/rolldice")
def roll_dice():
    return str(do_roll())

def do_roll():
        res = randint(1, 6)
	  Rollspan = trace.get_current_span()
        rollspan.set_attribute("roll.value", res)
        return res

You can then rerun the application

opentelemetry-instrument \
    --traces_exporter console \
    --metrics_exporter console \
    flask run --host=0.0.0.0

When a request is sent to the server, there should be two spans in the trace published to the console. The do_rollspan should register the parent as an auto-generated trace. You can then modify the app.py code to initialize a meter for creating a counter instrument. This object counts the number of rolls for every roll value:

from opentelemetry import trace
from opentelemetry import metrics

from random import randint
from flask import Flask, request

tracer = trace.get_tracer(__name__)
meter = metrics.get_meter(__name__)

roll_counter = meter.create_counter(
    "roll_counter",
    description="The roll value",
)

app = Flask(__name__)

@app.route("/rolldice")
def roll_dice():
    return str(do_roll())

def do_roll():
    with tracer.start_as_current_span("do_roll") as rollspan:
        res = randint(1, 6)
        rollspan.set_attribute("roll.value", res)
        roll_counter.add(1, {"roll.value": res})
        return res

Rerun the application:

opentelemetry-instrument \
    --traces_exporter console \
    --metrics_exporter console \
    flask run

When a request is sent to the server, the roll counter metric should be visible on the console, and each roll value should have a separate count. 

Step 5: Sending Telemetry to the Collector

The OpenTelemetry Collector is an important component in a typical production deployment. It can be useful for the following use cases: 

• Several services sharing one telemetry sink, reducing the overhead involved in switching exporters.
• Traces that are aggregated from multiple services and run on multiple hosts.
• A centralized location processes and exports traces to the backend.

To send telemetry to the OTel Collector, save the collector configuration code to a dedicated file in the /tmp/ directory:

# /tmp/otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
exporters:
  logging:
    loglevel: debug
processors:
  batch:
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [logging]
      processors: [batch]
    metrics:
      receivers: [otlp]
      exporters: [logging]
      processors: [batch]

Next, use the following command to get and run the Collector based on the specified configuration:

docker run -p 4317:4317 \
    -v /tmp/otel-collector-config.yaml:/etc/otel-collector-config.yaml \
    otel/opentelemetry-collector:latest \
    --config=/etc/otel-collector-config.yaml

\

There should now be an instance of the Collector running locally and listening on port 4317.

You can now modify the Docker command to send metrics and spans to the Collector using the OTLP exporter instead of the console. You will need to install the exporter package:

pip install opentelemetry-exporter-otlp

The opentelemetry-instrument agent should be able to detect the installed package and export via OTLP by default when it is run.

When you next run the application, you don’t need to export the telemetry to the console. You can now view the /rolldice route, and the output should be displayed in the Collector process, not the Flask process.

Microservices Monitoring with Lumigo

Lumigo is cloud native observability tool that provides automated distributed tracing of microservice applications and supports OpenTelemetry for reporting of tracing data and resources. With Lumigo, users can:

• See the end-to-end path of a transaction and full system map of applications
• Monitor and debug third party APIs and managed services (ex. Amazon DynamoDB, Twilio, Stripe)
• Go from alert to root cause analysis in one click
• Understand system behavior and explore performance and cost issues 
• Group services into business contexts

Get started with a free trial of Lumigo for your microservice applications