Send OpenTelemetry data from a Python app to Axiom

This guide explains how to send OpenTelemetry data from a Python app to Axiom using the Python OpenTelemetry SDK.

Prerequisites

• Install Python version 3.7 or higher.
• Create an Axiom account. To sign up for a free account, go to the Axiom app.
• Create a dataset in Axiom. This is where the Python app sends telemetry data. For more information, see Data Settings.
• Create an API key in Axiom with permissions to query and ingest data. For more information, see Access Settings.

Install required dependencies

To install the required Python dependencies, run the following code in your terminal:

pip install opentelemetry-api opentelemetry-sdk opentelemetry-instrumentation-flask opentelemetry-exporter-otlp Flask

Install dependencies with requirements file

Alternatively, if you use a requirements.txt file in your Python project, add these lines:

opentelemetry-api
opentelemetry-sdk
opentelemetry-instrumentation-flask
opentelemetry-exporter-otlp
Flask

Then run the following code in your terminal to install dependencies:

pip install -r requirements.txt

Create an app.py file

Create an app.py file with the following content. This file creates a basic HTTP server using Flask.

from flask import Flask
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from random import randint
import exporter  # Import the OpenTelemetry configuration and the exporter that sends trace data to Axiom.

# Initialize a new Flask app.
app = Flask(__name__)

# Instrument the Flask app with OpenTelemetry.
FlaskInstrumentor().instrument_app(app)

# Retrieve a tracer object from the exporter module for tracing operations.
tracer = exporter.service1_tracer

@app.route("/rolldice")
def roll_dice():
    # Define a route in the Flask app. When this route is accessed, it triggers the roll_dice function.
    with tracer.start_as_current_span("roll_dice_span"):
        # Create a new tracing span named `roll_dice_span`.
        return str(roll())  # Simulate a dice roll and return the result as a string.

def roll():
    # Define a function to simulate rolling a dice, returning a random integer between 1 and 6.
    return randint(1, 6)

# Main app runner. Only executed when the script is the main program.
if __name__ == "__main__":
    # Run the Flask app on port 8080 with debug mode enabled for additional debugging information.
    app.run(port=8080, debug=True)

Create an exporter.py file

Create an exporter.py file with the following content. This file establishes an OpenTelemetry configuration and sets up an exporter that sends trace data to Axiom.

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource, SERVICE_NAME
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

# Define the service name resource for the tracer.
resource = Resource(attributes={
    SERVICE_NAME: "NAME_OF_SERVICE" # Replace `NAME_OF_SERVICE` with the name of the service you want to trace.
})

# Create a TracerProvider with the defined resource for creating tracers.
provider = TracerProvider(resource=resource)

# Configure the OTLP/HTTP Span Exporter with Axiom headers and endpoint. Replace `API_TOKEN` with your Axiom API key, and replace `DATASET_NAME` with the name of the Axiom dataset where you want to send data.
otlp_exporter = OTLPSpanExporter(
    endpoint="https://api.axiom.co/v1/traces",
    headers={
        "Authorization": "Bearer API_TOKEN",
        "X-Axiom-Dataset": "DATASET_NAME"
    }
)

# Create a BatchSpanProcessor with the OTLP exporter to batch and send trace spans.
processor = BatchSpanProcessor(otlp_exporter)
provider.add_span_processor(processor)

# Set the TracerProvider as the global tracer provider.
trace.set_tracer_provider(provider)

# Define a tracer for external use in different parts of the app.
service1_tracer = trace.get_tracer("service1")

In the exporter.py file, make the following changes:

• Replace NAME_OF_SERVICE with the name of the service you want to trace. This is important for identifying and categorizing trace data, particularly in systems with multiple services.
• Replace API_TOKEN with your Axiom API key.
• Replace DATASET_NAME with the name of the Axiom dataset where you want to send data.

For more information on the libraries imported by the exporter.py file, see the Reference below.

Run the app

Run the following code in your terminal to run the Python project:

macOS/Linux

python3 app.py

Windows

py -3 app.py

In your browser, go to http://127.0.0.1:8080/rolldice to interact with your Python app. Each time you load the page, the app displays a random number and sends the collected traces to Axiom.

Observe the telemetry data in Axiom

In Axiom, go the Stream tab and click your dataset. This page displays the traces sent to Axiom and enables you to monitor and analyze your app's performance and behavior.

Dynamic OpenTelemetry traces dashboard

In Axiom, go the Dashboards tab and click OpenTelemetry Traces (python). This pre-built traces dashboard provides further insights into the performance and behavior of your app.

Send data from an existing Python project

Manual instrumentation

Manual instrumentation in Python with OpenTelemetry involves adding code to create and manage spans around the blocks of code you want to trace. This approach allows for precise control over the trace data.

1. Import and configure a tracer at the start of your main Python file. For example, use the tracer from the exporter.py configuration.

   import exporter
   tracer = exporter.service1_tracer

2. Enclose the code blocks in your app that you want to trace within spans. Start and end these spans in your code.

   with tracer.start_as_current_span("operation_name"):

3. Add relevant metadata and logs to your spans to enrich the trace data, providing more context for your data.

   with tracer.start_as_current_span("operation_name") as span:
       span.set_attribute("key", "value")

Automatic instrumentation

Automatic instrumentation in Python with OpenTelemetry simplifies the process of adding telemetry data to your app. It uses pre-built libraries that automatically instrument the frameworks and libraries.

1. Install the OpenTelemetry packages designed for specific frameworks like Flask or Django.

   pip install opentelemetry-instrumentation-flask

2. Configure your app to use these libraries that automatically generate spans for standard operations.

   from opentelemetry.instrumentation.flask import FlaskInstrumentor
   # This assumes `app` is your Flask app.
   FlaskInstrumentor().instrument_app(app)

After you set them up, these libraries automatically trace relevant operations without additional code changes in your app.

Reference

List of OpenTelemetry trace fields

List of imported libraries

The exporter.py file imports the following libraries:

from opentelemetry import trace

This module creates and manages trace data in your app. It creates spans and tracers which track the execution flow and performance of your app.

from opentelemetry.sdk.trace import TracerProvider

TracerProvider acts as a container for the configuration of your app's tracing behavior. It allows you to define how spans are generated and processed, essentially serving as the central point for managing trace creation and propagation in your app.

from opentelemetry.sdk.trace.export import BatchSpanProcessor

BatchSpanProcessor is responsible for batching spans before they are exported. This is an important aspect of efficient trace data management as it aggregates multiple spans into fewer network requests, reducing the overhead on your app's performance and the tracing backend.

from opentelemetry.sdk.resources import Resource, SERVICE_NAME

The Resource class is used to describe your app's service attributes, such as its name, version, and environment. This contextual information is attached to the traces and helps in identifying and categorizing trace data, making it easier to filter and analyze in your monitoring setup.

from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

The OTLPSpanExporter is responsible for sending your app's trace data to a backend that supports the OTLP such as Axiom. It formats the trace data according to the OTLP standards and transmits it over HTTP, ensuring compatibility and standardization in how telemetry data is sent across different systems and services.