OTEL Trace Ingestion

Publish OTEL Traces directly to your DBNL Deployment

Distributional is now Talaria Scientific. The DBNL product described in these docs has been sunset; this documentation is preserved for reference. Read the announcement.

OpenTelemetry (OTEL) Trace Ingestion allows for the richest data to be uploaded to your Project, but requires some off-platform coding and does not support backfilling data. This guide provides comprehensive instructions for instrumenting your AI agent application to send OpenTelemetry (OTEL) traces to DBNL.

Prerequisites

DBNL Credentials: You'll need:

• DBNL API URL (e.g., http://localhost:8080/api)

• API Token (Bearer token for authentication which can be generated at DBNL_API_URL/tokens)

• Project ID (your DBNL project identifier, typically starts with proj_ and is part of the URL for your project)

Implementation

Python

Quick Start (DBNL SDK)

The simplest way to get tracing working is with dbnl.init_tracing(), which handles provider setup, exporter configuration, and auto-instrumentation of supported libraries in a single call.

pip install 'dbnl[instrumentation]'

The instrumentation extra installs OpenInference instrumentors for popular libraries (OpenAI, LangChain, Anthropic, etc.) so that init_tracing() can auto-instrument them. If you only need the exporter without auto-instrumentation, pip install dbnl is sufficient.

import dbnl

dbnl.login()
dbnl.init_tracing(
    project_id="{PROJECT_ID}",
    service_name="my-agent",
)

init_tracing() registers a global TracerProvider with a BatchSpanProcessor pointing at your DBNL deployment, and automatically instruments any installed OpenInference-compatible libraries. Pass auto_instrument=False to disable this and instrument manually.

To tag traces with an application version or experiment variants, see App Versioning & Experiments.

Advanced: Raw OpenTelemetry Setup

If you need full control over the TracerProvider (for example to add additional exporters, custom resources, or a non-default span processor), you can manage your own provider while still using the SDK to handle exporter configuration.

Pass your own TracerProvider to init_tracing() and it will attach the dbnl span processor and run auto-instrumentation on it, without replacing the global provider:

import dbnl
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import Resource

dbnl.login()

resource = Resource.create({"service.name": "my-agent"})
tracer_provider = TracerProvider(resource=resource)

# Add your own processors/exporters here
# tracer_provider.add_span_processor(...)

dbnl.init_tracing(
    project_id="{PROJECT_ID}",
    tracer_provider=tracer_provider,
)

trace.set_tracer_provider(tracer_provider)

Pass auto_instrument=False if you want to skip auto-instrumentation and instrument libraries yourself.

For even more control, dbnl.get_dbnl_span_processor() returns a pre-configured BatchSpanProcessor you can add to any provider directly, but note that it does not run auto-instrumentation.

Fully Manual Setup (no SDK)

If you prefer not to depend on the dbnl package at runtime, you can wire the OTLP exporter directly:

pip install 'opentelemetry-sdk>=1.20.0' 'opentelemetry-exporter-otlp>=1.20.0'

For LangChain applications, also install OpenInference instrumentation:

pip install 'openinference-instrumentation-langchain>=0.1.0'

Create a telemetry initialization module (telemetry.py) in your application:

import os
import logging
from typing import Optional
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def create_dbnl_exporter() -> Optional[OTLPSpanExporter]:
    """Create OTLP exporter for DBNL"""
    # Get configuration from environment
    api_url = os.environ.get("DBNL_API_URL", "").strip()
    api_token = os.environ.get("DBNL_API_TOKEN", "").strip()
    project_id = os.environ.get("DBNL_PROJECT_ID", "").strip()

    # Validate configuration
    if not all([api_url, api_token, project_id]):
        logger.info("DBNL configuration incomplete. Set DBNL_API_URL, DBNL_API_TOKEN, and DBNL_PROJECT_ID.")
        return None

    # Create headers
    headers = {
        "Authorization": f"Bearer {api_token}",
        "x-dbnl-project-id": project_id,
        "Content-Type": "application/x-protobuf",
    }

    # Create exporter pointing at the DBNL OTLP endpoint
    endpoint = f"{api_url}/otel/v1/traces"
    exporter = OTLPSpanExporter(endpoint=endpoint, headers=headers)
    logger.info(f"✅ DBNL exporter configured: {endpoint}")
    return exporter

def initialize_telemetry():
    """Initialize OpenTelemetry with DBNL exporter"""
    # Create tracer provider with resource attributes
    resource = Resource.create({
        "service.name": os.environ.get("OTEL_SERVICE_NAME", "my-agent"),
    })

    tracer_provider = TracerProvider(resource=resource)
    trace.set_tracer_provider(tracer_provider)

    # Add DBNL exporter
    dbnl_exporter = create_dbnl_exporter()
    if dbnl_exporter:
        tracer_provider.add_span_processor(BatchSpanProcessor(dbnl_exporter))
        logger.info("📊 DBNL OTEL tracing enabled")

    return tracer_provider

# Initialize on import
tracer_provider = initialize_telemetry()
tracer = trace.get_tracer(__name__)

LangChain Integration

For LangChain applications, add OpenInference instrumentation:

from openinference.instrumentation.langchain import LangChainInstrumentor

def initialize_telemetry():
    """Initialize OpenTelemetry with DBNL exporter and LangChain instrumentation"""
    # ... (previous code) ...

    # Add LangChain instrumentation
    instrumentor = LangChainInstrumentor()
    instrumentor.instrument(tracer_provider=tracer_provider)

    return tracer_provider

Application Integration

Initialize telemetry early in your application startup:

FastAPI Example:

from fastapi import FastAPI
from telemetry import initialize_telemetry

app = FastAPI()

@app.on_event("startup")
async def startup_event():
    initialize_telemetry()

Standalone Script Example:

from telemetry import initialize_telemetry

if __name__ == "__main__":
    initialize_telemetry()

Message Content Capture

DBNL reads message content (prompts, completions, tool arguments) from span attributes only. There are two OTel instrumentor ecosystems for GenAI, and they handle content very differently:

• OpenInference instrumentors (openinference-instrumentation-*) always place message content on span attributes. This is a core design choice in the OpenInference spec, so DBNL ingests their content with no extra configuration.

• OTel GenAI semantic convention instrumentors (the gen_ai.* namespace) are split: the spec allows content on either span attributes or separate OTel log records, and individual instrumentors pick different defaults. DBNL ingests traces, not logs, so any instrumentor that emits content as log events will produce traces with empty input and output columns.

When an OpenInference instrumentor exists for your framework, prefer it. If you must use an OTel GenAI instrumentor, the table below covers the common ones and how to ensure content lands on span attributes.

OpenInference instrumentors (no configuration needed)

Framework	Instrumentor
LangChain	openinference-instrumentation-langchain
Google ADK	openinference-instrumentation-google-adk
Anthropic	openinference-instrumentation-anthropic
OpenAI (direct)	openinference-instrumentation-openai

See the OpenInference repo for the full list of supported frameworks.

OTel GenAI instrumentors (configuration required)

Framework	Instrumentor	Default	How to get span attributes
Pydantic AI	Native (InstrumentationSettings)	Span attrs (recent releases)	Use version=2 or higher; avoid version=1, which emits log events.
OpenAI Agents SDK	opentelemetry-instrumentation-openai-agents-v2	Off	Set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true.

If traces are appearing in DBNL but input / output are blank, the instrumentor is almost certainly emitting content as log events. Pick one of the configurations above.

Pydantic AI

from pydantic_ai.agent import Agent
from pydantic_ai.models.instrumented import InstrumentationSettings

agent = Agent(
    "openai:gpt-5-nano",
    instrument=InstrumentationSettings(version=2),
)

OpenAI Agents SDK

export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

from opentelemetry.instrumentation.openai_agents import OpenAIAgentsInstrumentor

OpenAIAgentsInstrumentor().instrument()

Node.js

Quick Start

Install the required OpenTelemetry packages:

npm install @opentelemetry/sdk-node @opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-proto tsx

Create an instrumentation.ts file that configures the OTLP exporter to send traces to DBNL:

import { NodeSDK } from "@opentelemetry/sdk-node";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";

const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(
      new OTLPTraceExporter({
        url: "{DBNL_API_URL}/otel/v1/traces",
        headers: {
          Authorization: "Bearer {API_TOKEN}",
          "x-dbnl-project-id": "{PROJECT_ID}",
        },
      })
    ),
  ],
});

sdk.start();

Use the --import flag to load instrumentation before your application code:

npx tsx --import ./instrumentation.ts app.ts

Auto-Instrumentation

For automatic tracing of LLM provider calls, add the relevant OpenInference JS instrumentation package and pass it to the NodeSDK. For example, to auto-instrument OpenAI calls:

npm install @arizeai/openinference-instrumentation-openai

import { NodeSDK } from "@opentelemetry/sdk-node";
import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai";

const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(
      new OTLPTraceExporter({
        url: "{DBNL_API_URL}/otel/v1/traces",
        headers: {
          Authorization: "Bearer {API_TOKEN}",
          "x-dbnl-project-id": "{PROJECT_ID}",
        },
      })
    ),
  ],
  instrumentations: [new OpenAIInstrumentation()],
});

sdk.start();

Instrumentation packages are available for OpenAI, Anthropic, LangChain, Bedrock, and others. See the full list in the OpenInference JS packages.

OTel Collector

Overview

If you already run an OpenTelemetry Collector, you can forward traces to DBNL by adding an OTLP/HTTP exporter to your collector configuration. This lets you centralize trace routing without changing application code.

Configuration

Add the following blocks to your collector YAML config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  otlphttp/dbnl:
    endpoint: "{DBNL_API_URL}/otel"
    headers:
      Authorization: "Bearer {API_TOKEN}"
      x-dbnl-project-id: "{PROJECT_ID}"

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlphttp/dbnl]

The exporter endpoint is {DBNL_API_URL}/otel (without /v1/traces) because the otlphttp exporter appends /v1/traces automatically.

If you already have a collector config, merge these blocks into your existing receivers, exporters, and service.pipelines sections rather than replacing them. You can add otlphttp/dbnl alongside your other exporters in the traces pipeline.

Semantic Conventions

The collector forwards traces as-is; it does not transform span attributes. Your application's instrumentation must emit traces using OpenInference Semantic Conventions so that DBNL can parse the required fields (input, output, etc.) from span attributes.

Required Trace Fields

The following fields are required regardless of which ingestion method you are using:

• input: The text input to the LLM as a string

• output: The text response from the LLM as a string

• timestamp: The UTC timecode associated with the LLM call as a timestamptz

You may choose to track other attributes such as total_token_count or feedback_score which are part of the DBNL semantic convention.

Custom Attributes

Custom metadata should be added as span attributes using the OpenInference semantic convention. These attributes are available within the spans data for analysis. Note that only columns defined in the DBNL Semantic Convention are supported as top-level columns; arbitrary custom columns are not ingested.

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("agent_execution") as span:
    # Set semantic attributes
    span.set_attribute("input.value", user_query)
    span.set_attribute("output.value", agent_response)
    
    # Add custom metadata
    span.set_attribute("session.id", session_id)
    span.set_attribute("conversation.id", conversation_id)
    span.set_attribute("tool.name", "search_symbol")
    span.set_attribute("tool.success", True)
    span.set_attribute("deployment.type", "web-application")

Advanced Configuration

Batch Processing

DBNL uses BatchSpanProcessor by default for efficient trace export. This batches spans before sending, reducing network overhead:

from opentelemetry.sdk.trace.export import BatchSpanProcessor

processor = BatchSpanProcessor(dbnl_exporter)
tracer_provider.add_span_processor(processor)

For immediate export (useful for debugging), use SimpleSpanProcessor:

from opentelemetry.sdk.trace.export import SimpleSpanProcessor

processor = SimpleSpanProcessor(dbnl_exporter)
tracer_provider.add_span_processor(processor)

Verification

Test Trace Export

Create a test span to verify traces are being sent:

from opentelemetry import trace
from telemetry import tracer_provider

tracer = trace.get_tracer(__name__)

# Create a test span
with tracer.start_as_current_span("test_dbnl_export") as span:
    span.set_attribute("input.value", "test input")
    span.set_attribute("output.value", "test output")
    span.set_attribute("test", True)

# Force flush to ensure export
tracer_provider.force_flush()
print("✅ Test span exported to DBNL")

View Traces in DBNL

After sending traces, verify they appear in your DBNL dashboard. By default, traces are processed into logs nightly so you will not see them right away.

1. Log into your DBNL deployment and go to your project

2. Check the Status page to confirm that they have been processed

3. Navigate to the Explorer or Logs section

4. Filter by your project ID or service name

5. Verify traces are appearing with the expected attributes

Troubleshooting

Traces Not Appearing in DBNL

1. Check Environment Variables: Verify all required variables are set:

   echo $DBNL_API_URL
   echo $DBNL_API_TOKEN
   echo $DBNL_PROJECT_ID

2. Verify API Endpoint: Test connectivity to DBNL:

   curl -H "Authorization: Bearer $DBNL_API_TOKEN" \
        -H "x-dbnl-project-id: $DBNL_PROJECT_ID" \
        $DBNL_API_URL/health

3. Check Logs: Look for DBNL exporter configuration messages:

   ✅ DBNL exporter configured: http://localhost:8080/otel/v1/traces
   📊 DBNL OTEL tracing enabled

4. Verify URL Formatting: Ensure the endpoint is correctly formatted:

   • Format: {DBNL_API_URL}/otel/v1/traces

   • Example: http://localhost:8080/otel/v1/traces

Common Issues

Issue: "DBNL configuration incomplete"

• Solution: Ensure DBNL_API_URL, DBNL_API_TOKEN, and DBNL_PROJECT_ID are all set

Issue: "Failed to configure DBNL exporter"

• Solution: Check that the API URL is valid and the token has proper permissions

Issue: Traces appear but missing attributes

• Solution: Ensure you're using OpenInference semantic conventions or manually setting required attributes (input, output, timestamp)

Issue: High latency or performance impact

• Solution: Use BatchSpanProcessor (default) instead of SimpleSpanProcessor for better performance

For issues or questions:

1. Check the troubleshooting section above

2. Review DBNL documentation

3. Verify your DBNL deployment has OTEL Trace Ingestion enabled

4. Contact DBNL support at [e-mail] with your project ID and API endpoint

Best Practices

1. Use Batch Processing: Always use BatchSpanProcessor in production for better performance

2. Use Semantic Conventions: Follow OpenInference conventions for automatic attribute mapping

3. Error Handling: Wrap exporter creation in try-except blocks to prevent application failures

4. Graceful Degradation: Allow your application to function even if DBNL configuration is incomplete

Additional Resources

• DBNL Semantic Convention - Learn about semantic conventions for better analytics

• OpenTelemetry Python Documentation - Official OpenTelemetry Python docs

• OpenTelemetry JavaScript Documentation - Official OpenTelemetry JS docs

• OpenTelemetry Collector Documentation - Official OpenTelemetry Collector docs

• OpenInference Documentation - OpenInference semantic conventions