Push data manually or as part of a daily orchestration job using our Python SDK. This ingestion method allows for the most flexibility, but requires the most off-platform coding.

The following fields are required regardless of which ingestion method you are using:

• input: The text input to the LLM as a string.

• output: The text response from the LLM as a string.

• timestamp: The UTC timecode associated with the LLM call. Must be a timezone-aware datetime in UTC (Python: datetime with tzinfo=UTC or pandas: datetime64[us, UTC]).

Check out the for an example of using the SDK Log Ingestion as a Data Connection.

The Logs page allows the user to inspect specific logs with certain properties defined by

• A specific time window (default: last 7 full days of data)

• Specific filters on Columns or Metrics (default: no filters)

Individual Logs can be viewed in a variety of ways:

• : All and of the log viewed together and optionally expanded.

• (if spans provided): The waterfall trace view of latency and timing for each individual span.

• (if session_id provided): All associated logs for the given session, along with .

As part of inspecting the logs the user can

• View the filtered logs as charts and tables in the

• Save the specific filters as a to publish it on the

• Filter logs by to compare different configurations

If your data includes the experiment_variants column (part of the ), you can filter logs by experiment name and variant. The experiment_variants column is a map in the form { [experiment_name]: experiment_variant }, for example {"model": "gpt-4o"} or {"model": "gpt-4o-mini", "prompt_version": "v2"}.

Anywhere a Filter Builder is available (including , , and creation), click "Add experiment filter" to add an experiment filter row. Each row allows you to specify:

• Experiment name: The name of the experiment (e.g., model)

• Operator: One of is, is not, contains, or does not contain

You can add multiple experiment filters. Like other filters, all rows are ANDed together.

All and of the log viewed together and optionally expanded.

The waterfall trace view of latency and timing for each individual span. Only available if spans was provided as part of the .

All associated logs for the given session, along with . Only available if session_id was provided as part of the .

Our Privacy Policy can be found here.

Organizations

Each DBNL deployment corresponds to a single Organization containing:

• All Namespaces

• All Users

Namespaces

A Namespace is a unit of isolation within an Organization containing:

• Projects

Namespaces can be created by Organization Admins from the Admin Dashboard.

Users are individuals with a login to an Organization and are defined by Roles related to the Organization and one or more Namespaces.

Users can be created from the Organization or Namespace Admin Dashboard.

There are currently three Roles that can be assigned to a User:

• Organization Admin: This User has read and write permissions for all Organization level resources and are the only Users that can create Namespaces. Only other Organization Admins can create or remove Organization Admins. By default, the first user in an Organization is assigned the Organization Admin Role.

• Namespace Admin: This User has read and write permissions for all Namespace level resources. They can create new Namespace Writer users and invite them to their Namespace. By default, when an Organization Admin creates a Namespace they become a Namespace Admin of that Namespace.

• Namespace Writer: This User can create, read, and write to Projects in their Namespace. Namespace Writers can be created by Organization Admins or Namespace Admins.

Roles can be modified from the Organization or Namespace Admin Dashboard.

The DBNL Query Language is a SQL-like language that allows for querying data in Runs for the purpose of drawing visualizations, defining metrics or evaluating tests.

An expression is a combination of literals, values, operators, and functions. Expressions can evaluate to scalar or columnar values depending on their types and inputs. There are three types of expressions that can be composed into arbitrarily complex expressions.

Literal expressions are constant-valued expressions.

The dbnl CLI is installed as part of the SDK and allows for interacting with the dbnl platform from the command line.

To install the SDK, run:

The dbnl CLI.

• Options

  • --version - Show the version and exit.

This section contains examples demonstrating how to use DBNL in various scenarios using simulated data in real world scenarios.

Outing Agent Prompt Optimization Walkthrough

The Python SDK can be used for programmatically creating projects and uploading data to them.

See SDK Log Ingestion for more information and examples on using the SDK to upload log data to your deployment.

Installation

To install the latest SDK, run:

pip install --upgrade dbnl

SDK Functions

• convert_otlp_traces_data()

• create_filter()

Ingress

Requirements

The DBNL platform needs to be hosted on a domain or subdomain (e.g. dbnl-example.com or dbnl.example.com). It cannot be hosted on a subpath.

HTTPS/SSL

It is recommended that the DBNL platform be served over HTTPS. Support for SSL termination at the load balancer is included.

Egress

Requirements

Currently, the dbnl platform cannot run in an air-gapped environment and requires a few URLs to be accessible via egress.

Artifacts Registry

Required to fetch the DBNL platform artifacts such as the Helm chart and Docker images for installation and upgrades.

• https://ghcr.io/dbnlai/

An Internal Object Store

Required for services to access an object store, this data does not leave your environment.

• https://{BUCKET}.s3.amazonaws.com/​ (if using S3)

• https://storage.googleapis.com/{BUCKET} (if using GCS)

• https://{STORAGE_ACCOUNT}.blob.core.windows.net (if using Azure)

OIDC

Required to validate OIDC tokens, if using a 3rd party OIDC provider.

• https://login.microsoftonline.com/{APP_ID}/v2.0/ (if using Microsoft EntraID)

• https://{ACCOUNT}.okta.com/ (if using Okta)

Our End User License Agreements (EULA) for deployments can be found here.

Our End User Service Agreement for SaaS can be found here.

Data Connections are how production AI log data is ingested into your DBNL Deployment as part of the Data Pipeline. Each Project has one ingestion method that is set at creation. If you need to change this later you can do this via the Project settings page.

DBNL supports two methods of data ingestion:

• OTEL Trace Ingestion: Publish OTEL traces directly to DBNL as the product runs.

• SDK Log Ingestion: Push data manually or as part of a daily orchestration job using the Python SDK.

From the landing page click on "Data Connections" on the left panel. On the Data Connections landing page "+ Add Data Connection" in the upper right. Provide a required name for the Data Connection and an optional description. All Data Connections will be available to any creating a in the .

Personal Access Tokens are used for API authentication and are required for use of the .

To create a Personal Access Token click on your profile badge in the lower left of the UI, then click on "Personal Access Token." We recommend saving this as an environment variable like DBNL_API_TOKEN for future use.

The DBNL platform uses or OIDC for user authentication. OIDC providers that are known to work with DBNL include:

An Insight is a detected behavioral signal generated from unsupervised analysis of enriched logs as part of the .

Insights represent signals that the user can triage and refine through the or inspection of and track as or . They represent clusters of log data defined by filters on Columns corresponding to unique patterns of behavior. Insights can point to errors, issues, or changes within your agentic application that can be used to inform where and how to improve or fix your agent as part of an . Insights can reveal new metrics to eval or incorporate into reward functions, they can also pinpoint specific segments of data that can be used for focued post-training optimization, whether that be fine tuning, reinforcement learning, , , or any other method.

Typical insight volume: Most projects generate 5-20 new insights per week. Projects with stable, consistent behavior may generate fewer insights, while projects with volatile or rapidly changing behavior may generate more.

Insight Structure

The Adaptive Analytics Workflow is the core mechanism for discovering, investigating, and tracking hidden behavioral signals from your production AI log data.

1. Discover: New signals are displayed as Insights and Dashboards within a Project.

2. Investigate: Signals can be triaged and refined through population and temporal comparisons with the Explorer or dive directly into the evidence with the corresponding subset of enriched Logs.

3. Track: Codify signals that are meaningful through specific filtered and new custom .

4. Repeat: Future analysis and Insights are impacted by all tracked signals.

• Ready to start using DBNL? Head straight to our to get set up on the platform and start testing your AI products right away for free.

• Want to understand more about the platform? Check out the , options, and other aspects of the .

The Explorer enables rapid analysis and triage of Segments by performing graphical and statistical comparison between different subsets of Logs over different time windows and/or filters.

Accessing Explorer

You can access the Explorer in three ways:

1. From the main navigation: Click "Explorer" in the left sidebar

2. From an Insight: Click the "View in Explorer" button on any Insight card

3. From the Logs page: Apply filters to your logs, then click "View in Explorer" in the top-right corner

When accessed from an Insight or Logs page, the Explorer will pre-populate with your current filters.

There are three main types of exploration afforded by the Explorer:

• : Quickly see all for a given time window and single filter on the . This allows for an aggregate view of all Metrics.

• : Compare two different filters on the or a filter and its compliment across the same time window. This allows for comparison of between filters or between a filters and the rest of the Log data.

• : Compare a single filter across two adjacent time windows. This allows for a comparison of before/after for a given .

View all for a given filter on the in a time window. The filter can be optionally saved as to be published to future as part of the .

Compare two different filters on the or a filter and its compliment across the same time window. This allows for comparison of between filters or between a filters and the rest of the Log data. Either of these filters can be optionally saved as to be published to future as part of the .

Compare a single filter across two adjacent time windows. This allows for a comparison of before/after for a given .

DBNL is openly distributed and free to deploy within your cloud environment or on-premise, keeping your data safe, secure, and always under your control.

There are three options to deploy the DBNL platform as a self-hosted deployment:

• Sandbox: The DBNL sandbox deployment bundles all of the DBNL services and dependencies into a single self-contained Docker container for quick proof of concepts.

• Helm Chart: The full DBNL platform can be deployed using a Helm chart to existing infrastructure provisioned by the customer.

• : The full DBNL platform can be deployed using a Terraform module on infrastructure provisioned by the module alongside the platform. This option is supported on AWS, GCP, and Azure.

What is DBNL?

DBNL is an Adaptive Analytics platform designed to discover and track hidden behavioral signals in production AI logs and traces so that product owners can confidently know exactly where and how to improve their AI products over time. The platform gives a detailed snapshot of agent behavior - the interplay and correlations between users, context, tools, models, and metrics. Patterns in behavioral signals are automatically surfaced as Insights that can be investigated and tracked. This empowers AI teams to accelerate the AI data flywheel by pinpointing the signals and specific examples they can use to improve their products with confidence.

Why DBNL?

The AI data flywheel promises better agentic performance over time through post-training optimization on real production data, but not all data is created equal. DBNL helps AI product owners fill the critical gap between high level monitoring tools (focused on aggregate performance through evals, logging, and tracing) and low level debugging tools (focused on single-trace observability) to pinpoint hidden behavioral signals and relevant example data for post-training optimization. This allows AI product owners to better understand agent and user behavior to know exactly where and how to improve AI products in production.

Who is DBNL for?

Distributional is built for AI product teams looking to understand and improve their AI agents that have

• Scale: More than 1,000 traces or logs per day (too many to manually inspect)

• Data: Access to full spans from OTEL trace data or similarly rich data for analysis (See our )

• Value: Quantifiable business metrics to track and improve

• Understanding: You already monitor aggregate performance (but need richer analysis to know where and how to improve and fix your AI agents)

DBNL is openly distributed and free to within your cloud or on-premises environment, keeping your data safe, secure, and always under your control. Head over to our to get started right away.

DBNL integrates with your existing AI tools to easily and securely perform analytics for any AI product. The ingests, enriches, and analyzes production AI logs and traces, surfacing behavioral signals. These signals are published to and as , allowing users to discover, investigate, and track them as part of the . This gives you concrete signals and relevant data to power improvements to your agent as part of an AI data flywheel.

• Ready to start using DBNL? Head straight to our to get set up on the platform and start testing your AI products right away for free.

• Want to learn more about the workflow? Check out the .

• Want to understand more about the platform? Check out the , options, and other aspects of the .

This section contains examples demonstrating how to get data into the DBNL platform using various methods all adhering to the DBNL Semantic Convention. Each example includes working code, detailed explanations, and guidance on when to use each approach.

Getting Started

If you're new to DBNL, start with the Quickstart which walks you through deploying a local sandbox and uploading your first data.

Data Input Examples

DBNL supports multiple ways to ingest data.

• Direct OTEL Ingestion: Stream traces in real-time from OTEL-instrumented applications

• SDK from JSON: Load trace data from JSONL files and upload via the Python SDK

• : Batch upload OpenTelemetry trace exports

• : Import traces exported from Langfuse

All example code is available in the GitHub repository.

Check out the and Walkthroughs to see DBNL in action for various end-to-end use cases.

The DBNL platform architecture consists of a set of Services packaged as Docker images and a set of standard Infrastructure components that are deployed into your infrastructure (e.g. a VPC in AWS or GCP, or on-premise). The platform is scalable, modular, and self contained. It does not require an external connection to hosted Distributional services to operate.

Infrastructure

The DBNL platform requires the following infrastructure:

• A Kubernetes cluster to host the DBNL platform services.

• A PostgreSQL database to store metadata.

• An object store bucket to store raw data (e.g. S3 or GCS).

• A Redis database to serve as a messaging queue.

• A load balancer to route traffic to the API or UI service.

Costs vary by cloud provider and region. Approximate ranges (as of 2025):

• Minimum Setup: $300-500/month (suitable for POC/testing)

• Recommended Production: $800-1500/month (handles typical production workloads)

• High Volume: $2000-5000+/month (depends on log volume and retention requirements)

The DBNL platform consists of three core services that run within the Kubernetes cluster:

• The API service (api-srv) serves the DBNL API and orchestrates work across the dbnl platform.

• The worker service (worker-srv) processes async jobs scheduled by the API service.

• The UI service (ui-srv) serves the DBNL UI assets.

The DBNL platform combines configurable infrastructure, secure data handling, and workspace administration so teams can deploy adaptive analytics in their own environments.

What’s Inside

• Deployment – Options for running DBNL from quick sandboxes to fully managed clusters.

• Architecture – Service layout, data flow, and operational considerations.

• Networking – Connectivity requirements for the platform and its integrations.

• – How DBNL stores, protects, and governs customer data.

• – User and API access, including personal access tokens.

• – Organizing projects, namespaces, and permissions.

Use these guides together to plan, install, and operate DBNL in your environment.

This section contains examples demonstrating how to use DBNL in various scenarios. Each example includes , detailed explanations, and guidance on when to use each approach.

The provides a comprehensive walkthrough of building an end-to-end analytics pipeline:

• Generate OTEL traces from a Google ADK calculator agent

• Convert and augment trace data with computed metrics

General

Company

Metrics

Deployment

Dashboards are collections of histograms, time series and statistics of monitored Columns, tracked Segments, and generated Metrics for user-driven analysis.

There are three default dashboards for each Project:

• Monitoring Dashboard: Distributional recommended graphs and statistics built from required Columns and data from the DBNL Semantic Convention

• Segments Dashboard: Count graphs and statistics for all tracked Segments

• : Histograms, time series and statistics of generated

When investigating an issue, start with the time series to identify when it started, then use the histogram to understand what values are problematic, and finally check the logs page to see which specific logs exhibit the behavior.

Sankey charts show how agentic tool calls chain together in a trace:

• Nodes: Boxes representing specific tool calls (ie llm:gpt-4o-mini, tool:web_search, etc)

• Flows: Bands connecting nodes - width represents volume/quantity

• Direction: Left-to-right shows progression through tool calls

Example: A Sankey chart with many repeated nodes represents tool calls failing or needing to be retried many times, which may indicate an underlying bug in the agent or context.

Histograms show how frequently different values occur:

• X-axis: The metric value (e.g., token count, score from 1-5)

• Y-axis: Number of logs with that value

• Shape insights:

  • Normal (bell curve): Most values cluster around the average - typical, healthy distribution

Example: A token count histogram with two peaks (at 100 and 500 tokens) suggests two distinct conversation types.

Time series show how values change over time:

• X-axis: Date

• Y-axis: Metric value

• Lines: Typically shows average (mean) and P95 (95th percentile)

• Patterns to watch for:

Example: User frustration P95 suddenly spiking while mean stays flat suggests a subset of users are becoming frustrated.

Statistics give you quick numerical insights:

• Compare Max vs P95: If very different, you have extreme outliers worth investigating

• Compare Mean vs Median: If very different, your data is skewed (not normally distributed)

• Track P95 over P99: P95 is more stable and actionable for most use cases

• Use Min/Max: Identify best and worst case examples to investigate

The Status page shows you all ongoing and previous DBNL Data Pipeline runs for your project.

These runs represent the entire Data Pipeline, including:

• Data ingestion from the specified Data Connection for the Project

• Log enrichment by appending Metrics using the Model Connection

• Analysis and publishing of

You can view the current status of each run grouped by data date range, which time window DBNL was ingesting data for. If a Data Pipeline run has errored you can hover over the error status to view the exception and restart the run by clicking on the restart button in the actions column.

Typical pipeline run times depend on log volume and Model Connection latency:

Pipeline stages and their typical durations:

1. Ingest (10-30 seconds): Upload and validate data

2. Enrich (60-80% of total time): Compute metrics using Model Connection

3. Analyze (10-20% of total time): Run unsupervised learning algorithms

4. Publish (30-60 seconds): Update dashboards and generate insights

Projects are the main organizational tool in DBNL. Generally, you'll create one Project for every AI application that you'd like to analyze with DBNL. After a Project is created, you can start analyzing signals from your Production AI application using the .

A Project is initially defined by

• A : This is how the production AI log data is ingested into DBNL, one of or .

• A default : This is how DBNL creates LLM-as-judge metrics by default for the project. This is also how DBNL generates some of the insights as part of the unsupervised analytics in the Analyze step.

Data does not leave your deployment. A DBNL is self contained and does not "call home" or send your data back to a hosted cloud service keeping your data safe, secure, and always under your control.

Data is split between Databases (e.g. postgres, redis, clickhouse) and an Object Store (e.g. S3, GCS).

• Databases contain:

  • Metadata (e.g. name, schema)

Why does DBNL require a Model Connection?

Model Connections are how DBNL interfaces with LLMs, which is required for each step of the DBNL Data Pipeline to function. It enables DBNL to

• Compute LLM-as-judge Metrics as part of the enrich step.

• Perform certain unsupervised analytics processes as part of the analysis step.

• Translate surfaced behavioral signals into human readable Insights as part of the publish step.

Fundamentally a Model Connection needs to be able to expose a LLM chat completion interface that is accessible by your DBNL deployment. It can be

• An externally managed service (e.g. together.ai, OpenAI, etc)

• A cloud managed service that is part of your VPC (e.g. Bedrock, Vertex, etc)

• A locally managed deployment (e.g. a cluster of NVIDIA NIMs running in your DBNL k8s cluster as part of your deployment)

There are pros and cons to each of these approaches:

The following models are known to work well for LLM-as-judge, analysis, and Insight generation.

We recommend using a similar "mid-size" model that trades off speed, cost, and quality well.

Model Connections are defined at the level of an and can be used by any within the Namespace. For convenience, a new Model Connection can be created as part of the flow as well.

A Model Connection has the following attributes:

• Name (required): How the Model Connection is referenced when setting a default Model Connection for a project or LLM-as-judge .

• Description (optional): Human readable description of the connection for reference.

• Model (required): The model name to be used as part of the API call (e.g. gpt-3.5-turbo, gemini-2.0-flash-001, etc). See the documentation for your model provider for more details.

Different providers require different configuration parameters:

A Model Connection can be edited or deleted by clicking on the "Model Connections" tab on the sidebar of the landing page.

A Model Connection can be tested by navigating to the specific Model Connection as above and clicking on the "Validate" button. This will send a simple request to the endpoint and inform you if it was able to complete the request.

• Ready to send data to your project? Start ingesting data into your project using your defined to kick off the .

• Want to understand more about the platform? Check out the , options, and other aspects of the .

The Helm chart option separates the infrastructure and permission provisioning process from the DBNL platform deployment process, allowing you to manage the infrastructure, permissions and Helm chart using your existing processes.

To get the Helm chart, see ghcr.io/dbnlai/charts/dbnl.

Jump straight to:

• Installation

• Upgrading

The following prerequisite steps are required before starting the Helm chart installation.

To successfully deploy the DBNL Helm chart, you will need the following infrastructure:

• A Kubernetes cluster (e.g. , , ).

  • An or controller (e.g. , , )

• A PostgreSQL database (e.g. , , ).

To configure the DBNL Helm chart, you will need:

• A hostname to host the DBNL platform (e.g. dbnl.example.com).

• A set of DBNL registry credentials to pull the DBNL artifacts (e.g. Docker images, Helm chart).

• An RSA key pair to sign the .

An RSA key pair can be generated with:

To install the DBNL Helm chart, you will need:

• Install and set the Kubernetes cluster context.

• Install .

For the services deployed by the Helm chart to work as expected, they will need the following permissions and network accesses:

• api-srv

  • Network access to the database.

  • Network access to the Redis database.

  • Permission to read, write and generate pre-signed URLs on the object store bucket.

The Helm chart can be installed directly using or using your chart release management tool of choice such as or .

The steps to install the Helm chart using the Helm CLI are as follows:

1. Create a minimal values.yaml file.

1. Install the Helm chart.

For more details on all the installation options, see the Helm chart README and values.yaml files. The chart can be inspected with:

Upgrading in place is as easy as running helm upgrade:

This should keep your data in place. If you experience any issues please reach out directly and we are happy to help at support@distributional.com.

Image pull errors:

Database connection failures:

Pods not starting:

Ingress not created:

OIDC authentication failures:

• Verify auth.oidc.issuer, auth.oidc.clientId, and auth.oidc.audience match your IDP configuration

• Check that redirect URIs in your IDP include https://YOUR_DOMAIN/auth/callback

• Ensure OIDC scopes include at minimum: openid email profile

After deployment, verify the installation:

Need more help? Contact