Data Connections
How to get data into DBNL
Data Connections are how production AI log data is ingested into your DBNL Deployment as part of the Data Pipeline. Each Project has one ingestion method that is set at creation. If you need to change this later you can do this via the Project settings page.
DBNL supports three methods of data ingestion:
OTEL Trace Ingestion: Publish OTEL traces directly to DBNL as the product runs.
SDK Log Ingestion: Push data manually or as part of a daily orchestration job using the Python SDK.
SQL Integration Ingestion: Pull data from a SQL table into DBNL on a schedule.
Regardless of the data ingestion method, make sure your data adheres to the DBNL Semantic Convention to enable the richest analysis of the data.
OTEL Trace Ingestion
Get rich data logged in a few lines of embedded code
Enables full trace inspection in Logs page
Automatically maps to DBNL Semantic Convention if using standard semantic types
Cannot backfill data, requiring a full week before first Insights
Clickhouse support needs to be enabled during installation of Deployment
SDK Log Ingestion
Most flexible, can contain a full trace as part of a log line
Can backfill previously logged data
Requires Python SDK code to be written and scheduled as part of external orchestration service
SQL Integration Ingestion
Leverages currently logged data in SQL tables
No code required
Scheduling from DBNL Platform
Can backfill previously logged data
Requires log data to be already flattened and stored in a SQL table
May require complex SQL queries to adapt to DBNL Semantic Convention
Managing Data Connections
Creating a New Data Connection
From the Namespace landing page click on "Data Connections" on the left panel. On the Data Connections landing page "+ Add Data Connection" in the upper right. Provide a required name for the Data Connection and an optional description. All Data Connections will be available to any User creating a Project in the Namespace.
Was this helpful?