SDK Log Ingestion

Use the Python SDK to upload log data

Push data manually or as part of a daily orchestration job using our Python SDK. This ingestion method allows for the most flexibility, but requires the most off-platform coding.

See the Python SDK docs for more detailed information about SDK installation and functions.

The following fields are required regardless of which ingestion method you are using:

• input: The text input to the LLM as a string.

• output: The text response from the LLM as a string.

• timestamp: The UTC timecode associated with the LLM call as a timestamptz.

See the DBNL Semantic Convention for other semantically recognized fields. Using these naming conventions will allow DBNL to map semantic meaning to those columns and provide better Insights.

Example Code

Your DBNL_API_URL is set during Deployment. See API Authentication docs for finding your DBNL_API_TOKEN.

DBNL_API_URL = "http://localhost:8080/api"
DBNL_API_TOKEN = ""

import random
from datetime import UTC, datetime, timedelta

import dbnl
import pandas as pd

# Login to dbnl.
dbnl.login(api_url=DBNL_API_URL, api_token=DBNL_API_TOKEN)
# Use current time as reference point.
now = datetime.now(tz=UTC)
# Get or create a new project.
project = dbnl.get_or_create_project(
    name=f"quickstart-{now.isoformat()}",
    schedule="daily",
)

# Backfill first 8 days of data.
now_date = now.replace(hour=0, minute=0, second=0, microsecond=0)
start_date = now_date - timedelta(days=8)
end_date = now_date - timedelta(days=1)
for dt in pd.date_range(start_date, end_date):
    dbnl.report_run_with_results(
        project=project,
        data_start_time=dt,
        data_end_time=dt + timedelta(days=1),
        column_data=pd.DataFrame([
            {
                "timestamp": dt + timedelta(minutes=30 * i),
                "input": f"Is {i} an even or odd number?",
                "output": random.choice(["even", "odd"]),
            }
            for i in range(20)
        ]).astype({
            "timestamp": "datetime64[us, UTC]",
            "input": "string",
            "output": "category",
        }),
    )