Quickstart

Get started with dbnl

This guide walks you through using the Distributional SDK to create your first project, submit two runs and create a test session to compare the behavior of your AI application over time.

1

Install the dbnl SDK.

First, if you haven't done so already, install the dbnl Python SDK.

pip install dbnl

2

Export the dbnl environment variables

If you haven't done so already, create a personal access token by going to ☰ > Personal Access Tokens.

Run this command to export the path to the API for your deployment and your personal access token as environment variables.

export DBNL_API_URL="YOUR_DBNL_API_URL"
export DBNL_API_TOKEN="YOUR_PERSONAL_ACCESS_TOKEN"

3

Create and test two Runs

In a Python script or notebook, add the following code to create two Runs and test them in a Test Session using the first run as the baseline and the second run as the experiment.

import random
from datetime import datetime

import dbnl
import pandas as pd

# Login to dbnl.
dbnl.login()

# Create a new project
now = datetime.now().isoformat()
project = dbnl.get_or_create_project(name=f"quickstart-{now}")

# Submit a first run (baseline)
run1 = dbnl.report_run_with_results(
    project=project,
    display_name="run1",
    column_data=pd.DataFrame([
        {
            "question": f"Is {i} an even or odd number?",
            "answer": random.choice(["even", "odd"]),
        }
        for i in range(20)
    ]).astype({"question": "string", "answer": "category"}),
)

# Submit a second run (experiment)
run2 = dbnl.report_run_with_results(
    project=project,
    display_name="run2",
    column_data=pd.DataFrame([
        {
            "question": f"Is {i} an even or odd number?",
            "answer": random.choice(["even", "odd"]),
        }
        for i in range(20)
    ]).astype({"question": "string", "answer": "category"}),
)

# Create a test session.
dbnl.set_run_as_baseline(run=run1)
dbnl.create_test_session(experiment_run=run2)

4

View your Test Session results

Congratulations! You ran your first Test Session. You can see the results of the Test Session by navigating to your project in the dbnl app and selecting your test session from the test session table.

By default, a similarity index test is added that tests whether your application has changed between the baseline and experiment run.

Next Steps

• Create a Project for your own AI application.

• Upload your own data as Runs to your Project.

• Define Metrics to augment your Runs with novel quantities.

• Add more Tests to ensure your application behaves as expected.

• Learn more about Similarity Index.

• Use Notifications to be alerted when tests fail.