dbnl

close_run

dbnl.close_run(*, run: Run, wait_for_close: bool = True) → None

Mark the specified dbnl Run status as closed. A closed run is finalized and considered complete. Once a Run is marked as closed, it can no longer be used for reporting Results.

Note that the Run will not be closed immediately. It will transition into a closing state and will be closed in the background. If wait_for_close is set to True, the function will block for up to 3 minutes until the Run is closed.

• Parameters:

  • run – The Run to be closed

  • wait_for_close – If True, the function will block for up to 3 minutes until the Run is closed, defaults to True

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

  • DBNLError – Run did not close after waiting for 3 minutes

IMPORTANT

A run must be closed for uploaded results to be shown on the UI.

copy_project

dbnl.copy_project(*, project: Project, name: str, description: str | None = None) → Project

Copy a Project; a convenience method wrapping exporting and importing a project with a new name and description

• Parameters:

  • project – The project to copy

  • name – A name for the new Project

  • description – An optional description for the new Project. Description is limited to 255 characters.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

  • DBNLConflictingProjectError – Project with the same name already exists

• Returns: The newly created Project

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_proj1")
proj2 = dbnl.copy_project(project=proj1, name="test_proj2")

assert proj2.name == "test_proj2"

create_metric

dbnl.create_metric(*, project: Project, name: str, expression_template: str, description: str | None = None, greater_is_better: bool | None = None) → Metric

Create a new Metric

• Parameters:

  • project – The Project to create the Metric for

  • name – Name for the Metric

  • expression_template – Expression template string e.g. token_count({RUN}.question)

  • description – Optional description of what computation the metric is performing

  • greater_is_better – Flag indicating whether greater values are semantically ‘better’ than lesser values

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

• Returns: Created Metric

create_project

dbnl.create_project(*, name: str, description: str | None = None) → Project

Create a new Project

• Parameters:

  • name – Name for the Project

  • description – Description for the Project, defaults to None. Description is limited to 255 characters.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLAPIValidationError – dbnl API failed to validate the request

  • DBNLConflictingProjectError – Project with the same name already exists

• Returns: Project

Examples:

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")

# DBNLConflictingProjectError: A Project with name test_p1 already exists.
proj_2 = dbnl.create_project(name="test_p1")

create_run

dbnl.create_run(*, project: Project, run_schema: RunSchema, display_name: str | None = None, metadata: dict[str, str] | None = None) → Run

Create a new Run

• Parameters:

  • project – The Project this Run is associated with.

  • run_schema – The schema for data that will be associated with this run. dbnl will validate data you upload against this schema.

  • display_name – An optional display name for the Run, defaults to None. display_name does not have to be unique.

  • metadata – Additional key-value pairs you want to track, defaults to None.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

• Returns: Newly created Run

create_run_query

dbnl.create_run_query(*, project: Project, name: str, query: dict[str, Any]) → RunQuery

Create a new RunQuery for a project to use as a baseline Run. Currently supports key=”offset_from_now” with value as a positive integer, representing the number of runs to go back for the baseline. For example, query={“offset_from_now”: 1} will use the latest run as the baseline, so that each run compares against the previous run.

• Parameters:

  • project – The Project to create the RunQuery for

  • name – A name for the RunQuery

  • query – A dict describing how to find a Run dynamically. Currently, only supports “offset_from_now”: int as a key-value pair.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

• Returns: A new dbnl RunQuery, typically used for finding a Dynamic Baseline for a Test Session

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
run_query1 = dbnl.create_run_query(
    project=project,
    name="look back 3",
    query={
        "offset_from_now": 3,
    },
)

create_run_schema

dbnl.create_run_schema(*, columns: Sequence[RunSchemaColumnSchemaDict], scalars: Sequence[RunSchemaScalarSchemaDict] | None = None, index: list[str] | None = None, components_dag: dict[str, list[str]] | None = None) → RunSchema

Create a new RunSchema

• Parameters:

  • columns – List of column schema specs for the uploaded data, required keys name and type, optional keys component, description and greater_is_better.

  • scalars – List of scalar schema specs for the uploaded data, required keys name and type, optional keys component, description and greater_is_better.

  • index – Optional list of column names that are the unique identifier.

  • components_dag – Optional dictionary representing the DAG of components.

• Returns: The RunSchema

Supported Types

• int

• float

• boolean

• string

• category

• list

Components

The optional component key is for specifying the source of the data column in relationship to the AI/ML app subcomponents. Components are used in visualizing the components DAG.

The components_dag dictionary specifies the topological layout of the AI/ML app. For each key-value pair, the key represents the source component, and the value is a list of the leaf components. The following code snippet describes the DAG shown above.

components_dags={
    "TweetSource": ["EntityExtractor", "SentimentClassifier"],
    "EntityExtractor": ["TradeRecommender"],
    "SentimentClassifier": ["TradeRecommender"],
    "TradeRecommender": [],
    "Global": [],
}

Examples:

Basic

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
schema = dbnl.create_run_schema(
    project=proj,
    columns=[
        {"name": "error_type", "type": "category"},
        {"name": "email", "type": "string", "description": "raw email text content from source"},
        {"name": "spam-pred", "type": "boolean"},
    ],
)

With `scalars`, `index`, and `components_dag`

import dbnl
dbnl.login()

proj = dbnl.get_or_create_project(name="test_p1")
schema = dbnl.create_run_schema(
    columns=[
        {"name": "error_type", "type": "category", "component": "classifier"},
        {"name": "email", "type": "string", "description": "raw email text content from source", "component": "input"},
        {"name": "spam-pred", "type": "boolean", "component": "classifier"},
        {"name": "email_id", "type": "string", "description": "unique id for each email"},
    ],
    scalars=[
        {"name": "model_F1", "type": "float"},
        {"name": "model_recall", "type": "float"},
    ],
    index=["email_id"],
    components_dag={
        "input": ["classifier"],
        "classifier": [],
    },
)

create_run_schema_from_results

dbnl.create_run_schema_from_results(*, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None, index: list[str] | None = None) → RunSchema

Create a new RunSchema from the column results, as well as scalar results if provided

• Parameters:

  • column_data – A pandas DataFrame with all the column results for which we want to generate a RunSchema.

  • scalar_data – A dict or pandas DataFrame with all the scalar results for which we want to generate a RunSchema.

  • index – An optional list of the column names that can be used as unique identifiers.

• Raises:DBNLInputValidationError – Input does not conform to expected format

• Returns: The RunSchema based on the provided results

Examples:

import dbnl
impodt pandas as pd

dbnl.login()

column_data = pd.DataFrame({
    "id": [1, 2, 3],
    "question": [
        "What is the meaning of life?",
        "What is the airspeed velocity of an unladen swallow?",
        "What is the capital of Assyria?",
    ],
})
scalar_data = {"int_scalar": 42, "string_scalar": "foobar"}

run_schema = dbnl.create_run_schema_from_results(
    column_data=column_data,
    scalar_data=scalar_data,
    index=["id"],
)

create_test_session

dbnl.create_test_session(*, experiment_run: Run, baseline: Run | RunQuery | None = None, include_tags: list[str] | None = None, exclude_tags: list[str] | None = None, require_tags: list[str] | None = None) → TestSession

Create a new TestSession with the given Run as the Experiment Run, and the given Run or RunQuery as the baseline if provided

• Parameters:

  • experiment_run – The Run to create the TestSession for

  • baseline – The Run or RunQuery to use as the Baseline Run, defaults to None. If None, the Baseline set for the Project is used.

  • include_tags – Optional list of Test Tag names to include in the Test Session.

  • exclude_tags – Optional list of Test Tag names to exclude in the Test Session.

  • require_tags – Optional list of Test Tag names to require in the Test Session.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

• Returns: The newly created TestSession

Calling this will start evaluating Tests associated with a Run. Typically, the Run you just completed will be the “Experiment” and you’ll compare it to some earlier “Baseline Run”.

IMPORTANT

Referenced Runs must already be closed before a Test Session can begin.

Managing Tags

Suppose we have the following Tests with the associated Tags in our Project

• Test1 with tags [“A”, “B”]

• Test2 with tags [“A”]

• Test3 with tags [“B”]

include_tags=[“A”, “B”] will trigger Tests 1, 2, and 3. require_tags=[“A”, “B”] will only trigger Test 1. exclude_tags=[“A”] will only trigger Test 3. include_tags=[“A”] and exclude_tags=[“B”] will only trigger Test 2.

Examples:

import dbnl
dbnl.login()

run = dbnl.get_run(run_id="run_0000000")
# Will default baseline to the Project's Baseline
dbnl.create_test_session(
    experiment_run=run,
)

delete_metric

dbnl.delete_metric(*, metric_id: str) → None

Delete a Metric by ID

• Parameters:metric_id – ID of the metric to delete

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLAPIValidationError – dbnl API failed to validate the request

• Returns: None

export_project_as_json

dbnl.export_project_as_json(*, project: Project) → dict[str, Any]

Export a Project alongside its Test Specs, Tags, and Notification Rules as a JSON object

• Parameters:project – The Project to export as JSON.

• Raises:DBNLNotLoggedInError – dbnl SDK is not logged in

• Returns: JSON object representing the Project

Sample Project JSON

{
    "project": {
        "name": "My Project",
        "description": "This is my project."
    },
    "notification_rules": [
        {
            "conditions": [
                {
                    "assertion_name": "less_than",
                    "assertion_params": { "other": 0.85 },
                    "query_name": "test_status_percentage_query",
                    "query_params": {
                        "exclude_tag_ids": [],
                        "include_tag_ids": [],
                        "require_tag_ids": [],
                        "statuses": ["PASSED"]
                    }
                }
            ],
            "name": "Alert if passed tests are less than 85%",
            "notification_integration_names": ["Notification channel"],
            "status": "ENABLED",
            "trigger": "test_session.failed"
        }
    ],
    "tags": [
        {
            "name": "my-tag",
            "description" :"This is my tag."
        }
    ],
    "test_specs": [
        {
            "assertion": { "name": "less_than", "params": { "other": 0.5 } },
            "description": "Testing the difference in the example statistic",
            "name": "Gr.0: Non Parametric Difference: Example_Statistic",
            "statistic_inputs": [
                {
                    "select_query_template": {
                        "filter": null,
                        "select": "{EXPERIMENT}.Example_Statistic"
                    }
                },
                {
                    "select_query_template": {
                        "filter": null,
                        "select": "{BASELINE}.Example_Statistic"
                    }
                }
            ],
            "statistic_name": "my_stat",
            "statistic_params": {},
            "tag_names": ["my-tag"]
        }
    ]
}

Examples:

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_proj")
export_json = dbnl.export_project_as_json(project=proj)

assert export_json["project"]["name"] == "test_proj"

get_column_results

dbnl.get_column_results(*, run: Run) → DataFrame

Get column results for a Run

• Parameters:run – The Run from which to retrieve the results.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

  • DBNLDownloadResultsError – Failed to download results (e.g. Run is not closed)

• Returns: A pandas DataFrame of the column results for the Run.

IMPORTANT

You can only retrieve results for a Run that has been closed.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
uploaded_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=test_data,
)

downloaded_data = dbnl.get_column_results(run=run)
assert downloaded_data.equals(uploaded_data)

get_latest_run

dbnl.get_latest_run(*, project: Project) → Run

Get the latest Run for a project

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLResourceNotFoundError – Run not found

• Parameters:project – The Project to get the latest Run for

• Returns: The latest Run

get_metric_by_id

dbnl.get_metric_by_id(*, metric_id: str) → Metric

Get a Metric by ID

• Parameters:metric_id – ID of the metric to get

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLAPIValidationError – dbnl API failed to validate the request

• Returns: The requested metric

get_my_namespaces

dbnl.get_my_namespaces() → list[Any]

Get all the namespaces that the user has access to

• Raises:DBNLNotLoggedInError – dbnl SDK is not logged in

• Returns: List of namespaces

get_or_create_project

dbnl.get_or_create_project(*, name: str, description: str | None = None) → Project

Get the Project with the specified name or create a new one if it does not exist

• Parameters:

  • name – Name for the Project

  • description – Description for the Project, defaults to None

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLAPIValidationError – dbnl API failed to validate the request

• Returns: Newly created or matching existing Project

Examples:

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")
proj_2 = dbnl.get_or_create_project(name="test_p1")

# Calling get_or_create_project will yield same Project object
assert proj_1.id == proj_2.id

get_project

dbnl.get_project(*, name: str) → Project

Retrieve a Project by name.

• Parameters:name – The name for the existing Project.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLProjectNotFoundError – Project with the given name does not exist.

• Returns: Project

Examples:

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")
proj_2 = dbnl.get_project(name="test_p1")

# Calling get_project will yield same Project object
assert proj_1.id == proj_2.id

# DBNLProjectNotFoundError: A dnnl Project with name not_exist does not exist
proj_3 = dbnl.get_project(name="not_exist")

get_results

dbnl.get_results(*, run: Run) → ResultData

Get all results for a Run

• Parameters:run – The Run from which to retrieve the results.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

  • DBNLDownloadResultsError – Failed to download results (e.g. Run is not closed)

• Returns: A named tuple comprised of columns and scalars fields. These are the pandas DataFrames of the uploaded data for the Run.

IMPORTANT

You can only retrieve results for a Run that has been closed.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")

uploaded_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=uploaded_data,
)

downloaded_data = dbnl.get_results(run=run)
assert downloaded_data.columns.equals(uploaded_data)

get_run

dbnl.get_run(*, run_id: str) → Run

Retrieve a Run with the given ID

• Parameters:run_id – The ID of the dbnl Run. Run ID starts with the prefix run_. Run ID can be found at the Run detail page.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

  • DBNLRunNotFoundError – A Run with the given ID does not exist.

• Returns: The Run with the given run_id.

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(project=proj1, columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_schema=schema1)

# Retrieving the Run by ID
run2 = dbnl.get_run(run_id=run1.id)
assert run1.id == run2.id

# DBNLRunNotFoundError: A Run with id run_0000000 does not exist.
run3 = dbnl.get_run(run_id="run_0000000")

get_run_query

dbnl.get_run_query(*, project: Project, name: str) → RunQuery

Retrieve a RunQuery with the given name, unique to a project

• Parameters:

  • project – The Project from which to retrieve the RunQuery.

  • name – The name of the RunQuery to retrieve.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLRessourceNotFoundError – RunQuery not found

• Returns: RunQuery with the given name.

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
run_query1 = dbnl.get_run_query(
    project=project,
    name="look back 3"
)

get_scalar_results

dbnl.get_scalar_results(*, run: Run) → DataFrame

Get scalar results for a Run

• Parameters:run – The Run from which to retrieve the scalar results.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

  • DBNLDownloadResultsError – Failed to download results (e.g. Run is not closed)

• Returns: A pandas DataFrame of the scalar results for the Run.

IMPORTANT

You can only retrieve results for a Run that has been closed.

Examples:

import dbnl
import pandas as pd
dbnl.login()

proj1 = dbnl.get_or_create_project(name="test_p1")

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=data,
    scalar_results={"rmse": 0.37}
)

downloaded_scalars = dbnl.get_scalar_results(run=run)

import_project_from_json

dbnl.import_project_from_json(*, params: dict[str, Any]) → Project

Create a new Project from a JSON object

• Parameters:params – JSON object representing the Project, generally based on a Project exported via export_project_as_json(). See export_project_as_json() for the expected format.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLAPIValidationError – dbnl API failed to validate the request

  • DBNLConflictingProjectError – Project with the same name already exists

• Returns: Project created from the JSON object

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_proj1")
export_json = dbnl.export_project_as_json(project=proj1)
export_json["project"]["name"] = "test_proj2"
proj2 = dbnl.import_project_from_json(params=export_json)

assert proj2.name == "test_proj2"

login

dbnl.login(*, api_token: str | None = None, namespace_id: str | None = None, api_url: str | None = None, app_url: str | None = None) → None

Setup dbnl SDK to make authenticated requests. After login is run successfully, the dbnl client will be able to issue secure and authenticated requests against hosted endpoints of the dbnl service.

• Parameters:

  • api_token – dbnl API token for authentication; token can be found at /tokens page of the dbnl app. If None is provided, the environment variable DBNL_API_TOKEN will be used by default.

  • namespace_id – The namespace ID to use for the session; available namespaces can be found with get_my_namespaces().

  • api_url – The base url of the Distributional API. For SaaS users, set this variable to api.dbnl.com. For other users, please contact your sys admin. If None is provided, the environment variable DBNL_API_URL will be used by default.

  • app_url – An optional base url of the Distributional app. If this variable is not set, the app url is inferred from the DBNL_API_URL variable. For on-prem users, please contact your sys admin if you cannot reach the Distributional UI.

report_column_results

dbnl.report_column_results(*, run: Run, data: DataFrame) → None

Report all column results to dbnl

• Parameters:

  • run – The Run that the results will be reported to

  • data – A pandas DataFrame with all the results to report to dbnl. The columns of the DataFrame must match the columns of the Run’s schema.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

IMPORTANT

All data should be reported to dbnl at once. Calling dbnl.report_column_results more than once will overwrite the previously uploaded data.

WARNING

Once a Run is closed, you can no longer call report_column_results to send data to dbnl.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_schema=schema1)

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_column_results(run=run1, data=data)

report_results

dbnl.report_results(*, run: Run, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None) → None

Report all results to dbnl

• Parameters:

  • run – The Run that the results will be reported to

  • column_data – A pandas DataFrame with all the results to report to dbnl. The columns of the DataFrame must match the columns of the Run’s schema.

  • scalar_data – A dictionary or single-row pandas DataFrame with the scalar results to report to dbnl, defaults to None.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

IMPORTANT

All data should be reported to dbnl at once. Calling dbnl.report_results more than once will overwrite the previously uploaded data.

WARNING

Once a Run is closed, you can no longer call report_results to send data to dbnl.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(
    columns=[{"name": "error", "type": "float"}],
    scalars=[{"name": "rmse": "type": "float"}],
)
run1 = dbnl.create_run(project=proj1, run_schema=schema1)
data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_results(run=run1, column_data=data, scalar_data={"rmse": 0.37})

report_run_with_results

dbnl.report_run_with_results(*, project: Project, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None, display_name: str | None = None, index: list[str] | None = None, run_schema: RunSchema | None = None, metadata: dict[str, str] | None = None, wait_for_close: bool = True) → Run

Create a new Run, report results to it, and close it.

• Parameters:

  • project – The Project to create the Run in.

  • column_data – A pandas DataFrame with the results for the columns.

  • scalar_data – An optional dictionary or DataFrame with the results for the scalars, if any.

  • display_name – An optional display name for the Run.

  • index – An optional list of column names to use as the unique identifier for rows in the column data.

  • run_schema – An optional RunSchema to use for the Run. Will be inferred from the data if not provided.

  • metadata – Any additional key:value pairs you want to track.

  • wait_for_close – If True, the function will block for up to 3 minutes until the Run is closed, defaults to True.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

• Returns: The closed Run with the uploaded data.

IMPORTANT

If no schema is provided, the schema will be inferred from the data. If provided, the schema will be used to validate the data.

Examples:

Implicit Schema

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})

run = dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
)

Explicit Schema

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run_schema = dbnl.create_run_schema(columns=[
{"name": "error", "type": "float"}
])

run = dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
    run_schema=run_schema
)

try:
run_schema = dbnl.create_run_schema(columns=[
    {"name": "error", "type": "string"}
])
dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
    run_schema=run_schema
)
except DBNLInputValidationError:
# We expect DBNLInputValidationError because the type of
# `error` in the input data is "float", but we provided a `RunSchema`
# which specifies the columm type as "string".
assert True
else:
# should not get here
assert False

report_run_with_results_and_start_test_session

dbnl.report_run_with_results_and_start_test_session(*, project: Project, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None, display_name: str | None = None, index: list[str] | None = None, run_schema: RunSchema | None = None, metadata: dict[str, str] | None = None, baseline: Run | RunQuery | None = None, include_tags: list[str] | None = None, exclude_tags: list[str] | None = None, require_tags: list[str] | None = None) → Run

Create a new Run, report results to it, and close it. Wait for close to finish and start a TestSession with the given inputs.

• Parameters:

  • project – The Project to create the Run in.

  • column_data – A pandas DataFrame with the results for the columns.

  • scalar_data – An optional dictionary or DataFrame with the results for the scalars, if any.

  • display_name – An optional display name for the Run.

  • index – An optional list of column names to use as the unique identifier for rows in the column data.

  • run_schema – An optional RunSchema to use for the Run. Will be inferred from the data if not provided.

  • metadata – Any additional key:value pairs you want to track.

  • wait_for_close – If True, the function will block for up to 3 minutes until the Run is closed, defaults to True.

  • baseline – The Run or RunQuery to use as the baseline run, defaults to None. If None, the baseline defined in the TestConfig is used.

  • include_tags – Optional list of Test Tag names to include in the Test Session.

  • exclude_tags – Optional list of Test Tag names to exclude in the Test Session.

  • require_tags – Optional list of Test Tag names to require in the Test Session.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

• Returns: The closed Run with the uploaded data.

IMPORTANT

If no schema is provided, the schema will be inferred from the data. If provided, the schema will be used to validate the data.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})

run = dbnl.report_run_with_results_and_start_test_session(
    project=proj,
    column_data=test_data,
)

report_scalar_results

dbnl.report_scalar_results(*, run: Run, data: dict[str, Any] | DataFrame) → None

Report scalar results to dbnl

• Parameters:

  • run – The Run that the scalars will be reported to

  • data – A dictionary or single-row pandas DataFrame with the scalar results to report to dbnl.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLInputValidationError – Input does not conform to expected format

IMPORTANT

All data should be reported to dbnl at once. Calling dbnl.report_scalar_results more than once will overwrite the previously uploaded data.

WARNING

Once a Run is closed, you can no longer call report_scalar_results to send data to dbnl.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(
    columns=[{"name": "error", "type": "float"}],
    scalars=[{"name": "rmse": "type": "float"}],
)
run1 = dbnl.create_run(project=proj1, run_schema=schema1)
dbnl.report_scalar_results(run=run1, data={"rmse": 0.37})

set_run_as_baseline

dbnl.set_run_as_baseline(*, run: Run) → None

Set the given Run as the Baseline Run in the Project’s Test Config

• Parameters:run – The Run to set as the Baseline Run.

• Raises:DBNLResourceNotFoundError – If the test configurations are not found for the project.

set_run_query_as_baseline

dbnl.set_run_query_as_baseline(*, run_query: RunQuery) → None

Set a given RunQuery as the Baseline Run in a Project’s Test Config

• Parameters:run_query – The RunQuery to set as the Baseline RunQuery.

• Raises:DBNLResourceNotFoundError – If the test configurations are not found for the project.

wait_for_run_close

dbnl.wait_for_run_close(*, run: Run, timeout_s: float = 180.0, polling_interval_s: float = 3.0) → Run

Wait for a Run to close. Polls every polling_interval_s seconds until it is closed.

• Parameters:

  • run – Run to wait for

  • timeout_s – Total wait time (in seconds) for Run to close, defaults to 180.0

  • polling_interval_s – Time between polls (in seconds), defaults to 3.0

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in

  • DBNLError – Run did not close after waiting for the timeout_s seconds

  • DBNLRunError – Run ended in ERRORED state