dbnl

close_run

dbnl.close_run(*, run: Run, wait_for_close: bool = True) → None

Mark the specified dbnl Run status as closed. A closed run is finalized and considered complete. Once a Run is marked as closed, it can no longer be used for reporting Results.

Note that the Run will not be closed immediately. It will transition into a closing state and will be closed in the background. If wait_for_close is set to True, the function will block for up to 3 minutes until the Run is closed.

Parameters:
- run – The Run to be closed
- wait_for_close – If True, the function will block for up to 3 minutes until the Run is closed, defaults to True
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
- DBNLError – Run did not close after waiting for 3 minutes

IMPORTANT

A run must be closed for uploaded results to be shown on the UI.

copy_project

dbnl.copy_project(*, project: Project, name: str, description: str | None = None) → Project

Copy a Project; a convenience method wrapping exporting and importing a project with a new name and description

Parameters:
- project – The project to copy
- name – A name for the new Project
- description – An optional description for the new Project. Description is limited to 255 characters.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
- DBNLConflictingProjectError – Project with the same name already exists
Returns: The newly created Project

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_proj1")
proj2 = dbnl.copy_project(project=proj1, name="test_proj2")

assert proj2.name == "test_proj2"

create_metric

dbnl.create_metric(*, project: Project, name: str, expression_template: str, description: str | None = None, greater_is_better: bool | None = None) → Metric

Create a new Metric

Parameters:
- project – The Project to create the Metric for
- name – Name for the Metric
- expression_template – Expression template string e.g. token_count({RUN}.question)
- description – Optional description of what computation the metric is performing
- greater_is_better – Flag indicating whether greater values are semantically ‘better’ than lesser values
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
Returns: Created Metric

create_project

dbnl.create_project(*, name: str, description: str | None = None) → Project

Create a new Project

Parameters:
- name – Name for the Project
- description – Description for the Project, defaults to None. Description is limited to 255 characters.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLAPIValidationError – dbnl API failed to validate the request
- DBNLConflictingProjectError – Project with the same name already exists
Returns: Project

Examples:

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")

# DBNLConflictingProjectError: A Project with name test_p1 already exists.
proj_2 = dbnl.create_project(name="test_p1")

create_run

dbnl.create_run(*, project: Project, run_schema: RunSchema, display_name: str | None = None, metadata: dict[str, str] | None = None) → Run

Create a new Run

Parameters:
- project – The Project this Run is associated with.
- run_schema – The schema for data that will be associated with this run. dbnl will validate data you upload against this schema.
- display_name – An optional display name for the Run, defaults to None. display_name does not have to be unique.
- metadata – Additional key-value pairs you want to track, defaults to None.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
Returns: Newly created Run

create_run_query

dbnl.create_run_query(*, project: Project, name: str, query: dict[str, Any]) → RunQuery

Create a new RunQuery for a project to use as a baseline Run. Currently supports key=”offset_from_now” with value as a positive integer, representing the number of runs to go back for the baseline. For example, query={“offset_from_now”: 1} will use the latest run as the baseline, so that each run compares against the previous run.

Parameters:
- project – The Project to create the RunQuery for
- name – A name for the RunQuery
- query – A dict describing how to find a Run dynamically. Currently, only supports “offset_from_now”: int as a key-value pair.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
Returns: A new dbnl RunQuery, typically used for finding a Dynamic Baseline for a Test Session

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
run_query1 = dbnl.create_run_query(
    project=project,
    name="look back 3",
    query={
        "offset_from_now": 3,
    },
)

create_run_schema

dbnl.create_run_schema(*, columns: Sequence[RunSchemaColumnSchemaDict], scalars: Sequence[RunSchemaScalarSchemaDict] | None = None, index: list[str] | None = None, components_dag: dict[str, list[str]] | None = None) → RunSchema

Create a new RunSchema

Parameters:
- columns – List of column schema specs for the uploaded data, required keys name and type, optional keys component, description and greater_is_better.
- scalars – List of scalar schema specs for the uploaded data, required keys name and type, optional keys component, description and greater_is_better.
- index – Optional list of column names that are the unique identifier.
- components_dag – Optional dictionary representing the DAG of components.
Returns: The RunSchema

Supported Types

int
float
boolean
string
category
list

Components

The optional component key is for specifying the source of the data column in relationship to the AI/ML app subcomponents. Components are used in visualizing the components DAG.

The components_dag dictionary specifies the topological layout of the AI/ML app. For each key-value pair, the key represents the source component, and the value is a list of the leaf components. The following code snippet describes the DAG shown above.

components_dags={
    "TweetSource": ["EntityExtractor", "SentimentClassifier"],
    "EntityExtractor": ["TradeRecommender"],
    "SentimentClassifier": ["TradeRecommender"],
    "TradeRecommender": [],
    "Global": [],
}

Examples:

Basic

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
schema = dbnl.create_run_schema(
    project=proj,
    columns=[
        {"name": "error_type", "type": "category"},
        {"name": "email", "type": "string", "description": "raw email text content from source"},
        {"name": "spam-pred", "type": "boolean"},
    ],
)

With `scalars`, `index`, and `components_dag`

import dbnl
dbnl.login()

proj = dbnl.get_or_create_project(name="test_p1")
schema = dbnl.create_run_schema(
    columns=[
        {"name": "error_type", "type": "category", "component": "classifier"},
        {"name": "email", "type": "string", "description": "raw email text content from source", "component": "input"},
        {"name": "spam-pred", "type": "boolean", "component": "classifier"},
        {"name": "email_id", "type": "string", "description": "unique id for each email"},
    ],
    scalars=[
        {"name": "model_F1", "type": "float"},
        {"name": "model_recall", "type": "float"},
    ],
    index=["email_id"],
    components_dag={
        "input": ["classifier"],
        "classifier": [],
    },
)

create_run_schema_from_results

dbnl.create_run_schema_from_results(*, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None, index: list[str] | None = None) → RunSchema

Create a new RunSchema from the column results, as well as scalar results if provided

Parameters:
- column_data – A pandas DataFrame with all the column results for which we want to generate a RunSchema.
- scalar_data – A dict or pandas DataFrame with all the scalar results for which we want to generate a RunSchema.
- index – An optional list of the column names that can be used as unique identifiers.
Raises:DBNLInputValidationError – Input does not conform to expected format
Returns: The RunSchema based on the provided results

Examples:

import dbnl
impodt pandas as pd

dbnl.login()

column_data = pd.DataFrame({
    "id": [1, 2, 3],
    "question": [
        "What is the meaning of life?",
        "What is the airspeed velocity of an unladen swallow?",
        "What is the capital of Assyria?",
    ],
})
scalar_data = {"int_scalar": 42, "string_scalar": "foobar"}

run_schema = dbnl.create_run_schema_from_results(
    column_data=column_data,
    scalar_data=scalar_data,
    index=["id"],
)

create_test_session

dbnl.create_test_session(*, experiment_run: Run, baseline: Run | RunQuery | None = None, include_tags: list[str] | None = None, exclude_tags: list[str] | None = None, require_tags: list[str] | None = None) → TestSession

Create a new TestSession with the given Run as the Experiment Run, and the given Run or RunQuery as the baseline if provided

Parameters:
- experiment_run – The Run to create the TestSession for
- baseline – The Run or RunQuery to use as the Baseline Run, defaults to None. If None, the Baseline set for the Project is used.
- include_tags – Optional list of Test Tag names to include in the Test Session.
- exclude_tags – Optional list of Test Tag names to exclude in the Test Session.
- require_tags – Optional list of Test Tag names to require in the Test Session.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
Returns: The newly created TestSession

Calling this will start evaluating Tests associated with a Run. Typically, the Run you just completed will be the “Experiment” and you’ll compare it to some earlier “Baseline Run”.

IMPORTANT

Referenced Runs must already be closed before a Test Session can begin.

Managing Tags

Suppose we have the following Tests with the associated Tags in our Project

Test1 with tags [“A”, “B”]
Test2 with tags [“A”]
Test3 with tags [“B”]

include_tags=[“A”, “B”] will trigger Tests 1, 2, and 3. require_tags=[“A”, “B”] will only trigger Test 1. exclude_tags=[“A”] will only trigger Test 3. include_tags=[“A”] and exclude_tags=[“B”] will only trigger Test 2.

Examples:

import dbnl
dbnl.login()

run = dbnl.get_run(run_id="run_0000000")
# Will default baseline to the Project's Baseline
dbnl.create_test_session(
    experiment_run=run,
)

delete_metric

dbnl.delete_metric(*, metric_id: str) → None

Delete a Metric by ID

Parameters:metric_id – ID of the metric to delete
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLAPIValidationError – dbnl API failed to validate the request
Returns: None

export_project_as_json

dbnl.export_project_as_json(*, project: Project) → dict[str, Any]

Export a Project alongside its Test Specs, Tags, and Notification Rules as a JSON object

Parameters:project – The Project to export as JSON.
Raises:DBNLNotLoggedInError – dbnl SDK is not logged in
Returns: JSON object representing the Project

Sample Project JSON

{
    "project": {
        "name": "My Project",
        "description": "This is my project."
    },
    "notification_rules": [
        {
            "conditions": [
                {
                    "assertion_name": "less_than",
                    "assertion_params": { "other": 0.85 },
                    "query_name": "test_status_percentage_query",
                    "query_params": {
                        "exclude_tag_ids": [],
                        "include_tag_ids": [],
                        "require_tag_ids": [],
                        "statuses": ["PASSED"]
                    }
                }
            ],
            "name": "Alert if passed tests are less than 85%",
            "notification_integration_names": ["Notification channel"],
            "status": "ENABLED",
            "trigger": "test_session.failed"
        }
    ],
    "tags": [
        {
            "name": "my-tag",
            "description" :"This is my tag."
        }
    ],
    "test_specs": [
        {
            "assertion": { "name": "less_than", "params": { "other": 0.5 } },
            "description": "Testing the difference in the example statistic",
            "name": "Gr.0: Non Parametric Difference: Example_Statistic",
            "statistic_inputs": [
                {
                    "select_query_template": {
                        "filter": null,
                        "select": "{EXPERIMENT}.Example_Statistic"
                    }
                },
                {
                    "select_query_template": {
                        "filter": null,
                        "select": "{BASELINE}.Example_Statistic"
                    }
                }
            ],
            "statistic_name": "my_stat",
            "statistic_params": {},
            "tag_names": ["my-tag"]
        }
    ]
}

Examples:

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_proj")
export_json = dbnl.export_project_as_json(project=proj)

assert export_json["project"]["name"] == "test_proj"

get_column_results

dbnl.get_column_results(*, run: Run) → DataFrame

Get column results for a Run

Parameters:run – The Run from which to retrieve the results.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
- DBNLDownloadResultsError – Failed to download results (e.g. Run is not closed)
Returns: A pandas DataFrame of the column results for the Run.

IMPORTANT

You can only retrieve results for a Run that has been closed.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
uploaded_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=test_data,
)

downloaded_data = dbnl.get_column_results(run=run)
assert downloaded_data.equals(uploaded_data)

get_latest_run

dbnl.get_latest_run(*, project: Project) → Run

Get the latest Run for a project

Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLResourceNotFoundError – Run not found
Parameters:project – The Project to get the latest Run for
Returns: The latest Run

get_metric_by_id

dbnl.get_metric_by_id(*, metric_id: str) → Metric

Get a Metric by ID

Parameters:metric_id – ID of the metric to get
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLAPIValidationError – dbnl API failed to validate the request
Returns: The requested metric

get_my_namespaces

dbnl.get_my_namespaces() → list[Any]

Get all the namespaces that the user has access to

Raises:DBNLNotLoggedInError – dbnl SDK is not logged in
Returns: List of namespaces

get_or_create_project

dbnl.get_or_create_project(*, name: str, description: str | None = None) → Project

Get the Project with the specified name or create a new one if it does not exist

Parameters:
- name – Name for the Project
- description – Description for the Project, defaults to None
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLAPIValidationError – dbnl API failed to validate the request
Returns: Newly created or matching existing Project

Examples:

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")
proj_2 = dbnl.get_or_create_project(name="test_p1")

# Calling get_or_create_project will yield same Project object
assert proj_1.id == proj_2.id

get_project

dbnl.get_project(*, name: str) → Project

Retrieve a Project by name.

Parameters:name – The name for the existing Project.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLProjectNotFoundError – Project with the given name does not exist.
Returns: Project

Examples:

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")
proj_2 = dbnl.get_project(name="test_p1")

# Calling get_project will yield same Project object
assert proj_1.id == proj_2.id

# DBNLProjectNotFoundError: A dnnl Project with name not_exist does not exist
proj_3 = dbnl.get_project(name="not_exist")

get_results

dbnl.get_results(*, run: Run) → ResultData

Get all results for a Run

Parameters:run – The Run from which to retrieve the results.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
- DBNLDownloadResultsError – Failed to download results (e.g. Run is not closed)
Returns: A named tuple comprised of columns and scalars fields. These are the pandas DataFrames of the uploaded data for the Run.

IMPORTANT

You can only retrieve results for a Run that has been closed.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")

uploaded_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=uploaded_data,
)

downloaded_data = dbnl.get_results(run=run)
assert downloaded_data.columns.equals(uploaded_data)

get_run

dbnl.get_run(*, run_id: str) → Run

Retrieve a Run with the given ID

Parameters:run_id – The ID of the dbnl Run. Run ID starts with the prefix run_. Run ID can be found at the Run detail page.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
- DBNLRunNotFoundError – A Run with the given ID does not exist.
Returns: The Run with the given run_id.

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(project=proj1, columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_schema=schema1)

# Retrieving the Run by ID
run2 = dbnl.get_run(run_id=run1.id)
assert run1.id == run2.id

# DBNLRunNotFoundError: A Run with id run_0000000 does not exist.
run3 = dbnl.get_run(run_id="run_0000000")

get_run_query

dbnl.get_run_query(*, project: Project, name: str) → RunQuery

Retrieve a RunQuery with the given name, unique to a project

Parameters:
- project – The Project from which to retrieve the RunQuery.
- name – The name of the RunQuery to retrieve.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLRessourceNotFoundError – RunQuery not found
Returns: RunQuery with the given name.

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
run_query1 = dbnl.get_run_query(
    project=project,
    name="look back 3"
)

get_scalar_results

dbnl.get_scalar_results(*, run: Run) → DataFrame

Get scalar results for a Run

Parameters:run – The Run from which to retrieve the scalar results.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
- DBNLDownloadResultsError – Failed to download results (e.g. Run is not closed)
Returns: A pandas DataFrame of the scalar results for the Run.

IMPORTANT

You can only retrieve results for a Run that has been closed.

Examples:

import dbnl
import pandas as pd
dbnl.login()

proj1 = dbnl.get_or_create_project(name="test_p1")

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=data,
    scalar_results={"rmse": 0.37}
)

downloaded_scalars = dbnl.get_scalar_results(run=run)

import_project_from_json

dbnl.import_project_from_json(*, params: dict[str, Any]) → Project

Create a new Project from a JSON object

Parameters:params – JSON object representing the Project, generally based on a Project exported via export_project_as_json(). See export_project_as_json() for the expected format.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLAPIValidationError – dbnl API failed to validate the request
- DBNLConflictingProjectError – Project with the same name already exists
Returns: Project created from the JSON object

Examples:

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_proj1")
export_json = dbnl.export_project_as_json(project=proj1)
export_json["project"]["name"] = "test_proj2"
proj2 = dbnl.import_project_from_json(params=export_json)

assert proj2.name == "test_proj2"

dbnl.login(*, api_token: str | None = None, namespace_id: str | None = None, api_url: str | None = None, app_url: str | None = None) → None

Setup dbnl SDK to make authenticated requests. After login is run successfully, the dbnl client will be able to issue secure and authenticated requests against hosted endpoints of the dbnl service.

Parameters:
- api_token – dbnl API token for authentication; token can be found at /tokens page of the dbnl app. If None is provided, the environment variable DBNL_API_TOKEN will be used by default.
- namespace_id – The namespace ID to use for the session; available namespaces can be found with get_my_namespaces().
- api_url – The base url of the Distributional API. For SaaS users, set this variable to api.dbnl.com. For other users, please contact your sys admin. If None is provided, the environment variable DBNL_API_URL will be used by default.
- app_url – An optional base url of the Distributional app. If this variable is not set, the app url is inferred from the DBNL_API_URL variable. For on-prem users, please contact your sys admin if you cannot reach the Distributional UI.

report_column_results

dbnl.report_column_results(*, run: Run, data: DataFrame) → None

Report all column results to dbnl

Parameters:
- run – The Run that the results will be reported to
- data – A pandas DataFrame with all the results to report to dbnl. The columns of the DataFrame must match the columns of the Run’s schema.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format

IMPORTANT

All data should be reported to dbnl at once. Calling dbnl.report_column_results more than once will overwrite the previously uploaded data.

WARNING

Once a Run is closed, you can no longer call report_column_results to send data to dbnl.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_schema=schema1)

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_column_results(run=run1, data=data)

report_results

dbnl.report_results(*, run: Run, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None) → None

Report all results to dbnl

Parameters:
- run – The Run that the results will be reported to
- column_data – A pandas DataFrame with all the results to report to dbnl. The columns of the DataFrame must match the columns of the Run’s schema.
- scalar_data – A dictionary or single-row pandas DataFrame with the scalar results to report to dbnl, defaults to None.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format

IMPORTANT

All data should be reported to dbnl at once. Calling dbnl.report_results more than once will overwrite the previously uploaded data.

WARNING

Once a Run is closed, you can no longer call report_results to send data to dbnl.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(
    columns=[{"name": "error", "type": "float"}],
    scalars=[{"name": "rmse": "type": "float"}],
)
run1 = dbnl.create_run(project=proj1, run_schema=schema1)
data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_results(run=run1, column_data=data, scalar_data={"rmse": 0.37})

report_run_with_results

dbnl.report_run_with_results(*, project: Project, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None, display_name: str | None = None, index: list[str] | None = None, run_schema: RunSchema | None = None, metadata: dict[str, str] | None = None, wait_for_close: bool = True) → Run

Create a new Run, report results to it, and close it.

Parameters:
- project – The Project to create the Run in.
- column_data – A pandas DataFrame with the results for the columns.
- scalar_data – An optional dictionary or DataFrame with the results for the scalars, if any.
- display_name – An optional display name for the Run.
- index – An optional list of column names to use as the unique identifier for rows in the column data.
- run_schema – An optional RunSchema to use for the Run. Will be inferred from the data if not provided.
- metadata – Any additional key:value pairs you want to track.
- wait_for_close – If True, the function will block for up to 3 minutes until the Run is closed, defaults to True.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
Returns: The closed Run with the uploaded data.

IMPORTANT

If no schema is provided, the schema will be inferred from the data. If provided, the schema will be used to validate the data.

Examples:

Implicit Schema

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})

run = dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
)

Explicit Schema

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run_schema = dbnl.create_run_schema(columns=[
{"name": "error", "type": "float"}
])

run = dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
    run_schema=run_schema
)

try:
run_schema = dbnl.create_run_schema(columns=[
    {"name": "error", "type": "string"}
])
dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
    run_schema=run_schema
)
except DBNLInputValidationError:
# We expect DBNLInputValidationError because the type of
# `error` in the input data is "float", but we provided a `RunSchema`
# which specifies the columm type as "string".
assert True
else:
# should not get here
assert False

report_run_with_results_and_start_test_session

dbnl.report_run_with_results_and_start_test_session(*, project: Project, column_data: DataFrame, scalar_data: dict[str, Any] | DataFrame | None = None, display_name: str | None = None, index: list[str] | None = None, run_schema: RunSchema | None = None, metadata: dict[str, str] | None = None, baseline: Run | RunQuery | None = None, include_tags: list[str] | None = None, exclude_tags: list[str] | None = None, require_tags: list[str] | None = None) → Run

Create a new Run, report results to it, and close it. Wait for close to finish and start a TestSession with the given inputs.

Parameters:
- project – The Project to create the Run in.
- column_data – A pandas DataFrame with the results for the columns.
- scalar_data – An optional dictionary or DataFrame with the results for the scalars, if any.
- display_name – An optional display name for the Run.
- index – An optional list of column names to use as the unique identifier for rows in the column data.
- run_schema – An optional RunSchema to use for the Run. Will be inferred from the data if not provided.
- metadata – Any additional key:value pairs you want to track.
- wait_for_close – If True, the function will block for up to 3 minutes until the Run is closed, defaults to True.
- baseline – The Run or RunQuery to use as the baseline run, defaults to None. If None, the baseline defined in the TestConfig is used.
- include_tags – Optional list of Test Tag names to include in the Test Session.
- exclude_tags – Optional list of Test Tag names to exclude in the Test Session.
- require_tags – Optional list of Test Tag names to require in the Test Session.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format
Returns: The closed Run with the uploaded data.

IMPORTANT

If no schema is provided, the schema will be inferred from the data. If provided, the schema will be used to validate the data.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})

run = dbnl.report_run_with_results_and_start_test_session(
    project=proj,
    column_data=test_data,
)

report_scalar_results

dbnl.report_scalar_results(*, run: Run, data: dict[str, Any] | DataFrame) → None

Report scalar results to dbnl

Parameters:
- run – The Run that the scalars will be reported to
- data – A dictionary or single-row pandas DataFrame with the scalar results to report to dbnl.
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLInputValidationError – Input does not conform to expected format

IMPORTANT

All data should be reported to dbnl at once. Calling dbnl.report_scalar_results more than once will overwrite the previously uploaded data.

WARNING

Once a Run is closed, you can no longer call report_scalar_results to send data to dbnl.

Examples:

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
schema1 = dbnl.create_run_schema(
    columns=[{"name": "error", "type": "float"}],
    scalars=[{"name": "rmse": "type": "float"}],
)
run1 = dbnl.create_run(project=proj1, run_schema=schema1)
dbnl.report_scalar_results(run=run1, data={"rmse": 0.37})

set_run_as_baseline

dbnl.set_run_as_baseline(*, run: Run) → None

Set the given Run as the Baseline Run in the Project’s Test Config

Parameters:run – The Run to set as the Baseline Run.
Raises:DBNLResourceNotFoundError – If the test configurations are not found for the project.

set_run_query_as_baseline

dbnl.set_run_query_as_baseline(*, run_query: RunQuery) → None

Set a given RunQuery as the Baseline Run in a Project’s Test Config

Parameters:run_query – The RunQuery to set as the Baseline RunQuery.
Raises:DBNLResourceNotFoundError – If the test configurations are not found for the project.

wait_for_run_close

dbnl.wait_for_run_close(*, run: Run, timeout_s: float = 180.0, polling_interval_s: float = 3.0) → Run

Wait for a Run to close. Polls every polling_interval_s seconds until it is closed.

Parameters:
- run – Run to wait for
- timeout_s – Total wait time (in seconds) for Run to close, defaults to 180.0
- polling_interval_s – Time between polls (in seconds), defaults to 3.0
Raises:
- DBNLNotLoggedInError – dbnl SDK is not logged in
- DBNLError – Run did not close after waiting for the timeout_s seconds
- DBNLRunError – Run ended in ERRORED state

Previousdbnl.experimental Nextdbnl.util

Was this helpful?

hashtagclose_run

hashtagIMPORTANT

hashtagcopy_project

hashtagExamples:

hashtagcreate_metric

hashtagcreate_project

hashtagExamples:

hashtagcreate_run

hashtagcreate_run_query

hashtagExamples:

hashtagcreate_run_schema

hashtagSupported Types

hashtagComponents

hashtagExamples:

hashtagcreate_run_schema_from_results

hashtagExamples:

hashtagcreate_test_session

hashtagIMPORTANT

hashtagManaging Tags

hashtagExamples:

hashtagdelete_metric

hashtagexport_project_as_json

hashtagSample Project JSON

hashtagExamples:

hashtagget_column_results

hashtagIMPORTANT

hashtagExamples:

hashtagget_latest_run

hashtagget_metric_by_id

hashtagget_my_namespaces

hashtagget_or_create_project

hashtagExamples:

hashtagget_project

hashtagExamples:

hashtagget_results

hashtagIMPORTANT

hashtagExamples:

hashtagget_run

hashtagExamples:

hashtagget_run_query

hashtagExamples:

hashtagget_scalar_results

hashtagIMPORTANT

hashtagExamples:

hashtagimport_project_from_json

hashtagExamples:

hashtaglogin

hashtagreport_column_results

hashtagIMPORTANT

hashtagWARNING

hashtagExamples:

hashtagreport_results

hashtagIMPORTANT

hashtagWARNING

hashtagExamples:

hashtagreport_run_with_results

hashtagIMPORTANT

hashtagExamples:

hashtagreport_run_with_results_and_start_test_session

hashtagIMPORTANT

hashtagExamples:

hashtagreport_scalar_results

hashtagIMPORTANT

hashtagWARNING

hashtagExamples:

hashtagset_run_as_baseline

hashtagset_run_query_as_baseline

hashtagwait_for_run_close

close_run

IMPORTANT

copy_project

Examples:

create_metric

create_project

Examples:

create_run

create_run_query

Examples:

create_run_schema

Supported Types

Components

Examples:

create_run_schema_from_results

Examples:

create_test_session

IMPORTANT

Managing Tags

Examples:

delete_metric

export_project_as_json

Sample Project JSON

Examples:

get_column_results

IMPORTANT

Examples:

get_latest_run

get_metric_by_id

get_my_namespaces

get_or_create_project

Examples:

get_project

Examples:

get_results

IMPORTANT

Examples:

get_run

Examples:

get_run_query

Examples:

get_scalar_results

IMPORTANT

Examples:

import_project_from_json

Examples:

login

report_column_results

IMPORTANT

WARNING

Examples:

report_results

IMPORTANT

WARNING

Examples:

report_run_with_results

IMPORTANT

Examples:

report_run_with_results_and_start_test_session

IMPORTANT

Examples:

report_scalar_results

IMPORTANT

WARNING

Examples:

set_run_as_baseline

set_run_query_as_baseline

wait_for_run_close