1 of 33

Functions

Click through to view all the SDK functions.

login

Authenticate dbnl SDK

dbnl.login(
    *,
    api_token: Optional[str] = None,
    namespace_id: Optional[str] = None,    
    api_url: Optional[str] = None,
    app_url: Optional[str] = None,  
) -> None

Setup dbnl SDK to make authenticated requests. After login is run successfully, the dbnl client will be able to issue secure and authenticated requests against hosted endpoints of the dbnl service.

dbnl.login must be run before any other functions in the DBNL workflow

Parameters

Arguments

Description

api_token

namespace_id

Namespace ID to use for the session; available namespaces can be found with get_my_namespaces().

api_url

The base url of the Distributional API. For SaaS users, set this variable to api.dbnl.com. For other users, please contact your sys admin.

app_url

An optional base url of the Distributional app. If this variable is not set, the app url is inferred from the DBNL_API_URL variable. For on-prem users, please contact your sys admin if you cannot reach the Distributional UI.

Examples

import dbnl
# when login() is called without specifying a token, 
# it will use the `DBNL_API_TOKEN` env var
dbnl.login()

# login() can be called with a specific API Token
dbnl.login(api_token="YOUR_TOKEN_AAAA_BBBB_CCCC_DDDD")

Project

Functions that interact with a dbnl

create_project

Create a new dbnl Project

dbnl.create_project(
    *,
    name: str,
    description: Optional[str] = None,
) -> :

Parameters

Arguments

Description

Returns

Type

Description

Examples

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")

# DBNLConflictingProjectError: A DBNL Project with name test_p1 already exists.
proj_2 = dbnl.create_project(name="test_p1")

copy_project

Copy a dbnl Project with a new name and description

dbnl.copy_project(
    *,
    project: ,
    name: str,
    description: Optional[str] = None,
) -> :

Parameters

Arguments

Description

project

name

description

An optional description for the dbnl Project, defaults to None. Description is limited to 255 characters.

Returns

Type

Description

The newly created dbnl Project.

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_proj1")
proj2 = dbnl.copy_project(project=proj1, name="test_proj2")

assert proj2.name == "test_proj2"

export_project_as_json

Export a dbnl Project alongside its Test Specs and Tags as a JSON object

dbnl.export_project_as_json(
    *,
    project: ,
) -> dict[str, Any]:

Parameters

Arguments

Description

project

Returns

Type

Description

dict[str, Any]

JSON object representing the Project. Example:

{
    "project": {
        "name": "My Project",
        "description": "This is my project."
    },
    "tags": [
        {
            "name": "my-tag",
            "description" :"This is my tag."
        }
    ],
    "test_specs": [
        {
            "assertion": { "name": "less_than", "params": { "other": 0.5 } },
            "description": "Testing the difference in the example statistic",
            "name": "Gr.0: Non Parametric Difference: Example_Statistic",
            "statistic_inputs": [
                {
                    "select_query_template": {
                        "filter": null,
                        "select": "{EXPERIMENT}.Example_Statistic"
                    }
                },
                {
                    "select_query_template": {
                        "filter": null,
                        "select": "{BASELINE}.Example_Statistic"
                    }
                }
            ],
            "statistic_name": "my_stat",
            "statistic_params": {},
            "tag_names": ["my-tag"]
        }
    ]
}

Examples

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_proj")
export_json = dbnl.export_project_as_json(project=proj)

assert export_json["project"]["name"] == "test_proj"

get_project

Retrieve a dbnl Project

dbnl.get_project(
    *,
    name: str,
) -> :

Parameters

Arguments

Description

name

Returns

Type

Description

The dbnl Project with the given name.

Examples

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")
proj_2 = dbnl.get_project(name="test_p1")

# Calling get_project will yield same Project object
assert proj_1.id == proj_2.id

# DBNLProjectNotFoundError: A dnnl Project with name not_exist does not exist
proj_3 = dbnl.get_project(name="not_exist")

get_or_create_project

Retrieve the specified dbnl Project or create a new one if it does not exist

dbnl.get_or_create_project(
    *,
    name: str,
    description: Optional[str] = None,
) -> :

Parameters

Arguments

Description

name

description

An optional description for the dbnl Project, defaults to None. Description is limited to 255 characters.

Description cannot be updated with this function.

Returns

Type

Description

A new Project will be created with the specified name if there does not exist a Project with this name already. If there does exist a project with the name, the pre-existing Project will be returned.

Examples

import dbnl
dbnl.login()


proj_1 = dbnl.create_project(name="test_p1")
proj_2 = dbnl.get_or_create_project(name="test_p1")

# Calling get_or_create_project will yield same Project object
assert proj_1.id == proj_2.id

import_project_from_json

Create a new dbnl Project from a JSON object

dbnl.import_project_from_json(
    *,
    params: dict[str, Any],
) -> :

Parameters

Arguments

Description

Returns

Type

Description

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_proj1")
export_json = dbnl.export_project_as_json(project=proj1)
export_json["project"]["name"] = "test_proj2"
proj2 = dbnl.import_project_from_json(params=export_json)

assert proj2.name == "test_proj2"

Run Config

Functions related to dbnl

create_run_config

Create a new dbnl RunConfig

dbnl.create_run_config(
    *,
    project: ,
    columns: list[dict[str, Any]],
    scalars: Optional[list[dict[str, Any]]] = None,
    description: Optional[str] = None,
    display_name: Optional[str] = None,
    row_id: Optional[list[str]] = None,
    components_dag: Optional[dict[str, list[str]]] = None,
) -> :

Parameters

Arguments

Description

Column Schema

Column Names

Column names can only be alphanumeric characters and underscores.

Supported Types

The following type supported as type in column schema

Components

Components DAG

The components_dag dictionary specifies the topological layout of the AI/ML app. For each key-value pair, the key represents the source component, and the value is a list of the leaf components. The following code snippet describes the DAG shown above.

components_dags={
    "TweetSource": ["EntityExtractor", "SentimentClassifier"],
    "EntityExtractor": ["TradeRecommender"],
    "SentimentClassifier": ["TradeRecommender"],
    "TradeRecommender": [],
    "Global": [],
}

Returns

Examples

Basic Usage

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
# create a new RunConfig
runcfg1 = dbnl.create_run_config(
    project=proj,
    columns=[
        {"name": "error_type", "type": "category"},
        {"name": "email", "type": "string", "description": "raw email text content from source"},
        {"name": "spam-pred", "type": "boolean"},
    ],
    display_name="Basic RunConfig for spam prediction",
)

RunConfig with DAG

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
# create a new RunConfig with DAG
runcfg1 = dbnl.create_run_config(
    project=proj,
    columns=[
        {"name": "error_type", "type": "category"},
        {"name": "email", "type": "string", "component": "data_source", "description": "raw email text content from source"},
        {"name": "spam-pred", "type": "boolean", "component": "spam_classifier"},
    ],
    display_name="Basic RunConfig for spam prediction",
    components_dag={
        "data_source": ["spam_classifier"]
        "spam_classifier": []
)

RunConfig with row_id

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
# create a new RunConfig
runcfg1 = dbnl.create_run_config(
    project=proj,
    columns=[
        {"name": "error_type", "type": "category"},
        {"name": "email", "type": "string", "description": "raw email text content from source"},
        {"name": "spam-pred", "type": "boolean"},
        {"name": "email_id", "type": "string", "description": "unique id for each email"},
    ],
    display_name="Basic RunConfig for spam prediction",
    row_id=["email_id"],
)

RunConfig with scalars

import dbnl
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
# create a new RunConfig
runcfg1 = dbnl.create_run_config(
    project=proj,
    columns=[
        {"name": "error_type", "type": "category"},
        {"name": "email", "type": "string", "description": "raw email text content from source"},
        {"name": "spam-pred", "type": "boolean"},
        {"name": "email_id", "type": "string", "description": "unique id for each email"},
    ],
    scalars=[
        {"name": "model_F1", "type": "float"},
        {"name": "model_recall", "type": "float"},
    ],
    display_name="Basic RunConfig for spam prediction",
)

get_latest_run_config

Retrieve the most recent dbnl RunConfig

dbnl.get_latest_run_config(
    *,
    project: ,
) -> :

Parameters

Arguments

Description

project

Returns

Type

Description

The dbnl RunConfig most recently created in the Project.

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])

# Retrieving the latest RunConfig
runcfg2 = dbnl.get_latest_run_config(project=proj1)
assert runcfg1.id == runcfg2.id

get_run_config

Retrieve a dbnl RunConfig

dbnl.get_run_config(
    *,
    run_config_id: str,
) -> :

Parameters

Arguments

Description

Returns

Type

Description

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])

# Retrieving the RunConfig by ID
runcfg2 = dbnl.get_run_config(run_config_id=runcfg1.id)
assert runcfg1.id == runcfg2.id

# DBNLRunConfigNotFoundError: A DBNL RunConfig with id not_exist does not exist
run_config3 = dbnl.get_run_config(run_config_id="runcfg_not_exist")

get_run_config_from_latest_run

Retrieve a dbnl RunConfig from the most recent Run in a Project

dbnl.get_run_config(
    *,
    project: ,
) -> :

Parameters

Arguments

Description

project

Returns

Type

Description

The dbnl RunConfig from the most recent run in the Project.

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(
    project=proj1, 
    run_config=runcfg1, 
)
# Retrieving the RunConfig by ID
runcfg2 = dbnl.get_run_config_from_latest_run(project=proj1)
assert runcfg1.id == runcfg2.id

Run Results

Functions related to Column and Scalar data uploaded within a Run.

As a convenience for reporting results and creating a Run, you can also check out report_run_with_results

get_column_results

Retrieve results from dbnl

dbnl.get_column_results(
    *,
    run: ,
) -> pandas.DataFrame:

Parameters

Arguments

Description

run

Returns

Type

Description

pandas.DataFrame

You can only call get_column_results after the run is closed.

Examples

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
uploaded_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=test_data,
)

downloaded_data = dbnl.get_column_results(run=run)
assert downloaded_data.equals(uploaded_data)

get_scalar_results

Retrieve results from dbnl

dbnl.get_scalar_results(
    *,
    run: ,
) -> pandas.DataFrame:

Parameters

Arguments

Description

run

Returns

Type

Description

pandas.DataFrame

You can only call get_scalar_results after the run is closed.

Examples

import dbnl
import pandas as pd
dbnl.login()

proj1 = dbnl.get_or_create_project(name="test_p1")

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=data,
    scalar_results={"rmse": 0.37}
)

downloaded_scalars = dbnl.get_scalar_results(run=run)

get_results

Retrieve results from dbnl

dbnl.get_results(
    *,
    run: ,
) -> ResultData:

Parameters

Arguments

Description

run

Returns

Type

Description

ResultData

You can only call get_results after the run is closed.

Examples

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")

uploaded_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
run = dbnl.report_run_with_results(
    project=proj,
    column_results=uploaded_data,
)

downloaded_data = dbnl.get_results(run=run)
assert downloaded_data.columns.equals(uploaded_data)

report_column_results

Report all column results to dbnl

dbnl.report_results(
    *,
    run: ,
    data: ,
) -> None:

Parameters

Arguments

Description

run

data

Limitations

All data should be reported to dbnl at once. Calling dbnl.report_column_results more than once will overwrite the previously uploaded data.

Once a Run is closed. You can no longer call report_column_results to send data to dbnl.

Examples

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_config=runcfg1)

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_column_results(run=run1, data=data)

report_scalar_results

Report all scalar results to dbnl

dbnl.report_scalar_results(
    *,
    run: ,
    Union[dict[str, Any], pd.DataFrame]
) -> None:

Parameters

Arguments

Description

Limitations

All data should be reported to dbnl at once. Calling dbnl.report_scalar_results more than once will overwrite the previously uploaded data.

Once a Run is . You can no longer call report_scalar_results to send data to DBNL.

Examples

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(
    project=proj1, 
    columns=[{"name": "error", "type": "float"}],
    scalars=[{"name": "rmse": "type": "float"}],
)
run1 = dbnl.create_run(project=proj1, run_config=runcfg1)
dbnl.report_scalar_results(run=run1, data={"rmse": 0.37})

report_results

Report all results to dbnl

dbnl.report_results(
    *,
    run: ,
    column_data: ,
    scalar_data: dict[str, Any] | pandas.DataFrame | None = None
) -> None:

Parameters

Arguments

Description

run

column_data

scalar_data

report_results is the equivalent of calling both report_column_results and report_scalar_results .

Limitations

All data should be reported to dbnl at once. Calling dbnl.report_results more than once will overwrite the previously uploaded data.

Once a Run is closed. You can no longer call report_results to send data to DBNL.

Examples

import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_config=runcfg1)

data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_results(run=run1, column_data=data)


import dbnl
import pandas as pd
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(
    project=proj1, 
    columns=[{"name": "error", "type": "float"}],
    scalars=[{"name": "rmse": "type": "float"}],
)
run1 = dbnl.create_run(project=proj1, run_config=runcfg1)
data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})
dbnl.report_results(run=run1, column_data=data, scalar_data={"rmse": 0.37})

Run

Functions interacting with dbnl Run

close_run

Finalize a Run

dbnl.close_run(
    *,
    run: ,
) -> None:

Mark the specified dbnl Run status as completed. Once a Run is marked as closed, it can no longer be used for .

A Run must be closed for all to be shown on the UI.

Parameters

Arguments

Description

Examples

import dbnl
dbnl.login()

dbnl.close_run(run=my_run)

create_run

Create a new dbnl Run

dbnl.create_run(
    *,
    project: ,
    run_config: ,
    display_name: Optional[str] = None,
    metadata: Optional[Dict[str, str]] = None,
) -> :

Parameters

Arguments

Description

project

run_config

display_name

An optional display name for the Run. Display names do not have to be unique.

metadata

Any additional key-value pairs information the user wants to track.

Returns

Type

Description

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])

run1 = dbnl.create_run(
    project=proj1, 
    run_config=runcfg1, 
    metadata={"mode": "dev"},
)

get_run

Retrieve a dbnl Run

dbnl.get_run(
    *,
    run_id: str,
) -> :

Parameters

Arguments

Description

run_id

Returns

Type

Description

The dbnl Run with the given ID.

Examples

import dbnl
dbnl.login()


proj1 = dbnl.get_or_create_project(name="test_p1")
runcfg1 = dbnl.create_run_config(project=proj1, columns=[{"name": "error", "type": "float"}])
run1 = dbnl.create_run(project=proj1, run_config=runcfg1)

# Retrieving the Run by ID
run2 = dbnl.get_run(run_id=run1.id)
assert run1.id == run2.id

# DBNLRunNotFoundError: A DBNL Run with id run_0000000 does not exist.
run3 = dbnl.get_run(run_id="run_0000000")

report_run_with_results

Create a new Run, report results to it, and close it.

dbnl.report_run_with_results(
    project: ,
    column_data: pd.DataFrame,
    scalar_data: Optional[Union[dict[str, Any], pd.DataFrame]] = None
    display_name: Optional[str] = None,
    row_id: Optional[list[str]] = None,
    run_config_id: Optional[str] = None,
    metadata: Optional[dict[str, str]] = None,
) -> Run:

Parameters

Arguments

Description

project

column_data

scalar_data

display_name

An optional display name for the Run. Display names do not have to be unique.

row_id

An optional list of the column names that can be used as unique identifiers.

run_config_id

ID of the RunConfig to use for the Run, defaults to None. If provided, the RunConfig is used as is and the results are validated against it. If not provided, a new Run Config is inferred from the column_data.

metadata

Any additional key-value pairs information the user wants to track.

Returns

Type

Description

The closed Run with the uploaded data.

Examples

import dbnl
import pandas as pd
dbnl.login()


proj = dbnl.get_or_create_project(name="test_p1")
test_data = pd.DataFrame({"error": [0.11, 0.33, 0.52, 0.24]})

run = dbnl.report_run_with_results(
    project=proj,
    column_data=test_data,
    row_id=["idx"],
)

Baseline

Functions that interact with dbnl Baseline concept

create_run_query

dbnl.create_run_query(
    *,
    project: Project,
    name: str,
    query: dict[str, Any]
) -> RunQuery:

Parameters

Arguments

Description

project

name

Descriptive name for this Run Query. Must be unique at the Project level

query

dict describing how to find a Run dynamically. Currently, only supports "offset_from_now": int as a key-value pair.

Returns

Type

Description

Example

dbnl.create_run_query(
  project=project,
  name="look back 3",
  query={
    "offset_from_now": 3,
  },
)

get_run_query

Retrieve a dbnl RunQuery with the given name

dbnl.get_run_query(
    project: Project,
    name: str,
) -> RunQuery:

Parameters

Arguments

Description

project

The dbnl this is associated with

name

Name of the Run Query.

Returns

Type

Description

The dbnl RunQuery, typically used for finding a for a Test Session

Example

query = dbnl.get_run_query(
  project=project,
  name="look back 3"
)

set_run_as_baseline

Set a given Run as the Baseline Run in a Project's Test Config

dbnl.set_run_as_baseline(
    *,
    run: ,
) -> None:

Parameters

Arguments

Description

set_run_query_as_baseline

Set a given RunQuery as the Baseline Run in a Project's Test Config

dbnl.set_run_query_as_baseline(
    *,
    run_query: ,
) -> None:

Parameters

Arguments

Description

Test Session

Functions that interact with dbnl

create_test_session

Create a TestSession

dbnl.create_test_session(
    *,
    experiment_run: Run,
    baseline: Optional[Union[Run, RunQuery]] = None,
    include_tags: Optional[List[str]] = None,
    exclude_tags: Optional[List[str]] = None,
    require_tags: Optional[List[str]] = None,
) -> :

Start evaluating Tests associated with a Run. Typically, the Run you just completed will be the "Experiment" and you'll compare it to some earlier "Baseline Run".

The Run must already have and be closed before a Test Session can begin.

A Run must be closed for all to be shown on the UI.

Parameters

Arguments

Description

Managing Tags

Suppose we have the following Tests with the associated Tags in our Project

Test1 with tags ["A", "B"]
Test2 with tags ["A"]
Test3 with tags ["B"]

dbnl.create_test_session(..., include_tags=["A", "B"]) will trigger Tests 1, 2, 3 to be executed.

dbnl.create_test_session(..., require_tags=["A", "B"]) will only trigger Test 1.

dbnl.create_test_session(..., exclude_tags=["A"]) will trigger Test 3.

dbnl.create_test_session(..., include_tags=["A"], exclude_tags=["B"]) will trigger Test 2.

Examples

Basic example

dbnl.create_test_session(
  experiment_run=new_run,
  baseline=baseline_run,
)

Using a Run Query as a Baseline

dbnl.create_test_session(
  experiment_run=new_run,
  baseline=baseline_run_query,
)

When Baseline Run has already been set

dbnl.create_test_session(
  experiment_run=new_run,
)