dbnl.experimental

create_test

dbnl.experimental.create_test(*, test_spec_dict: TestSpecDict) → dict[str, Any]

Create a new Test Spec

• Parameters:test_spec_dict – A dictionary containing the Test Spec schema.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in.

  • DBNLAPIValidationError – Test Spec does not conform to expected format.

  • DBNLDuplicateError – Test Spec with the same name already exists in the Project.

• Returns: The JSON dict of the created Test Spec object. The return JSON will contain the id of the Test Spec.

Test Spec JSON Structure

{
    "project_id": string,

    # Test data
    "name": string, // must be unique to Project
    "description": string | null,
    "statistic_name": string,
    "statistic_params": map[string, any],
    "statistic_inputs": list[
        {
            "select_query_template": {
                "select": string, // a column or a function on column(s)
                "filter": string | null
            }
        }
    ],
    "assertion": {
        "name": string,
        "params": map[string, any]
    },
    "tag_ids": string[] | null
}

create_test_generation_session

dbnl.experimental.create_test_generation_session(*, run: Run, columns: list[str | dict[Literal['name'], str]] | None = None) → TestGenerationSession

Create a Test Generation Session

• Parameters:

  • run – The Run to use when generating tests.

  • columns – List of columns in the Run to generate tests for. If None, all columns in the Run will be used, defaults to None. If a list of strings, each string is a column name. If a list of dictionaries, each dictionary must have a ‘name’ key, and the value is the column name.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in.

  • DBNLInputValidationError – arguments do not conform to expected format.

• Returns: The TestGenerationSession that was created.

Examples:

import dbnl
dbnl.login()


run = dbnl.get_run(run_id="run_0000000")
dbnl.experimental.create_test_generation_session(
    run=run,
    columns=["col1", "col4"],
)

create_test_recalibration_session

dbnl.experimental.create_test_recalibration_session(*, test_session: TestSession, feedback: str, test_ids: list[str] | None = None) → TestRecalibrationSession

Create a Test Recalibration Session by redefining the expected output for tests in a Test Session

• Parameters:

  • test_session – Test Session to recalibrate

  • feedback – Feedback for the recalibration. Can be ‘PASS’ or ‘FAIL’.

  • test_ids – List of test IDs to recalibrate, defaults to None. If None, all tests in the Test Session will be recalibrated.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in.

  • DBNLInputValidationError – arguments do not conform to expected format.

• Returns: Test Recalibration Session

IMPORTANT

If some generated Tests failed when they should have passed and some passed when they should have failed, you will need to submit 2 separate calls, one for each feedback result.

get_or_create_tag

dbnl.experimental.get_or_create_tag(*, project_id: str, name: str, description: str | None = None) → dict[str, Any]

Get the specified Test Tag or create a new one if it does not exist

• Parameters:

  • project_id – The id of the Project that this Test Tag is associated with.

  • name – The name of the Test Tag to create or retrieve.

  • description – An optional description of the Test Tag. Limited to 255 characters.

• Returns: The dictionary containing the Test Tag

• Raises:DBNLNotLoggedInError – dbnl SDK is not logged in.

Sample Test Tag JSON

{
    # Tag metadata
    "id": string,
    "org_id": string,
    "created_at": timestamp,
    "updated_at": timestamp,

    # Tag data
    "name": string,
    "author_id": string,
    "description": string?,
    "project_id": string,
}

get_test_sessions

dbnl.experimental.get_test_sessions(*, project: Project) → list[TestSession]

Get all Test Sessions in the given Project

• Parameters:project – Project from which to retrieve Test Sessions

• Returns: List of Test Sessions

• Raises:DBNLNotLoggedInError – dbnl SDK is not logged in.

get_tests

dbnl.experimental.get_tests(*, test_session_id: str) → list[dict[str, Any]]

Get all Tests executed in the given Test Session

• Parameters:test_session_id – Test Session ID

• Returns: List of test JSONs

• Raises:DBNLNotLoggedInError – dbnl SDK is not logged in.

Sample Test JSON

{
    # Test metadata
    "id": string,
    "org_id": string,
    "created_at": timestamp,
    "updated_at": timestamp,
    "test_session_id": string,

    # Test data
    "author_id": string,
    "value": any?,
    "failure": string?,
    "status": enum(PENDING, RUNNING, PASSED, FAILED),
    "started_at": timestamp?,
    "completed_at": timestamp?,

    # Test Spec data
    "test_spec_id": id,
    "name": string,
    "description": string?,
    "statistic_name": string,
    "statistic_params": map[string, any],
    "assertion": {
        "name": string,
        "params": map[string, any]
        "status": enum(...),
        "failure": string?
    },
    "statistic_inputs": list[
        {
        "select_query_template": {
            "select": string
        }
        }
    ],
    "tag_ids": string[]?,
    }

prepare_incomplete_test_spec_payload

dbnl.experimental.prepare_incomplete_test_spec_payload(*, test_spec_dict: IncompleteTestSpecDict, project_id: str | None = None) → TestSpecDict

Formats a Test Spec payload for the API. Add project_id if it is not present. Replace tag_names with tag_ids.

• Parameters:

  • test_spec_dict – A dictionary containing the Test Spec schema.

  • project_id – The Project ID, defaults to None. If project_id does not exist in test_spec_dict, it is required as an argument.

• Raises:DBNLInputValidationError – Input does not conform to expected format

• Returns: The dictionary containing the newly formatted Test Spec payload.

wait_for_test_generation_session

dbnl.experimental.wait_for_test_generation_session(*, test_generation_session: TestGenerationSession, timeout_s: int = 180) → TestGenerationSession

Wait for a Test Generation Session to finish. Polls every 3 seconds until it is completed.

• Parameters:

  • test_generation_session – The TestGenerationSession to wait for.

  • timeout_s – The total wait time (in seconds) for Test Generation Session to complete, defaults to 180.

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in.

  • DBNLError – Test Generation Session did not complete after waiting for the timeout_s seconds

• Returns: The completed TestGenerationSession

wait_for_test_recalibration_session

dbnl.experimental.wait_for_test_recalibration_session(*, test_recalibration_session: TestRecalibrationSession, timeout_s: int = 180) → TestRecalibrationSession

Wait for a Test Recalibration Session to finish. Polls every 3 seconds until it is completed.

• Parameters:

  • test_recalibration_session – The TestRecalibrationSession to wait for.

  • timeout_s – The total wait time (in seconds) for Test Recalibration Session to complete, defaults to 180.

• Returns: The completed TestRecalibrationSession

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in.

  • DBNLError – Test Recalibration Session did not complete after waiting for the timeout_s seconds

wait_for_test_session

dbnl.experimental.wait_for_test_session(*, test_session: TestSession, timeout_s: int = 180) → TestSession

Wait for a Test Session to finish. Polls every 3 seconds until it is completed.

• Parameters:

  • test_session – The TestSession to wait for

  • timeout_s – The total wait time (in seconds) for Test Session to complete, defaults to 180.

• Returns: The completed TestSession

• Raises:

  • DBNLNotLoggedInError – dbnl SDK is not logged in.

  • DBNLError – Test Session did not complete after waiting for the timeout_s seconds