create_test_session

Create a TestSession

dbnl.create_test_session(
    *,
    experiment_run: Run,
    baseline: Optional[Union[Run, RunQuery]] = None,
    include_tags: Optional[List[str]] = None,
    exclude_tags: Optional[List[str]] = None,
    require_tags: Optional[List[str]] = None,
) -> :

Start evaluating Tests associated with a Run. Typically, the Run you just completed will be the "Experiment" and you'll compare it to some earlier "Baseline Run".

The Run must already have Results reported and be closed before a Test Session can begin.

A Run must be closed for all uploaded results to be shown on the UI.

Parameters

Arguments	Description
experiment_run	The dbnl Run to create the TestSession for.
baseline	The dbnl Run or RunQuery to compare the experiment_runagainst. If None(or omitted), will use the baselinedefined in the TestConfig
include_tags	An optional list of Test Tags to be included. All Tests with any of the tags in this list will be ran after the run is complete.
exclude_tags	An optional list of Test Tags to be excluded. All Tests with any of the tags in this list will be skipped after the run is complete.
require_tags	An optional list of Test Tags that are required. Only tests with all the tags in this list will be ran after the run is complete.

Managing Tags

Suppose we have the following Tests with the associated Tags in our Project

• Test1 with tags ["A", "B"]

• Test2 with tags ["A"]

• Test3 with tags ["B"]

dbnl.create_test_session(..., include_tags=["A", "B"]) will trigger Tests 1, 2, 3 to be executed.

dbnl.create_test_session(..., require_tags=["A", "B"]) will only trigger Test 1.

dbnl.create_test_session(..., exclude_tags=["A"]) will trigger Test 3.

dbnl.create_test_session(..., include_tags=["A"], exclude_tags=["B"]) will trigger Test 2.

Examples

Basic example

dbnl.create_test_session(
  experiment_run=new_run,
  baseline=baseline_run,
)

Using a Run Query as a Baseline

dbnl.create_test_session(
  experiment_run=new_run,
  baseline=baseline_run_query,
)

When Baseline Run has already been set

dbnl.create_test_session(
  experiment_run=new_run,
)