> For the complete documentation index, see [llms.txt](https://docs.dbnl.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.dbnl.com/v0.21.x/using-distributional/testing/testing-strategies/test-that-distributions-are-not-the-same.md).

# Test That Distributions Are Not the Same

Another testing case is testing whether two distributions are not the same. Such a test involves the same statistics as a test of consistency, but a different assertion. One example could be to change the assertion from `close_to` to `greater_than` and thereby state that a passed test requires a difference bigger than some threshold.

<details>

<summary>Example Test Spec</summary>

```json
{
    "name": "test_lower_toxicity_score",
    "description": "Test mean of the toxicity score is lower than baseline",
    "statistic_name": "diff_mean",
    "statistic_params": {},
    "assertion": {
        "name": "less_than",
        "params": {
            "other": -0.1,
        },
    },
    "statistic_inputs": [
        {
            "select_query_template": {
                "select": "{EXPERIMENT}.toxicity_score"
            }
        },
        {
            "select_query_template": {
                "select": "{BASELINE}.toxicity_score"
            }
        },
    ],
}
```

</details>

<figure><img src="/files/nUOgN4LZWKyfUvssdk1M" alt=""><figcaption><p>Example Test on signed difference of mean of toxicity_score</p></figcaption></figure>