> For the complete documentation index, see [llms.txt](https://docs.dbnl.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/application-metric-sets.md).

# Application Metric Sets

The metric set helpers return an adaptive list of metrics, relevant to the application type

### `text_metrics()`

Basic metrics for generic text comparison and monitoring

* [`token_count`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.token_count-text_col_name-str-metric)
* [`word_count`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.word_count-text_col_name-str-metric)
* [`flesch_kincaid_grade`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.flesch_kincaid_grade-text_col_name-str-metric)
* [`automated_readability_index`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.automated_readability_index-text_col_name-str-metric)
* [`bleu`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.bleu-prediction-str-target-str-metric)
* [`levenshtein`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.levenshtein-prediction-str-target-str-metric)
* [`rouge1`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.rouge1-prediction-str-target-str-score_type-rougescoretype-rougescoretype.fmeasure)
* [`rouge2`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.rouge2-prediction-str-target-str-score_type-rougescoretype-rougescoretype.fmeasure)
* [`rougeL`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.rougel-prediction-str-target-str-score_type-rougescoretype-rougescoretype.fmeasure)
* [`rougeLsum`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.rougelsum-prediction-str-target-str-score_type-rougescoretype-rougescoretype.fmeas)
* [`llm_text_toxicity_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_viability_llm_text_toxicity-prediction-str-eval_llm_client-llmclient-metric)
* [`llm_sentiment_assessment_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_viability_llm_sentiment_assessment-prediction-str-eval_llm_client-llmclient)
* [`llm_reading_complexity_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_viability_llm_reading_complexity-prediction-str-eval_llm_client-llmclient-m)
* [`llm_grammar_accuracy_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_quality_llm_grammar_accuracy-prediction-str-eval_llm_client-llmclient-metri)
* [`inner_product`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.inner_product_target_prediction-prediction-str-target-str-eval_embedding_client-em)
* [`llm_text_similarity_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.quality_llm_text_similarity-prediction-str-target-str-eval_llm_client-llmclient-me)

### `question_and_answer_metrics()`

Basic metrics for RAG / question answering

* [`llm_accuracy_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_quality_llm_accuracy-input-str-context-str-prediction-str-eval_llm_client-l)
* [`llm_completeness_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_quality_llm_completeness-input-str-prediction-str-eval_llm_client-llmclient)
* [`answer_similarity_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_quality_llm_answer_similarity-input-str-prediction-str-target-str-eval_llm)
* [`faithfulness_v0`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.answer_quality_llm_faithfulness-input-str-context-str-prediction-str-eval_llm_clie)
* [`mrr`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.mrr-ground_truth_document_id-str-retrieved_document_ids-str-metric)
* [`context_hit`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.context_hit-ground_truth_document_id-str-retrieved_document_ids-str-metric)

The metric set helpers are adaptive in that :

1. The metrics returned encode which columns of the dataframe are input to the metric computation\
   e.g.,  `rougeL_prediction__ground_truth` is the `rougeL` metric run with both the column named `prediction` and the column named `ground_truth` as input
2. The metrics returned support any additional optional column info and LLM-as-judge or embedding model clients. If any of this optional info is not provided, the metric set will exclude any metrics that depend on that information

```python
def text_metrics(
    prediction: str,
    target: Optional[str] = None,
    eval_llm_client: Optional[LLMClient] = None,
    eval_embedding_client: Optional[EmbeddingClient] = None,
) -> list[Metric]:
    """
    Returns a set of metrics relevant for a generic text application

    :param prediction: prediction column name (i.e. generated text)
    :param target: target column name (i.e. expected text)
    :return: list of metrics
    """
```

See the [How-To section](/v0.21.x/using-distributional/python-sdk/eval-module/how-to-faq.md) for concrete examples of adaptive [`text_metrics()`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.text_metrics-prediction-str-target-str-or-none-none-eval_llm_client-llmclient-or-n) usage

See the [RAG example](/v0.21.x/using-distributional/python-sdk/eval-module/rag-question-answer-example.md) for [`question_and_answer_metrics()`](https://docs.dbnl.com/v0.21.x/using-distributional/python-sdk/eval-module/pages/AnAiwh9fVe2BWEPoFKmz#dbnl.eval.metrics.question_and_answer_metrics-prediction-str-target-str-or-none-none-input-str-or-no) usage