LLM Text Summarization
In this advanced tutorial, we demonstrate how to use dbnl to automatically evaluate the consistency of summarization output on a fixed set of documents.
Was this helpful?
v0.23.x
In this advanced tutorial, we demonstrate how to use dbnl to automatically evaluate the consistency of summarization output on a fixed set of documents.
Was this helpful?
The data files required for this tutorial are available in the following files.
While summarization is the focus of this tutorial, the same principles can be applied to any task involving text generation. The goal is to evaluate the consistency of the generated text with the input text. Other tasks involving text generation are entity recognition, question answering, and machine translation.
This tutorial assumes that you have already the following tutorials: Quickstart and ideally Trading Strategy.
This tutorial requires a good deal of preparation, so it has been divided into the following four sections:
Defining the text summarization problem of interest, including the data source and the metrics,
Creating a constrained optimization problem to govern the development of a text summarization app in dbnl,
Managing the integration testing process for consistent testing after such an app has been created,
Practical considerations which would arise when actually building an LLM summarization tool.