Dashboards

Discover signals by viewing tracked Columns, Segments, and Metrics.

Dashboards are collections of histograms, time series and statistics of monitored Columns, tracked Segments, and generated Metrics for user-driven analysis.

There are three default dashboards for each Project:

• Monitoring Dashboard: Distributional recommended graphs and statistics built from required Columns and data from the DBNL Semantic Convention

• Segments Dashboard: Count graphs and statistics for all tracked Segments

• Metrics Dashboard: Histograms, time series and statistics of generated Metrics

Monitoring Dashboard

Default dashboard displaying recommended graphs and statistics for a specific time window (default: last 7 days) such as:

• Log Count: Time series of number of logs ingested per day in the time range.

• Total Logs: Total number of logs ingested from the time range.

• Total Cost: Total estimated cost for the underlying AI product being analyzed during the time range. Requires total_cost to be reported as part of log ingestion.

• Total Token Count Distribution: Histogram of token counts for all logs in the time range. Requires total_token_count to be reported as part of log ingestion.

• Total Token Count over time: Time series of total token count per day ingested in the time range. Requires total_token_count to be reported as part of log ingestion.

• Token Count Summary: Statistics for the Token Count column for the time range. Requires total_token_count to be reported as part of log ingestion. Included statistics are:

  • Max: The highest total_token_count observed over the time range.

  • P95: The 95th percentile of total_token_count observed over the time range.

  • Mean: The average total_token_count observed over the time range.

  • P5: The 5th percentile of total_token_count observed over the time range.

  • Min: The minimum total_token_count observed over the time range.

  • Sum: The sum of all total_token_count observed over the time range.

• User Frustration Distribution: Histogram of the default user_frustration Metric for all logs in the time range.

• User Frustration over time: Time series of the daily default user_frustration Metric for in the time range.

• User Frustration Summary: Statistics for the default user_frustration Metric for the time range.

  • Max: The highest of the default user_frustration Metric observed over the time range.

  • Mean: The average of the default user_frustration Metric observed over the time range.

  • Median: The 50th percentile of the default user_frustration Metric observed over the time range.

  • Min: The minimum of the default user_frustration Metric observed over the time range.

• Answer Relevancy Distribution: Histogram of the default answer_relevancy Metric for all logs in the time range.

Segments Dashboard

Dashboard displaying all tracked Segments as time series of daily counts for each Segment within a specific time range (default: last 7 days). Count histograms can be displayed as:

• Ratio: The percentage of total logs for each day in the time range that are within the Segment.

• Count: The total number of logs for each day in the time range that are within the Segment.

Metrics Dashboard

Dashboard displaying all custom Metrics as histograms, time series, and statistics summaries for all logs within a specific time range (default: last 7 days).

• Distribution: The histogram of observed values for the Metric during the time range.

• Daily Trend: The time series of the average and 95th percentile of the Metric for each day in the time range.

• Statistics Summary: Statistics for the Metric in the time range consisting of:

  • Max: The maximum observed value for the Metric in the time range.

  • P95: The 95th percentile of the observed values of the Metric in the time range.

  • Mean: The average of the observed values of the Metric in the time range.

  • Median: The 50th percentile of the observed values of the Metric in the time range.

  • P5: The 5th percentile of the observed values of the Metric in the time range.

  • Min: The minimum observed value for the Metric in the time range.