Glossary
Key terms and concepts in DBNL
Adaptive Analytics
The core mechanism for discovering, investigating, and tracking hidden behavioral signals from production AI log data. Adaptive Analytics continuously analyzes and updates the definition of "normal" behavior as new data becomes available, enabling deeper insights over time.
Related Terms: Adaptive Analytics Flywheel, Behavioral Signals
Learn More: Adaptive Analytics Workflow, Overview
Adaptive Analytics Flywheel
The continuous 8-step cycle that powers DBNL's analysis: Ingest → Enrich → Analyze → Publish → Discover → Investigate → Track → Repeat. This flywheel adapts to previously tracked signals, providing deeper and more customized analytics over time.
Related Terms: Data Pipeline, Workflow
Learn More: Overview, Adaptive Analytics Workflow
Analyze
The third step of the Data Pipeline where unsupervised learning and statistical techniques are applied to the distributional fingerprint to discover Insights such as behavioral changes, clusters, and outliers.
Related Terms: Data Pipeline, Insights, Unsupervised Learning
Learn More: Data Pipeline, Overview
Answer Relevancy
A default LLM-as-Judge Metric that determines if the AI's output is relevant to the user's input. One of the core metrics computed automatically for every project.
Related Terms: Default Metrics, LLM-as-Judge Metrics
Learn More: Metrics, LLM-as-Judge Templates
Behavioral Fingerprint
A statistical profile representing the expected behavior of an AI application, derived from distributions of historical data for each attribute. Also called a Distributional Fingerprint, it serves as a baseline to detect deviations and changes over time.
Related Terms: Behavioral Signals, Model Drift
Learn More: FAQ, Adaptive Analytics
Behavioral Signals
Key insights or patterns extracted from AI production data that indicate specific behaviors. Signals can highlight daily shifts, clusters of similar behaviors, or outliers that deviate from the norm.
Related Terms: Insights, Adaptive Analytics, Behavioral Fingerprint
Classifier Metric
A type of LLM-as-Judge Metric that outputs a categorical value equal to one of a predefined set of classes. Example: llm_answer_groundedness outputs grounded or not_grounded.
Related Terms: LLM-as-Judge Metrics, Scorer Metric
Learn More: Metrics, LLM-as-Judge Templates
Columns
Data fields extracted from logs and flattened according to the DBNL Semantic Convention. Required columns are: input, output, and timestamp.
Related Terms: DBNL Semantic Convention, Logs
Learn More: Data Pipeline, DBNL Semantic Convention
Dashboards
Collections of histograms, time series, and statistics of monitored Columns, tracked Segments, and generated Metrics for user-driven analysis. DBNL includes three default dashboards: Monitoring, Segments, and Metrics.
Related Terms: Metrics Dashboard, Segments Dashboard, Monitoring Dashboard
Learn More: Dashboards
Data Connections
The method by which production AI log data is ingested into DBNL, kickstarting the Data Pipeline. Options include OTEL Trace Ingestion, SDK Log Ingestion, and SQL Integration Ingestion.
Related Terms: Data Pipeline, Ingest
Learn More: Data Connections
Data Pipeline
The process that converts raw production AI log data into actionable insights and dashboards. Consists of four key steps: Ingest, Enrich, Analyze, and Publish.
Related Terms: Adaptive Analytics Flywheel, Pipeline Run
Learn More: Data Pipeline, Status
DBNL Semantic Convention
A mapping from well-known formats into types and names that DBNL recognizes. Enables automatic and consistent data interpretation across different ingestion methods, including standard fields like input, output, timestamp, model, total_token_count, and total_cost.
Related Terms: Columns, Data Connections
Learn More: DBNL Semantic Convention
Default Metrics
Built-in metrics computed automatically for every project using the required input and output fields and the default Model Connection. Includes answer_relevancy, user_frustration, topic, conversation_summary, and summary_embedding.
Related Terms: Metrics, LLM-as-Judge Metrics
Learn More: Metrics
Deployment
A complete DBNL installation in a user's infrastructure, whether cloud VPC, on-premise, or sandbox environment. DBNL can be deployed using the Sandbox, Helm Chart, or Terraform Module.
Related Terms: Sandbox, Organization
Learn More: Deployment, Architecture
Embeddings
Vector representations of text (like conversation summaries) used for semantic analysis and clustering. DBNL generates summary_embedding as a default immutable metric for topic generation.
Related Terms: Topic Classification, Default Metrics
Learn More: Metrics
Enrich
The second step of the Data Pipeline where data is augmented with LLM-as-Judge, NLP, and other behavioral Metrics to create rich behavioral information vectors for every log.
Related Terms: Data Pipeline, Metrics, Model Connections
Learn More: Data Pipeline, Overview
Explorer
A tool for rapid analysis and triage of Segments by performing graphical and statistical comparison between different subsets of Logs over time windows and/or filters. Supports Single Segment, Segment Comparison, and Temporal Comparison views.
Related Terms: Segment Comparison, Temporal Comparison
Learn More: Explorer
Ingest
The first step of the Data Pipeline where raw production log data is flattened into Columns using the DBNL Semantic Convention.
Related Terms: Data Pipeline, Data Connections
Learn More: Data Pipeline, Overview
Insights
Human-readable explanations and quantifications of Behavioral Signals generated from unsupervised analysis of enriched logs. Can be investigated through the Explorer and tracked as Metrics or Segments. Three types: Temporal Insights, Segment Insights, and Outlier Insights.
Related Terms: Behavioral Signals, Analyze
Learn More: Insights
LLM-as-Judge Metrics
Evaluations that require an LLM to compute a score or classification based on a prompt. Includes Scorer Metrics (output 1-5) and Classifier Metrics (output predefined categories). Used for semantic understanding like relevance, tone, quality, and groundedness.
Related Terms: Metrics, Model Connections, Standard Metrics
Learn More: Metrics, LLM-as-Judge Templates
Logs
Individual records from production AI applications, displayed with filterable Columns and Metrics. Can be viewed in Detail, Trace, or Session views.
Related Terms: Columns, Session, Trace
Learn More: Logs
Metrics
A mapping from Columns into meaningful numeric values representing cost, quality, performance, or behavioral characteristics. Computed for every log as part of the Data Pipeline. Two main types: LLM-as-Judge Metrics and Standard Metrics.
Related Terms: LLM-as-Judge Metrics, Standard Metrics, Default Metrics
Learn More: Metrics
Metrics Dashboard
Dashboard displaying all custom Metrics as histograms (distribution), time series (daily trends), and statistics summaries for all logs within a specific time range.
Related Terms: Dashboards, Metrics
Learn More: Dashboards
Model Connections
How DBNL interfaces with LLMs for computing LLM-as-Judge Metrics, performing unsupervised analytics, and translating signals into human-readable Insights. Supports providers like AWS Bedrock, Azure OpenAI, Google Vertex AI, OpenAI, and NVIDIA NIM.
Related Terms: LLM-as-Judge Metrics, Enrich
Learn More: Model Connections
Model Drift
When AI behavior deviates significantly from the established Behavioral Fingerprint. DBNL detects drift through temporal analysis and alerts users to changes before they cause impact.
Related Terms: Behavioral Fingerprint, Temporal Insights
Monitoring Dashboard
Default dashboard displaying recommended graphs and statistics for a specific time window, including log counts, token usage, costs, and default metrics like user_frustration and answer_relevancy.
Related Terms: Dashboards, Default Metrics
Learn More: Dashboards
Namespace
A unit of isolation within an Organization containing Projects, Data Connections, Model Connections, and Notification Connections. Enables multi-tenancy and access control.
Related Terms: Organization, Projects
Learn More: Administration
Notification Connections
Integration channels (Email, Slack, PagerDuty) that inform users when specific DBNL actions are completed, such as data runs finishing or new Insights being generated.
Related Terms: Projects, Insights
Learn More: Notification Connections
Organization
A DBNL Deployment containing all Namespaces and users for a single organization. The top-level entity in DBNL's hierarchy.
Related Terms: Namespace, Deployment, Users
Learn More: Administration
OTEL Trace Ingestion
Publish OpenTelemetry (OTEL) traces directly to DBNL as the product runs. Enables the richest data with full trace inspection through Spans but doesn't support backfilling historical data.
Related Terms: Data Connections, Spans, Trace
Learn More: OTEL Trace Ingestion
Outlier Insights
Specific instances or sets of logs that deviate significantly from expected behavior related to one or more Metrics. Represents one of three types of Insights.
Related Terms: Insights, Metrics
Learn More: Insights
Pipeline Run
An execution of the complete Data Pipeline for a specific date range, including Ingest, Enrich, Analyze, and Publish steps. Can be monitored and restarted from the Status page.
Related Terms: Data Pipeline, Status
Learn More: Status, Data Pipeline
Projects
The main organizational tool in DBNL; typically one project per AI application to analyze. Contains Data Connections, Model Connections, Logs, Metrics, Segments, and Insights.
Related Terms: Namespace, Data Pipeline
Learn More: Projects
Publish
The fourth step of the Data Pipeline where Dashboards are updated and new Insights are generated to represent newly observed and discovered behavior from the latest production data.
Related Terms: Data Pipeline, Insights, Dashboards
Learn More: Data Pipeline, Overview
Query Language
DBNL's language for creating Standard Metrics using functions like word_count, flesch_kincaid_grade, levenshtein, contains, and more. Enables fast, deterministic calculations without requiring an LLM.
Related Terms: Standard Metrics, Query Functions
Learn More: Query Language, Functions
Query Functions
Built-in functions available in the Query Language for creating Standard Metrics. Includes text analysis (word_count, character_count), readability scores (flesch_kincaid_grade), string operations (contains, levenshtein), and more.
Related Terms: Query Language, Standard Metrics
Learn More: Functions
Roles
Permission levels assigned to Users in DBNL. Options include Organization Admin (full access), Namespace Admin (manage specific namespaces), and Namespace Writer (create/edit within namespaces).
Related Terms: Users, Namespace, Organization
Learn More: Administration
Sandbox
A self-contained Docker container that bundles all DBNL services and dependencies for local testing and development. Not suitable for production but ideal for POCs and learning DBNL.
Related Terms: Deployment
Learn More: Sandbox, Quickstart
Scorer Metric
A type of LLM-as-Judge Metric that outputs an integer in the range [1, 2, 3, 4, 5]. Example: llm_text_frustration scores user frustration from 1 (not frustrated) to 5 (very frustrated).
Related Terms: LLM-as-Judge Metrics, Classifier Metric
Learn More: Metrics, LLM-as-Judge Templates
SDK Log Ingestion
Push data manually or as part of a daily orchestration job using the DBNL Python SDK. The most flexible ingestion method but requires code and external scheduling.
Related Terms: Data Connections, Python SDK
Learn More: SDK Log Ingestion, Python SDK
Segment Comparison
An Explorer view that compares two different filters on Logs across the same time window. Allows comparison of Metrics between segments or between a segment and the rest of the log data.
Related Terms: Explorer, Segments, Temporal Comparison
Learn More: Explorer
Segment Insights
Detected clusters related to filters on Columns that correspond to unique behavior patterns. Bifurcates log data based on specific conditions. One of three types of Insights.
Related Terms: Insights, Segments
Learn More: Insights
Segments
Saved filters on log data corresponding to specific Behavioral Signals. Automatically computed and published to the Segments Dashboard; inform and adapt future analytics.
Related Terms: Behavioral Signals, Segment Insights
Learn More: Segments
Segments Dashboard
Dashboard displaying all tracked Segments as time series of daily counts (or ratios) for each segment within a specific time range.
Related Terms: Dashboards, Segments
Learn More: Dashboards
Session
A group of related logs identified by session_id. Allows viewing all associated logs for a given session together with their Metrics in Session View.
Learn More: Logs, DBNL Semantic Convention
Spans
Individual trace segments with timing and latency information, including attributes, events, and status. Used in OTEL Trace Ingestion to provide detailed execution visibility.
Related Terms: OTEL Trace Ingestion, Trace
Learn More: DBNL Semantic Convention, Logs
SQL Integration Ingestion
Pull data from a SQL table (BigQuery, Databricks, Snowflake, Redshift) into DBNL on a schedule. Leverages existing data infrastructure with no code required but needs pre-flattened data.
Related Terms: Data Connections
Learn More: SQL Integration Ingestion
Standard Metrics
Functions that can be computed using non-LLM methods like NLP metrics, statistical operations, and Query Language Functions. Faster and cheaper than LLM-as-Judge Metrics.
Related Terms: Metrics, Query Language, LLM-as-Judge Metrics
Learn More: Metrics, Query Language
Status
The Status page shows all ongoing and previous Data Pipeline runs for a project, including current status, errors, and the ability to restart failed runs. Displays expected pipeline duration based on log volume.
Related Terms: Data Pipeline, Pipeline Run
Learn More: Status
Temporal Comparison
An Explorer view that compares a single filter across two adjacent time windows. Allows before/after Metric comparison for a given Segment.
Related Terms: Explorer, Temporal Insights, Segment Comparison
Learn More: Explorer
Temporal Insights
Detected changes or shifts in behavior related to one or more Columns over time, defined by a time split showing "before" and "after" within a time window. One of three types of Insights.
Related Terms: Insights, Temporal Comparison
Learn More: Insights
Topic Classification
A default LLM-as-Judge Metric that classifies conversations into topics based on input and output. Topics are automatically generated after 7 days of ingested data and can be manually adjusted.
Related Terms: Default Metrics, Classifier Metric
Learn More: Metrics, Topic Template
Trace
A waterfall view of latency and timing for individual Spans in a request. Only available if spans data is provided through OTEL Trace Ingestion.
Related Terms: Spans, OTEL Trace Ingestion, Logs
Learn More: Logs, OTEL Trace Ingestion
Unsupervised Learning
Automated machine learning techniques applied to enriched data to discover behavioral patterns without labeled training data. Used in the Analyze step of the Data Pipeline to generate Insights.
Related Terms: Analyze, Insights, Behavioral Signals
Learn More: Data Pipeline, FAQ
User Frustration
A default LLM-as-Judge Metric (Scorer Metric) that assesses the level of frustration in user input based on tone, word choice, and other properties. Scored from 1-5.
Related Terms: Default Metrics, Scorer Metric
Learn More: Metrics, User Frustration Template
Users
Individuals with login credentials to an Organization, defined by Roles and Namespace permissions. Can be authenticated via username/password or OIDC.
Related Terms: Organization, Roles, Namespace
Learn More: Administration, Authentication
Python SDK
The DBNL Python SDK for programmatically interacting with the platform, including data ingestion, project management, and metric creation. Installed via pip install dbnl.
Related Terms: SDK Log Ingestion, CLI
Learn More: Python SDK, SDK Log Ingestion
CLI
The DBNL Command Line Interface for interacting with the platform from the command line. Primarily used for authentication and managing the Sandbox deployment. Installed alongside the Python SDK.
Related Terms: Python SDK, Sandbox
Learn More: CLI
Was this helpful?