# Model Connections
### Why does DBNL require a Model Connection?
Model Connectors are how DBNL interfaces with LLMs, which is required for each step of the [DBNL Data Pipeline](/v0.26.x/configuration/data-pipeline.md) to function. It enables DBNL to
* Compute LLM-as-judge Metrics as part of the enrich step.
* Perform certain unsupervised analytics processes as part of the analysis step.
* Translate surfaced behavioral signals into human readable [Insights](/v0.26.x/workflow/insights.md) as part of the publish step.
### Types of Model Connections
Fundamentally a Model Connection needs to be able to expose a LLM chat completion interface that is accessible by your DBNL deployment. It can be
* An externally managed service (e.g. together.ai, OpenAI, etc)
* A cloud managed service that is part of your VPC (e.g. Bedrock, Vertex, etc)
* A locally managed deployment (e.g. a cluster of NVIDIA NIMs running in your DBNL k8s cluster as part of your deployment)
There are pros and cons to each of these approaches:
| Model Connector Type | Pros | Cons |
| ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
Externally managed service (together.ai, OpenAI, etc)
|
Fast and easy to set up (just provide keys)
Model and scaling flexibility
|
Requires sending data outside of your cloud environment
Higher cost, on demand model
|
|
Cloud managed service (Bedrock, Vertex, etc)
|
Data stays within your cloud provider
Often managed by another team within the organization
|
Can be higher cost than locally running models
Usage, rate limits are typically shared across organization
|
|
Locally managed deployment (NVIDIA NIMs in k8s cluster)
|
Data stays within your local deployment
Cheaper than a managed service
Maximum control of cost vs timing tradeoffs
|
Requires access to GPU resources
Can require local admin and debugging
|
### Creating a Model Connection
Model Connections are defined at the [Namespace](/v0.26.x/platform/administration.md#namespaces) level of an [Organization](/v0.26.x/platform/administration.md#organizations) and can be used by any [Projects](/v0.26.x/workflow/projects.md) within the Namespace. For connivence, a new Model Connection can be created as part of the [Project Creation](/v0.26.x/workflow/projects.md#creating-a-project) flow as well.
A Model Connection has the following attributes:
* **Name** (required): How the Model Connection is referenced when setting a default Model Connection for a project or LLM-as-judge [Metric](/v0.26.x/workflow/metrics.md).
* **Description** (optional): Human readable description of the connection for reference.
* **Model** (required): The model name to be used as part of the API call (e.g. `gpt-3.5-turbo`, `gemini-2.0-flash-001`, etc). See the documentation for your model provider for more details.
* **Provider** (required): One of
* [AWS Bedrock](https://aws.amazon.com/bedrock): Managed AWS service for foundation models.
* [AWS Sagemaker](https://aws.amazon.com/sagemaker): Platform to build, train, and deploy machine learning models by AWS (not recommended for production DBNL deployments).
* [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-foundry/models/openai): Microsoft service providing OpenAI models via Azure cloud.
* [Google Gemini](https://gemini.google.com/): Google’s AI model for chat, code, and reasoning.
* [Google Vertex AI](https://cloud.google.com/vertex-ai): Managed GCP service for building and deploying models.
* [NVIDIA NIM](https://developer.nvidia.com/nim): NVIDIA microservices for deploying optimized AI models easily.
* [OpenAI](https://openai.com/api/): Managed service for advanced language and reasoning models
* OpenAI-compatible: Any provider that exposes an "OpenAI-like" API, like [together.ai](https://www.together.ai/)
* **Configuration Parameters** (required): Depending on the provider selected, you may need to provide additional required information like Access Key IDs, Secret Access Keys, preferred regions, endpoints/URLs, etc.
### Editing a Model Connection
A Model Connection can be edited or deleted by clicking on the "Model Connections" tab on the sidebar of the [Namespace](/v0.26.x/platform/administration.md#namespaces) landing page.
### Debugging a Model Connection
A Model Connection can be tested by navigating to the specific Model Connection as above and clicking on the "Validate" button. This will send a simple request to the endpoint and inform you if it was able to complete the request.
### Next Steps
* **Ready to send data to your project?** Start ingesting data into your project using your defined [Data Connection](/v0.26.x/configuration/data-connections.md) to kick off the [Adaptive Analytics Workflow](/v0.26.x/workflow/adaptive-analytics-workflow.md).
* **Want to understand more about the platform?** Check out the [Architecture](/v0.26.x/platform/architecture.md), [Deployment](/v0.26.x/platform/deployment.md) options, and other aspects of the [Platform](/v0.26.x/platform/deployment.md).
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.dbnl.com/v0.26.x/configuration/model-connections.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.