> For the complete documentation index, see [llms.txt](https://docs.dbnl.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.dbnl.com/v0.26.x/configuration/model-connections.md).
# Model Connections
### Why does DBNL require a Model Connection?
Model Connectors are how DBNL interfaces with LLMs, which is required for each step of the [DBNL Data Pipeline](/v0.26.x/configuration/data-pipeline.md) to function. It enables DBNL to
* Compute LLM-as-judge Metrics as part of the enrich step.
* Perform certain unsupervised analytics processes as part of the analysis step.
* Translate surfaced behavioral signals into human readable [Insights](/v0.26.x/workflow/insights.md) as part of the publish step.
### Types of Model Connections
Fundamentally a Model Connection needs to be able to expose a LLM chat completion interface that is accessible by your DBNL deployment. It can be
* An externally managed service (e.g. together.ai, OpenAI, etc)
* A cloud managed service that is part of your VPC (e.g. Bedrock, Vertex, etc)
* A locally managed deployment (e.g. a cluster of NVIDIA NIMs running in your DBNL k8s cluster as part of your deployment)
There are pros and cons to each of these approaches:
| Model Connector Type | Pros | Cons |
| ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
Externally managed service (together.ai, OpenAI, etc)
|
Fast and easy to set up (just provide keys)
Model and scaling flexibility
|
Requires sending data outside of your cloud environment
Higher cost, on demand model
|
|
Cloud managed service (Bedrock, Vertex, etc)
|
Data stays within your cloud provider
Often managed by another team within the organization
|
Can be higher cost than locally running models
Usage, rate limits are typically shared across organization
|
|
Locally managed deployment (NVIDIA NIMs in k8s cluster)
|
Data stays within your local deployment
Cheaper than a managed service
Maximum control of cost vs timing tradeoffs
|
Requires access to GPU resources
Can require local admin and debugging
|
### Creating a Model Connection
Model Connections are defined at the [Namespace](/v0.26.x/platform/administration.md#namespaces) level of an [Organization](/v0.26.x/platform/administration.md#organizations) and can be used by any [Projects](/v0.26.x/workflow/projects.md) within the Namespace. For connivence, a new Model Connection can be created as part of the [Project Creation](/v0.26.x/workflow/projects.md#creating-a-project) flow as well.
A Model Connection has the following attributes:
* **Name** (required): How the Model Connection is referenced when setting a default Model Connection for a project or LLM-as-judge [Metric](/v0.26.x/workflow/metrics.md).
* **Description** (optional): Human readable description of the connection for reference.
* **Model** (required): The model name to be used as part of the API call (e.g. `gpt-3.5-turbo`, `gemini-2.0-flash-001`, etc). See the documentation for your model provider for more details.
* **Provider** (required): One of
* [AWS Bedrock](https://aws.amazon.com/bedrock): Managed AWS service for foundation models.
* [AWS Sagemaker](https://aws.amazon.com/sagemaker): Platform to build, train, and deploy machine learning models by AWS (not recommended for production DBNL deployments).
* [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-foundry/models/openai): Microsoft service providing OpenAI models via Azure cloud.
* [Google Gemini](https://gemini.google.com/): Google’s AI model for chat, code, and reasoning.
* [Google Vertex AI](https://cloud.google.com/vertex-ai): Managed GCP service for building and deploying models.
* [NVIDIA NIM](https://developer.nvidia.com/nim): NVIDIA microservices for deploying optimized AI models easily.
* [OpenAI](https://openai.com/api/): Managed service for advanced language and reasoning models
* OpenAI-compatible: Any provider that exposes an "OpenAI-like" API, like [together.ai](https://www.together.ai/)
* **Configuration Parameters** (required): Depending on the provider selected, you may need to provide additional required information like Access Key IDs, Secret Access Keys, preferred regions, endpoints/URLs, etc.
### Editing a Model Connection
A Model Connection can be edited or deleted by clicking on the "Model Connections" tab on the sidebar of the [Namespace](/v0.26.x/platform/administration.md#namespaces) landing page.
### Debugging a Model Connection
A Model Connection can be tested by navigating to the specific Model Connection as above and clicking on the "Validate" button. This will send a simple request to the endpoint and inform you if it was able to complete the request.
### Next Steps
* **Ready to send data to your project?** Start ingesting data into your project using your defined [Data Connection](/v0.26.x/configuration/data-connections.md) to kick off the [Adaptive Analytics Workflow](/v0.26.x/workflow/adaptive-analytics-workflow.md).
* **Want to understand more about the platform?** Check out the [Architecture](/v0.26.x/platform/architecture.md), [Deployment](/v0.26.x/platform/deployment.md) options, and other aspects of the [Platform](/v0.26.x/platform/deployment.md).