Welcome to Distributional
Introduction to Distributional AI Testing Platform
Was this helpful?
v0.20.x
Introduction to Distributional AI Testing Platform
Was this helpful?
At Distributional, we make AI testing easy, so you can build with confidence. Here’s how it works:
✅ Connect to your existing data sources.
🔄 Run automated tests on a regular schedule.
📢 Get alerts when your AI application needs your attention.
Simple, seamless, and built for peace of mind. Let's help you improve your AI uptime.
When do you need AI Testing? To get a sense of what testing could look like in practice, here are some questions that you can answer through AI Testing across the AI Software Development Lifecycle:
How well do my evals map to production behavior?
When do I increase coverage in my golden dataset to address edge cases?
If something changes in one component of my application, how do I assess the cascading impact to other dependent components?
Are end-users catching issues that my evals miss?
How do I compare new application updates to prior ones in my production environment?
What’s the impact to behavior if the model, data, or usage shifts?
Do I know when shifts happen?
How do I understand what’s causing anomalous behavior?
What happens if I switch to another LLM?
How do I give other teams visibility into shifts that happen?
What’s the impact of pushing any change to my application? Am I able to push changes to production or is it a new dev cycle?
If you are interested in finding the answers to the above, Distributional can help. The Distributional platform provides a standardized approach to AI testing across all of your applications.
Ready to start using Distributional? Head straight to Getting Started to get set up on the platform and start testing your AI application.
If you would first like to learn more about the Distributional platform, head over to the Learning About Distributional section. If you are new to AI Testing or would like to know how it fits in to the AI Software Development Cycle, continue forward in the Introduction to AI Testing section, starting with the Motivation and What is AI Testing pages.
