LogoLogo
AboutBlogLaunch app ↗
v0.20.x
v0.20.x
  • Introduction to AI Testing
  • Welcome to Distributional
  • Motivation
  • What is AI Testing?
  • Stages in the AI Software Development Lifecycle
    • Components of AI Testing
  • Distributional Testing
  • Getting Access to Distributional
  • Learning about Distributional
    • The Distributional Framework
    • Defining Tests in Distributional
      • Automated Production test creation & execution
      • Knowledge-based test creation
      • Comprehensive testing with Distributional
    • Reviewing Test Sessions and Runs in Distributional
      • Reviewing and recalibrating automated Production tests
      • Insights surfaced elsewhere on Distributional
      • Notifications
    • Data in Distributional
      • The flow of data
      • Components and the DAG for root cause analysis
      • Uploading data to Distributional
      • Living in your VPC
  • Using Distributional
    • Getting Started
    • Access
      • Organization and Namespaces
      • Users and Permissions
      • Tokens
    • Data
      • Data Objects
      • Run-Level Data
      • Data Storage Integrations
      • Data Access Controls
    • Testing
      • Creating Tests
        • Test Page
        • Test Drawer Through Shortcuts
        • Test Templates
        • SDK
      • Defining Assertions
      • Production Testing
        • Auto-Test Generation
        • Recalibration
        • Notable Results
        • Dynamic Baseline
      • Testing Strategies
        • Test That a Given Distribution Has Certain Properties
        • Test That Distributions Have the Same Statistics
        • Test That Columns Are Similarly Distributed
        • Test That Specific Results Have Matching Behavior
        • Test That Distributions Are Not the Same
      • Executing Tests
        • Manually Running Tests Via UI
        • Executing Tests Via SDK
      • Reviewing Tests
      • Using Filters
        • Filters in the Compare Page
        • Filters in Tests
    • Python SDK
      • Quick Start
      • Functions
        • login
        • Project
          • create_project
          • copy_project
          • export_project_as_json
          • get_project
          • get_or_create_project
          • import_project_from_json
        • Run Config
          • create_run_config
          • get_latest_run_config
          • get_run_config
          • get_run_config_from_latest_run
        • Run Results
          • get_column_results
          • get_scalar_results
          • get_results
          • report_column_results
          • report_scalar_results
          • report_results
        • Run
          • close_run
          • create_run
          • get_run
          • report_run_with_results
        • Baseline
          • create_run_query
          • get_run_query
          • set_run_as_baseline
          • set_run_query_as_baseline
        • Test Session
          • create_test_session
      • Objects
        • Project
        • RunConfig
        • Run
        • RunQuery
        • TestSession
        • TestRecalibrationSession
        • TestGenerationSession
        • ResultData
      • Experimental Functions
        • create_test
        • get_tests
        • get_test_sessions
        • wait_for_test_session
        • get_or_create_tag
        • prepare_incomplete_test_spec_payload
        • create_test_recalibration_session
        • wait_for_test_recalibration_session
        • create_test_generation_session
        • wait_for_test_generation_session
      • Eval Module
        • Quick Start
        • Application Metric Sets
        • How-To / FAQ
        • LLM-as-judge and Embedding Metrics
        • RAG / Question Answer Example
        • Eval Module Functions
          • Index of functions
          • eval
          • eval.metrics
    • Notifications
    • Release Notes
  • Tutorials
    • Instructions
    • Hello World (Sentiment Classifier)
    • Trading Strategy
    • LLM Text Summarization
      • Setting the Scene
      • Prompt Engineering
      • Integration testing for text summarization
      • Practical considerations
Powered by GitBook

© 2025 Distributional, Inc. All Rights Reserved.

On this page
  • Feb 18, 2025 - Version 0.20.0
  • New Features
  • Improvements
  • Bug Fixes
  • New SDK Version
  • Jan 13, 2025 - Version 0.19.0

Was this helpful?

Export as PDF
  1. Using Distributional

Release Notes

PreviousNotificationsNextInstructions

Was this helpful?

Feb 18, 2025 - Version 0.20.0

This release adds a number of new features, improvements, and bug fixes.

New Features

  • New Test History feature in Test Spec detail page that enables users to understand a single tests' behavior over time, including its recalibration history

  • Added support for in UI

Improvements

  • Close runs in the UI

  • Viewable dbnl version in sidebar

  • UI performance improvements

  • Update color of links

  • Preserve input expressions in test spec editor

  • Extend scope of Project export

    • Export Tags by name for Project export

    • Better error messages for Project import

  • Improve namespace support for multi-org users

  • Miscellaneous package updates

  • Validate results on run close

  • /projects redirects to home page

Bug Fixes

  • Fix broken pagination

  • Fix broken Histogram title

  • Fix Results Table rendering issue for some Tests

  • Fix support for decimal values in assertion params

  • Fix rendering of = and != Assertions

  • Test spec editor navigation bugfix

New SDK Version

  • Check compatibility with API version

  • Add support for double and long values

  • Improved errors for invalid API URL configuration

  • Remove en-core-web-sm from requirements to enable PyPI support

  • Updated helm-charts for on-prem

Jan 13, 2025 - Version 0.19.0

Highlights in this version:

  • Improvements to the project import/export feature including support for notifications.

  • Support for new versioning and release process.

  • Better dependency management.

  • Too many bug fixes and UX improvements to list in detail.

Slack notifications