How to use ValidMind Library features

Published

June 2, 2026

Browse our range of Jupyter Notebooks demonstrating how to use the core features of the ValidMind Library. Use these how-to notebooks to get familiar with the library's capabilities and apply them to your own use cases.

How-to by topic

Configure judge LLM and judge embeddings

This notebook shows how to configure and validate the default judge LLM and judge embeddings used by the ValidMind Library for LLM-focused tests.

Customize test result descriptions

When you run ValidMind tests, test descriptions are automatically generated with LLM using the test results, the test name, and the static test definitions provided in the test's docstring. While this metadata offers valuable high-level overviews of tests, insights produced by the LLM-based descriptions may not always align with your specific use…

Dataset Column Filters when Running Tests

To run a test on a dataset but only include certain columns from that dataset, you can use the new columns option. This is done by passing a dictionary for the dataset input for the test instead of the dataset object or dataset input ID directly. This dictionary should have the following keys:

Document multiple results for the same test

Documentation templates facilitate the presentation of multiple unique test results for a single test.

Enable PII detection in tests

Learn how to enable and configure Personally Identifiable Information (PII) detection when running tests with the ValidMind Library. Choose whether or not to include PII in test descriptions generated, or whether or not to include PII in test results logged to the ValidMind Platform.

Explore test suites

Explore ValidMind test suites, pre-built collections of related tests used to evaluate specific aspects of your model. Retrieve available test suites and details for tests within a suite to understand their functionality, allowing you to select the appropriate test suites for your use cases.

Explore tests

Explore the individual out-the-box tests available in the ValidMind Library, and identify which tests to run to evaluate different aspects of your model. Browse available tests, view their descriptions, and filter by tags or task type to find tests relevant to your use case.

Implement custom tests

Custom tests extend the functionality of ValidMind, allowing you to document any model or use case with added flexibility.

Integrate external test providers

Register a custom test provider with the ValidMind Library to run your own tests.

Run comparison tests

Learn how to use the ValidMind Library to run comparison tests that take any datasets or records (models) as inputs. Identify comparison tests to run, initialize ValidMind dataset and model objects in preparation for passing them to tests, and then run tests — generating outputs automatically logged to your documentation in the ValidMind Platform.

Run dataset-based tests

Learn how to use the ValidMind Library to run tests that take any dataset or record (model) as input. Identify specific tests to run, initialize ValidMind dataset objects in preparation for passing them to your tests, and then run the chosen tests — generating outputs that can be automatically logged to your documentation in the ValidMind Platform.

Run documentation tests with custom configurations

When running documentation tests, you can configure inputs and parameters for individual tests by passing a config as a parameter.

Run individual documentation sections

For targeted testing, you can run tests on individual sections or specific groups of sections in your model documentation.

Run tests with multiple datasets

To support running tests that require more than one dataset, ValidMind provides a mechanim that allows you to pass multiple datasets as inputs.

Understand and utilize `RawData` in ValidMind tests

Test functions in ValidMind can return a special object called RawData, which holds intermediate or unprocessed data produced somewhere in the test logic but not returned as part of the test's visible output, such as in tables or figures.

Configure dataset features

When initializing a ValidMind dataset object, you can pass in a list of features to use instead of utilizing all dataset columns when running tests.

Introduction to ValidMind Dataset and Model Objects

When writing custom tests, it is essential to be aware of the interfaces of the ValidMind Dataset and ValidMind Model, which are used as input arguments.

Load dataset predictions

To enable tests to make use of predictions, you can load predictions in ValidMind dataset objects in multiple different ways.

Intro to Unit Metrics

To turn complex evidence into actionable insights, you can run a unit metric as a single-value measure to quantify and monitor risks throughout a model's lifecycle.

Log metrics over time

Learn how to track and visualize the temporal evolution of key record (model) performance metrics with ValidMind.

Generate qualitative text with the ValidMind library

This notebook shows how to generate qualitative documentation content directly from the ValidMind library using both vm.run_text_generation() and vm.generate_documentation_text(). Instead of switching to the UI to write text manually or trigger generation one section at a time, you can generate content for documentation text blocks…

Intro to Assign Scores

The assign_scores() method is a powerful feature that allows you to compute and add scorer scores as new columns in your dataset. This method takes a model and metric(s) as input, computes the specified metrics from the ValidMind scorer library, and adds them as new columns. The computed metrics provide per-row values, giving you granular…