ValidMind Library releases

Latest: v2.10.0

Published

October 16, 2025

Add new `deepeval` tests in library

validmind-library

2.10.0

enhancement

This update enhances the integration between ValidMind and DeepEval by introducing primary scorers within a dedicated deepeval namespace. You can now evaluate large language model (LLM) outputs using standardized metrics. Primary scorers are essential tools for evaluating LLM outputs. We have added several DeepEval-based LLM scorers, including:

Add support for the DeepEval dataset in LLM tests

validmind-library

2.9.5

documentation

enhancement

highlight

This update enhances the integration between DeepEval and the ValidMind library by adding support for a new dataset type specific to large language models (LLMs). You can now use various LLM tests from the DeepEval library. We have introduced new row-level metrics that return arrays.

Removed legacy notebooks

validmind-library

2.9.5

documentation

enhancement

This update removes multiple legacy agent demo notebooks and associated code samples that were previously used to demonstrate the integration of LangChain, LangGraph, and banking-specific agent functionality with the ValidMind Library.

Enable custom test description output structure

validmind-library

2.9.5

enhancement

This update offers enhanced customization for LLM-generated test descriptions in the ValidMind Library. You can now manage test descriptions using a new context parameter in run_test(). This parameter accepts a dictionary with three optional keys:

Support Python 3.12 and optimize dependencies in ValidMind Library

validmind-library

2.9.3

enhancement

This update introduces support for Python 3.12 in the ValidMind Library. We’ve optimized dependency management to make the core library lighter and ensure easy installation across various Python environments.

Enable PII detection in tests with new notebook

validmind-library

2.9.3

documentation

enhancement

Learn how to enable and configure Personally Identifiable Information (PII) detection when running tests with the ValidMind Library using the new enable_pii_detection.ipynb notebook. This update replaces the deprecated configure_pii_detection.ipynb notebook. You can choose to include PII in test descriptions or in the test results logged to the ValidMind platform.

Enhance `list_tests()` to show artifact types in output

validmind-library

2.8.29

enhancement

We have enhanced the list_tests() function to include additional columns in its output. These columns now display the types of artifacts each test in the ValidMind Library produces, providing clearer insights into test results. Additionally, you now benefit from explicit return type annotations and updated function signatures with appropriate type hints, ensuring better clarity and precision in the codebase.

Enhance `predict_fn` in `init_model` to support multiple output columns

validmind-library

2.8.29

documentation

This update enhances the predict_fn callable function parameter in init_model to support multiple output columns. You can now return either a single value or a dictionary from the function. If a dictionary is returned, the key named prediction will be used as the prediction column. Additional keys in the dictionary can be included as extra columns in the dataset object.

Add section to inject custom context via docstring

validmind-library

2.8.26

documentation

enhancement

This update enhances the Jupyter notebook add_context_to_llm_descriptions.ipynb by introducing a new section. This section guides you on embedding explicit instructions within a test’s docstring to influence LLM-generated test result descriptions. Titled Add test-specific context using the docstring, it addresses the issue where non-instructional context in docstrings was previously ignored by the LLM. You are now advised to format specific lines as instructions using syntax like: INSTRUCTION…

Interfaces to support code explainer feature in ValidMind

validmind-library

2.8.26

documentation

enhancement

highlight

This update introduces an experimental feature for text generation tasks within the ValidMind project. It includes interfaces to utilize the code_explainer LLM feature, currently in the experimental namespace to gather feedback.

Support qualitative text generation in `run_task` function

validmind-library

2.8.26

enhancement

This update enhances the run_task function in the validmind/experimental/agents.py file to support a new task type. You can now generate qualitative text for a specific section of a document using the vm.experimental.agents.run_task method. The generated content will be displayed in the model_overview text block section.

Update quickstart for model documentation in Jupyter notebooks

validmind-library

2.8.26

enhancement

We’ve improved our Jupyter notebooks to make the model documentation quickstart guide more user-friendly and informative for beginners. A new “quickstart” directory has been added to notebooks/, along with an updated README to guide you:

Add demo notebook for code explainer

validmind-library

2.8.26

enhancement

highlight

This update introduces a comprehensive script and Jupyter notebook for documenting and explaining a customer churn prediction model using the ValidMind library.

Add status flags to ongoing monitoring tests in ValidMind Library

validmind-library

2.8.26

enhancement

This update enhances the log_metric and alog_metric functions in the ValidMind Library by adding visual status indicators for monitoring metrics. You can now use the passed parameter in the log_metric() function to include status badges in Metrics Over Time blocks. The passed parameter accepts a boolean value: passed=True shows a green “Satisfactory” badge, while passed=False shows a yellow “Requires Attention” badge.

Quickstart guide for model validation with ValidMind Library

validmind-library

2.8.26

highlight

Get started with model validation using the ValidMind Library with our new quickstart guide:

Update registration and login info in docs site and notebooks

validmind-library

2.8.26

This update standardizes the alert messages displayed across various notebooks by updating the text and some CSS styling details in the HTML blocks.

Support customizable judge LLM and embeddings in ValidMind tests

validmind-library

2.8.26

enhancement

This update enhances your experience by allowing you to use your own LLM. All existing tests that use an LLM now support a user-defined LLM or Embedding model, provided it is compatible with the Langchain Chat/Embedding framework. This functionality is available for RAGAS and prompt validation tests, offering greater customization and flexibility in your AI and prompt validation processes.

Fix pandas DataFrame dtype preservation in VMDataset initialization

validmind-library

2.8.22

bug

enhancement

You can now reduce memory usage when initializing VMDataset objects with vm.init_dataset(). We’ve introduced a copy_data option that lets you avoid copying the input dataframe, which is useful for handling large datasets in environments with limited memory. By default, copy_data is set to True. Here’s how to use it:

Add RAG benchmarking demo notebook

validmind-library

2.8.20

documentation

enhancement

We have introduced a comprehensive notebook, rag_benchmarking_demo.ipynb, to help you benchmark Retrieval-Augmented Generation (RAG) models using the ValidMind library. This notebook allows you to compare multiple configurations for the RAG RFP use case.

New notebook series for model validation with ValidMind

validmind-library

2.8.20

documentation

highlight

Explore our new series of introductory notebooks designed for model validation with ValidMind. These notebooks guide you through the entire process, tailored to common scenarios:

Add text support for custom test descriptions

validmind-library

2.8.20

documentation

enhancement

highlight

We have enhanced the custom test framework, allowing you to define custom descriptions for your tests. If a test returns a string, it will be used as the test description, overriding the automatic description generation.

New Jupyter notebook for validating an application scorecard model

validmind-library

2.8.17

documentation

enhancement

Discover how to independently assess an application scorecard model with our new Jupyter notebook, Validate an application scorecard model. You can use ValidMind to evaluate a model’s development through comprehensive testing and analysis, including benchmarking with challenger models.

Expose log text interface for qualitative text sections

validmind-library

2.8.17

documentation

enhancement

highlight

You can now log free-form text content in the ValidMind Library to enhance your model documentation. This update introduces the log_text function, allowing you to easily integrate text data into your documentation.