Learn more — ValidMind Library
The ValidMind Library is a Python library and documentation engine designed to streamline the process of documenting various types of models.
December 6, 2024
We’ve chosen some new names for our product offerings, expanded our training courses, introduced new large language model (LLM) features — and much, much, more!
Release highlights
ValidMind Library (v2.5.25)
ValidMind Developer Framework > ValidMind Library
To reduce ambiguity and highlight the capabilities of our developer tools, as of v2.5.25 what was previously referred to as the ValidMind Developer Framework is now known as the ValidMind Library.
Code samples
RAG Model Documentation Demo
Check out our RAG Model Documentation Demo Jupyter Notebook!
Learn how to build a basic RAG (Retrieval-Augmented Generation) model to help automatically answer RFP (Request for Proposal) questions using GenAI.
From in-depth tests assessing generation quality to checks detecting bias and toxicity, use LangChain2 and ValidMind to build, evaluate, and document a simple RAG model as it is being developed.
ValidMind Platform (v1.27.7)
ValidMind Platform UI > ValidMind Platform
To reduce ambiguity and highlight the capabilities of our cloud-hosted interface, as of v1.27.7 what was previously referred to as the ValidMind Platform UI is now simply known as the ValidMind Platform.
Submit product feedback directly
You can now submit feature requests, bug reports, or any other feedback directly within the ValidMind Platform.
Click on the Talk to us. tab on the right-hand side of any screen and follow three easy steps:
Talk to us. feedback panel within the ValidMind PlatformSupport for custom analytics
You can now add, edit, or remove custom data to your analytics within the ValidMind Platform in two steps:
- Add custom analytic pages — Custom analytics pages allow you to add custom visualizations on top of the out-of-the-box reports provided by ValidMind.
- Add custom visualizations — A visualization is a custom reporting widget that retrieves data on your metric of choice. Choose from bar charts, pie charts, or a simple counter.
Documentation
User guide updates
Updated product terminology
In concert with our product renaming across the ValidMind Library and the ValidMind Platform, we’ve edited all user guides and external facing documentation to reflect these updates.
Of note is the Developers drop-down menu, previously called Developer Framework.
New ValidMind branding
Along with keeping our content up to date,4 we’ve refreshed the look and feel of our documentation to reflect ValidMind’s new brand colors used on our main website5 and within the ValidMind Platform.
Improved documentation homepage
We’ve overhauled the documentation homepage. Check it out:
- More quick links out to popular user guides,
- New sections for training collateral and latest releases,
- And updated search functionality!
Updated logo and theme
The ValidMind logo has been updated across the documentation, and a new style applied to our side navigation and table of contents to unify the experience across our products:
ValidMind Academy portal
To complement our rebranding,6 our ValidMind Academy training now features its own theme and landing page to distinguish the material from the rest of the documentation along with content updates.7
- This new training homepage makes it easier for you to find and register for courses.
- Each course offered now includes a preview of the learning objectives and an estimated duration, to inform your instruction experience.
ValidMind LLM feature overview
ValidMind offers several specialized features that use large language models (LLMs), such as our Document Checker. These features streamline model risk management and ensure regulatory compliance.
We’ve created a guide on how we approach these features and what you need to know:
Enhancements
ValidMind Library (v2.5.25)
Added parameter grid support in comparison tests
In addition to comparison tests using the input_grid parameter in run_test() functionality, we’ve added similar support for params:
- Useful when you want to run the same test against multiple combinations of test parameters and create a single documentation block that compares the individual results, the updated
run_test()function allows you to pass aparam_grid. - This grid runs a test for all combinations of parameters.
Example
For this param grid:
param_grid = {
"param1": [1],
"param2": [0.1, 0.2],
}A test runs once for each of the following param groups:
{"param1": 1, "param2": 0.1}
{"param1": 1, "param2": 0.2}ValidMind Platform (v1.27.7)
User experience
Updated user interface theme
With a new name8 comes a fresh skin — the ValidMind Platform’s theme now matches our main website and documentation, featuring our new logo and accent color:
Accepting AI terms of use
- During registration with the ValidMind Platform,9 new users are now asked to read and accept our terms and conditions, including our AI Usage Policy.
- Existing users will see a dialog popup next time they log in, asking them to review and accept the same terms and conditions before continuing.
Model workflows
Support for attaching files on workflow transitions
We now support including attachment fields as a request field on a User Action step within workflows, allowing you to directly request supporting documentation for transitioning a model into its next lifecycle status.
Ability to configure divergent paths in workflows
We continue to add new functionality to model workflows within the ValidMind Platform, including the ability to now select a User Role or a User Type inventory field in an Approval step, and the ability to configure diverging paths with Condition Branch steps.
Model inventory fields
Ability to configure calculated model inventory fields
Custom model inventory fields are now more flexible than ever — introducing the ability to define a formula(params) function that automatically calculates and returns a value based on the params dictionary, which includes selected custom field keys retrieved from your other inventory model fields.
Example formula:
def formula(params):
# High Risk: If materiality is high risk, return high risk regardless of complexity
if params.materiality == "High Risk":
return "High Risk"
# Medium Risk: If materiality is low risk but complexity is high risk, return medium risk
if params.materiality == "Low Risk" and params.complexity == "High Risk":
return "Medium Risk"
# Low Risk: Both materiality and complexity are low risk
return "Low Risk"Ability to add descriptions to attachments
You can now annotate attachments uploaded to both models or model findings, allowing you to describe the contents of the file or provide commentary.
Model documentation
Ability to filter and sort historical test runs
If more than one set of test results has been logged with the ValidMind Library to the ValidMind Platform, you can now filter your historical test results for ease of access.
Documentation sections linked to evidence
When working with validation reports, linking developer evidence is now associated with a documentation section, making validation reports clearer and the process easier for you to assess compliance.
Bug fixes
ValidMind Platform (v1.27.7)
- Previously, the last updated value attached to templates did not accurately display the date of last modification — this has now been updated to reflect the correct last adjustment date.
- Previously, display issues were present when adding or removing roles from users within the User Directory — these issues have now been resolved.
Documentation
User guide updates
We’ve been working hard to keep our documentation up to date with our lightning-fast feature releases. Below is a round-up of some notable user guide improvements not covered by independent feature release announcements:
Configuration
Register with ValidMind
Signing up for ValidMind is free — and now you can follow our step-by-step registration guide to get you started.
Managing your organization
After you sign up for ValidMind, refer to our guides on setting up your organization, including options for establishing a private connection between ValidMind and your company network.
Managing users
We’ve clarified prerequisites for tasks and the permissions default roles have and improved our information on user management to help you understand how access to ValidMind works.
Model inventory
Register models in the inventory
To complement the ability to add your own model inventory fields, model registration within ValidMind Platform is now a two-step process, broken down into Basic Model Information and Additional Model Information with a matching user guide.
View model activity
To better align documentation with what users see within the ValidMind Platform, our previous article titled “View documentation activity” has been renamed to View model activity and now lives under the model inventory sub-section under Guides.
This new guide features expanded information on the auditing feature, including information on how to filter tracked activity.
Model documentation
Working with documentation templates
We’ve expanded our reference guides on working with documentation templates, including updated information on customizing templates, and swapping between versions of different types of templates.
Work with content blocks
We’ve expanded our reference guide on working with content blocks, including updated information on different content block types, an overview of our content editing toolbar, and a section on how to insert math formulas right into your documentation.
Model validation
Working with model findings
Our reference material on model findings has been expanded to include key concepts, and have been broken down into two sub-guides detailing viewing and filtering model findings, and adding and managing model findings.
Reporting
On top of support for custom analytics,10 we’ve updated our feature documentation for reporting to include overviews of default report pages provided out-of-the-box by ValidMind.
Information on available analytics and setting up reporting has also moved into its own section under Guides in reporting.
Developers
Documenting models
We’ve refined our developer guide on how to use the ValidMind Library and ValidMind Platform document models complete with supporting test results.
Upgrade ValidMind
In our Jupyter Notebook samples, we’ve now included information within the notebooks themselves on how to upgrade to the latest version of the ValidMind Library.
Feature renaming
- resource statuses > model lifecycle statuses
- We previously referred to model lifecycle statuses as “resource statuses.”
Model lifecycle statuses have been renamed in the ValidMind Platform and documentation updated to reflect this shift.
- custom fields > model inventory fields
- We previously referred to model inventory fields as “custom fields.”
Custom fields have been renamed in the ValidMind Platform and documentation updated to reflect this change.
ValidMind Academy updates
Updated validator training
- We expanded our validator training and along with that, our validator user guides.
- Now they both include more guidance for tracking and resolving issues identified in model findings.
Site enhancements
Mobile responsive redesign
- Previously, we made use of an outdated format of rendering columns that resulted in some of our documentation pages displaying poorly on mobile-width screens.
- We’ve now reimplemented these pages — including our index landing page — to be mobile responsive:
Style and accuracy edits
Improved inline link styling
- Previously, we employed heavy use of inline links in instructional text that took the reader prematurely out of the context.
- Now, we use of footnotes to indicate supplemental information, providing a cleaner and easier to follow experience:
Enlargable lightbox-style images
- Previously, we embedded detailed images in our user guides without providing the ability to enlarge them.
- Now, images that appear small within the user guides can be clicked on to expand them for ease of readability:
Authoring enhancements
Did you know that along with the ValidMind Library, the code for our documentation site is also open-source?
Below are some improvements we’ve made to our authoring tools to streamline the process for contributors:
Variables and single-sourcing
- Renaming our product offerings was a breeze thanks to our established variable infrastructure:
- When possible, we prefer to single-source our content to reduce content silos and to make updating content efficient. For example, the instructions on How to upgrade at the bottom of every release is simply a single-sourced file!11
Style guide for training materials
To support the release of ValidMind Academy, we’ve included a section in our style guide to cover guidelines unique to our training materials.
404 page
Previously, visiting a non-existing page within the documentation would present you with a poor user experience in the form of a default XML error.
We’ve now introduced a custom 404 page where you can perform a search instead.
How to upgrade
ValidMind Platform
To access the latest version of the ValidMind Platform,12 hard refresh your browser tab:
- Windows:
Ctrl+Shift+RORCtrl+F5 - MacOS:
⌘ Cmd+Shift+ROR hold down⌘ Cmdand click theReloadbutton
ValidMind Library
To upgrade the ValidMind Library:13
In your Jupyter Notebook:
Then within a code cell or your terminal, run:
%pip install --upgrade validmind
You may need to restart your kernel after running the upgrade package for changes to be applied.