July 22, 2024
Release highlights
On top of our increased support for tests and arrays in the ValidMind Developer Framework and our first ValidMind Academy training modules, we’ve introduced customizable model workflows, new role functionality, and new report types.
Support for comparison tests
You can now run comparison tests with the ValidMind Developer Framework.
- This feature is useful when you want to run the same test against multiple combinations of models or datasets.
- A comparison test creates a single documentation block that compares the individual results.
The updated run_test()
function allows you to pass an input_grid
which runs a test for all combinations of inputs.
Input grid example
= {
input_grid "model": ["XGBoost"],
"dataset": ["train_dataset", "test_dataset"],
}
A test runs once for each of the following input groups:
{"model": "XGBoost",
"dataset": "train_dataset"
}
{"model": "XGBoost",
"dataset": "test_dataset"
}
Example function calls
from validmind.tests import run_test
= {
input_grid "model": ["XGBoost"],
"dataset": ["train_dataset", "test_dataset"],
}
= run_test(
result "validmind.model_validation.sklearn.ClassifierPerformance",
input_grid,
)
= run_test(
result "validmind.model_validation.sklearn.ConfusionMatrix",
input_grid,
)
= run_test(
result "validmind.model_validation.sklearn.ROCCurve",
input_grid, )
Customizable model workflows
You can now manage lifecycle processes within your ValidMind Platform UI setup using customizable model workflows.
- You use these workflows to match your organizational needs for overseeing model development, validation, or implementation activities.
- A default workflow is provided for you as suggestion.
Enhancements
Support for 2D arrays in add_extra_column
- With enhancements to the
VMDataset
class, theadd_extra_column()
method now supports adding 2D arrays as single columns in the dataset’s DataFrame. - These changes ensure that 2D arrays are stored in a single column without altering the DataFrame structure.
For example:
import numpy as np
import pandas as pd
# sample DataFrame
= pd.DataFrame({
df 'A': [1, 2, 3],
'B': [4, 5, 6]
})
# sample 2D array
= np.array([
array_2d 1, 2],
[3, 4],
[5, 6]
[
])
= vm.init_dataset(
vm_dataset =df,
dataset=dummy_ds
input_id
)
vm_dataset.add_extra_columns("dummy_column",
array_2d )
Add new role functionality
Users with the Create Role
permission can now add a new role under Settings.
New report types
We added a number of new report types:
- Number of models by business unit and tier
- Number of models by status
- Avgerage number of days models spend in a status
- Models by number of findings
Bug fixes
Updated test descriptions
We fixed a number of missing test descriptions that were caused by a scripting issue.
Documentation
New training collateral
We’re introducing the first training modules that are part of our ValidMind Academy training program for:
- Model developers
- Model validators
- Administrators
How to upgrade
ValidMind Platform UI
To access the latest version of the ValidMind Platform UI,1 hard refresh your browser tab:
- Windows:
Ctrl
+Shift
+R
ORCtrl
+F5
- MacOS:
⌘ Cmd
+Shift
+R
OR hold down⌘ Cmd
and click theReload
button
ValidMind Developer Framework
To upgrade to the latest version of the ValidMind Developer Framework:2
In your Jupyter Notebook:
Then within a code cell or your terminal, run:
%pip install --upgrade validmind
You may need to restart your kernel after running the upgrade package for changes to be applied.