ValidMind for validation 4 — Finalize testing and reporting

Learn how to use ValidMind for your end-to-end validation process with our series of four introductory notebooks. In this last notebook, finalize the compliance assessment process and have a complete validation report ready for review.

This notebook will walk you through how to supplement ValidMind tests with your own custom tests and include them as additional evidence in your validation report. A custom test is any function that takes a set of inputs and parameters as arguments and returns one or more outputs:

The function can be as simple or as complex as you need it to be — it can use external libraries, make API calls, or do anything else that you can do in Python.
The only requirement is that the function signature and return values can be "understood" and handled by the ValidMind Library. As such, custom tests offer added flexibility by extending the default tests provided by ValidMind, enabling you to document any type of record (model) or use case.

For a more in-depth introduction to custom tests, refer to our Implement custom tests notebook.

Learn by doing

Our course tailor-made for validators new to ValidMind combines this series of notebooks with more a more in-depth introduction to the ValidMind Platform — Validator Fundamentals

Prerequisites

In order to finalize validation and reporting, you'll need to first have:

Registered a model within the ValidMind Platform and granted yourself access to the model as a validator
Installed the ValidMind Library in your local environment, allowing you to access all its features
Learned how to import and initialize datasets and models for use with ValidMind
Understood the basics of how to identify and run validation tests
Run validation tests for your champion and challengers, and logged the results of those tests to the ValidMind Platform
Inserted your logged test results into your validation report
Added some preliminary artifacts (findings) to your validation report

Need help with the above steps?

Refer to the first three notebooks in this series:

Setting up

This section should be very familiar to you now — as we performed the same actions in the previous two notebooks in this series.

Initialize the ValidMind Library

As usual, let's first connect up the ValidMind Library to our model we previously registered in the ValidMind Platform:

On the left sidebar that appears for your model, select Getting Started and select Validation from the Document drop-down menu.
Click Copy snippet to clipboard.
Next, load your model identifier credentials from an .env file or replace the placeholder with your own code snippet:

# Make sure the ValidMind Library is installed

%pip install -q validmind

# Load your model identifier credentials from an `.env` file

%load_ext dotenv
%dotenv .env

# Or replace with your code snippet

import validmind as vm

vm.init(
    # api_host="...",
    # api_key="...",
    # api_secret="...",
    # model="...",
    document="validation-report",
)

Note: you may need to restart the kernel to use updated packages.

2026-07-31 16:51:25,442 - INFO(validmind.api_client): 🎉 Connected to ValidMind!
📊 Model: [ValidMind Academy] Model validation (ID: cmalguc9y02ok199q2db381ib)
📁 Document Type: validation_report

Import the sample dataset

Next, we'll load in the same sample Bank Customer Churn Prediction dataset used to develop the champion that we will independently preprocess:

# Load the sample dataset
from validmind.datasets.classification import customer_churn as demo_dataset

print(
    f"Loaded demo dataset with: \n\n\t• Target column: '{demo_dataset.target_column}' \n\t• Class labels: {demo_dataset.class_labels}"
)

raw_df = demo_dataset.load_data()

Loaded demo dataset with: 

    • Target column: 'Exited' 
    • Class labels: {'0': 'Did not exit', '1': 'Exited'}

# Initialize the raw dataset for use in ValidMind tests
vm_raw_dataset = vm.init_dataset(
    dataset=raw_df,
    input_id="raw_dataset",
    target_column="Exited",
)

import pandas as pd

raw_copy_df = raw_df.sample(frac=1)  # Create a copy of the raw dataset

# Create a balanced dataset with the same number of exited and not exited customers
exited_df = raw_copy_df.loc[raw_copy_df["Exited"] == 1]
not_exited_df = raw_copy_df.loc[raw_copy_df["Exited"] == 0].sample(n=exited_df.shape[0])

balanced_raw_df = pd.concat([exited_df, not_exited_df])
balanced_raw_df = balanced_raw_df.sample(frac=1, random_state=42)

Let’s also quickly remove highly correlated features from the dataset using the output from a ValidMind test:

# Register new data and now 'balanced_raw_dataset' is the new dataset object of interest
vm_balanced_raw_dataset = vm.init_dataset(
    dataset=balanced_raw_df,
    input_id="balanced_raw_dataset",
    target_column="Exited",
)

# Run HighPearsonCorrelation test with our balanced dataset as input and return a result object
corr_result = vm.tests.run_test(
    test_id="validmind.data_validation.HighPearsonCorrelation",
    params={"max_threshold": 0.3},
    inputs={"dataset": vm_balanced_raw_dataset},
)

❌ High Pearson Correlation

The High Pearson Correlation test evaluates pairwise linear relationships between features to identify potentially redundant variables or multicollinearity. The result table reports the top feature pairs ranked by Pearson correlation coefficient together with a pass/fail assessment based on the configured absolute correlation threshold of 0.3. Across the 10 reported pairs, coefficients range from -0.1947 to 0.3478, and only one pair exceeds the threshold. The largest observed relationship is between Age and Exited, while the remaining listed pairs are below the threshold and marked as passing.

Key insights:

Single threshold breach observed: The pair (Age, Exited) has a Pearson correlation coefficient of 0.3478, which exceeds the configured threshold of 0.3 and is the only reported failing relationship.
All other reported pairs remain below threshold: The other 9 listed feature pairs have absolute correlation values below 0.3, with the next largest magnitudes being -0.1947 for (IsActiveMember, Exited) and -0.1772 for (Balance, NumOfProducts).
Reported correlations are generally weak in magnitude: Aside from the Age–Exited pair, the reported coefficients are relatively small, including 0.1549 for (Balance, Exited), -0.0595 for (NumOfProducts, Exited), and several values close to zero such as 0.0470, -0.0459, -0.0399, 0.0376, and -0.0356.
Both positive and negative relationships are present: The reported results include positive correlations, such as (Age, Exited) at 0.3478 and (Balance, Exited) at 0.1549, as well as negative correlations, such as (IsActiveMember, Exited) at -0.1947 and (Balance, NumOfProducts) at -0.1772.

Overall, the reported correlation structure is limited in magnitude across most listed feature pairs, with one exception. The only pair exceeding the configured threshold is Age and Exited, while all other reported relationships remain below the threshold and are marked as passing. This result indicates that the top reported linear relationships are concentrated in a single flagged pair rather than broadly distributed across the feature set.

Parameters:

{
  "max_threshold": 0.3
}

Tables

Columns	Coefficient	Pass/Fail
(Age, Exited)	0.3478	Fail
(IsActiveMember, Exited)	-0.1947	Pass
(Balance, NumOfProducts)	-0.1772	Pass
(Balance, Exited)	0.1549	Pass
(NumOfProducts, Exited)	-0.0595	Pass
(NumOfProducts, IsActiveMember)	0.0470	Pass
(Tenure, IsActiveMember)	-0.0459	Pass
(Age, NumOfProducts)	-0.0399	Pass
(Age, Balance)	0.0376	Pass
(Age, HasCrCard)	-0.0356	Pass

# From result object, extract table from `corr_result.tables`
features_df = corr_result.tables[0].data
features_df

	Columns	Coefficient	Pass/Fail
0	(Age, Exited)	0.3478	Fail
1	(IsActiveMember, Exited)	-0.1947	Pass
2	(Balance, NumOfProducts)	-0.1772	Pass
3	(Balance, Exited)	0.1549	Pass
4	(NumOfProducts, Exited)	-0.0595	Pass
5	(NumOfProducts, IsActiveMember)	0.0470	Pass
6	(Tenure, IsActiveMember)	-0.0459	Pass
7	(Age, NumOfProducts)	-0.0399	Pass
8	(Age, Balance)	0.0376	Pass
9	(Age, HasCrCard)	-0.0356	Pass

# Extract list of features that failed the test
high_correlation_features = features_df[features_df["Pass/Fail"] == "Fail"]["Columns"].tolist()
high_correlation_features

['(Age, Exited)']

# Extract feature names from the list of strings
high_correlation_features = [feature.split(",")[0].strip("()") for feature in high_correlation_features]
high_correlation_features

['Age']

# Remove the highly correlated features from the dataset
balanced_raw_no_age_df = balanced_raw_df.drop(columns=high_correlation_features)

# Re-initialize the dataset object
vm_raw_dataset_preprocessed = vm.init_dataset(
    dataset=balanced_raw_no_age_df,
    input_id="raw_dataset_preprocessed",
    target_column="Exited",
)

# Re-run the test with the reduced feature set
corr_result = vm.tests.run_test(
    test_id="validmind.data_validation.HighPearsonCorrelation",
    params={"max_threshold": 0.3},
    inputs={"dataset": vm_raw_dataset_preprocessed},
)

✅ High Pearson Correlation

The High Pearson Correlation test evaluates pairwise linear relationships among features to identify potentially redundant variables or multicollinearity. The result table lists the ten strongest feature-pair correlations observed in the dataset, along with their Pearson coefficients and pass/fail status against the configured absolute threshold of 0.3. All reported coefficients are below the threshold, with values ranging from -0.1947 to 0.1549, and each pair is marked as Pass.

Key insights:

No pair exceeds threshold: All ten reported feature pairs pass the test, as none of the absolute Pearson correlation coefficients exceeds the configured threshold of 0.3.
Strongest observed relationship is limited: The largest absolute correlation is between IsActiveMember and Exited at -0.1947, which remains materially below the threshold.
Top correlations are weak in magnitude: The next largest relationships, Balance with NumOfProducts (-0.1772) and Balance with Exited (0.1549), also remain well below the threshold, indicating limited linear association among the strongest reported pairs.
Remaining feature pairs are near zero: The other listed coefficients range from -0.0595 to 0.0189, showing very weak linear relationships across those feature combinations.

The reported correlation structure does not show evidence of high pairwise linear dependence under the configured test threshold. The strongest observed associations remain below 0.2 in absolute value, and the majority of listed pairs are closer to zero. Collectively, the result indicates low pairwise linear correlation among the top reported feature combinations in this dataset.

Parameters:

{
  "max_threshold": 0.3
}

Tables

Columns	Coefficient	Pass/Fail
(IsActiveMember, Exited)	-0.1947	Pass
(Balance, NumOfProducts)	-0.1772	Pass
(Balance, Exited)	0.1549	Pass
(NumOfProducts, Exited)	-0.0595	Pass
(NumOfProducts, IsActiveMember)	0.0470	Pass
(Tenure, IsActiveMember)	-0.0459	Pass
(HasCrCard, IsActiveMember)	-0.0330	Pass
(HasCrCard, Exited)	-0.0265	Pass
(Tenure, EstimatedSalary)	0.0254	Pass
(CreditScore, Balance)	0.0189	Pass

Split the preprocessed dataset

With our raw dataset rebalanced with highly correlated features removed, let's now spilt our dataset into train and test in preparation for model evaluation testing:

# Encode categorical features in the dataset
balanced_raw_no_age_df = pd.get_dummies(
    balanced_raw_no_age_df, columns=["Geography", "Gender"], drop_first=True
)
balanced_raw_no_age_df.head()

	CreditScore	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited	Geography_Germany	Geography_Spain	Gender_Male
1589	716	3	128743.80	1	0	0	197322.13	0	True	False	False
4269	598	5	0.00	2	1	1	83103.46	0	False	True	False
2960	709	0	112949.71	1	0	0	155231.55	0	True	False	True
4495	636	7	124447.73	1	1	1	141364.62	1	False	True	True
6493	640	9	94752.49	1	1	0	184006.36	1	True	False	False

from sklearn.model_selection import train_test_split

# Split the dataset into train and test
train_df, test_df = train_test_split(balanced_raw_no_age_df, test_size=0.20)

X_train = train_df.drop("Exited", axis=1)
y_train = train_df["Exited"]
X_test = test_df.drop("Exited", axis=1)
y_test = test_df["Exited"]

# Initialize the split datasets
vm_train_ds = vm.init_dataset(
    input_id="train_dataset_final",
    dataset=train_df,
    target_column="Exited",
)

vm_test_ds = vm.init_dataset(
    input_id="test_dataset_final",
    dataset=test_df,
    target_column="Exited",
)

Import the champion model

With our raw dataset assessed and preprocessed, let's go ahead and import the champion submitted by the development team in the format of a .pkl file: lr_model_champion.pkl

# Import the champion model
import pickle as pkl

with open("lr_model_champion.pkl", "rb") as f:
    log_reg = pkl.load(f)

/opt/hostedtoolcache/Python/3.11.15/x64/lib/python3.11/site-packages/sklearn/base.py:525: InconsistentVersionWarning: Trying to unpickle estimator LogisticRegression from version 1.3.2 when using version 1.9.0. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(

Train potential challenger model

We'll also train our random forest classification challenger to see how it compares:

# Import the Random Forest Classification model
from sklearn.ensemble import RandomForestClassifier

# Create the model instance with 50 decision trees
rf_model = RandomForestClassifier(
    n_estimators=50,
    random_state=42,
)

# Train the model
rf_model.fit(X_train, y_train)

RandomForestClassifier(n_estimators=50, random_state=42)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

RandomForestClassifier

?Documentation for RandomForestClassifieriFitted

Parameters

	n_estimators n_estimators: int, default=100 The number of trees in the forest. .. versionchanged:: 0.22 The default value of ``n_estimators`` changed from 10 to 100 in 0.22.	50
	random_state random_state: int, RandomState instance or None, default=None Controls both the randomness of the bootstrapping of the samples used when building trees (if ``bootstrap=True``) and the sampling of the features to consider when looking for the best split at each node (if ``max_features < n_features``). See :term:`Glossary <random_state>` for details.	42
	criterion criterion: {"gini", "entropy", "log_loss"}, default="gini" The function to measure the quality of a split. Supported criteria are "gini" for the Gini impurity and "log_loss" and "entropy" both for the Shannon information gain, see :ref:`tree_mathematical_formulation`. Note: This parameter is tree-specific.	'gini'
	max_depth max_depth: int, default=None The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.	None
	min_samples_split min_samples_split: int or float, default=2 The minimum number of samples required to split an internal node: - If int, then consider `min_samples_split` as the minimum number. - If float, then `min_samples_split` is a fraction and `ceil(min_samples_split * n_samples)` are the minimum number of samples for each split. .. versionchanged:: 0.18 Added float values for fractions.	2
	min_samples_leaf min_samples_leaf: int or float, default=1 The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least ``min_samples_leaf`` training samples in each of the left and right branches. This may have the effect of smoothing the model, especially in regression. - If int, then consider `min_samples_leaf` as the minimum number. - If float, then `min_samples_leaf` is a fraction and `ceil(min_samples_leaf * n_samples)` are the minimum number of samples for each node. .. versionchanged:: 0.18 Added float values for fractions.	1
	min_weight_fraction_leaf min_weight_fraction_leaf: float, default=0.0 The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node. Samples have equal weight when sample_weight is not provided.	0.0
	max_features max_features: {"sqrt", "log2", None}, int or float, default="sqrt" The number of features to consider when looking for the best split: - If int, then consider `max_features` features at each split. - If float, then `max_features` is a fraction and `max(1, int(max_features * n_features_in_))` features are considered at each split. - If "sqrt", then `max_features=sqrt(n_features)`. - If "log2", then `max_features=log2(n_features)`. - If None, then `max_features=n_features`. .. versionchanged:: 1.1 The default of `max_features` changed from `"auto"` to `"sqrt"`. Note: the search for a split does not stop until at least one valid partition of the node samples is found, even if it requires to effectively inspect more than ``max_features`` features.	'sqrt'
	max_leaf_nodes max_leaf_nodes: int, default=None Grow trees with ``max_leaf_nodes`` in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.	None
	min_impurity_decrease min_impurity_decrease: float, default=0.0 A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:: N_t / N * (impurity - N_t_R / N_t * right_impurity - N_t_L / N_t * left_impurity) where ``N`` is the total number of samples, ``N_t`` is the number of samples at the current node, ``N_t_L`` is the number of samples in the left child, and ``N_t_R`` is the number of samples in the right child. ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum, if ``sample_weight`` is passed. .. versionadded:: 0.19	0.0
	bootstrap bootstrap: bool, default=True Whether bootstrap samples are used when building trees. If False, the whole dataset is used to build each tree.	True
	oob_score oob_score: bool or callable, default=False Whether to use out-of-bag samples to estimate the generalization score. By default, :func:`~sklearn.metrics.accuracy_score` is used. Provide a callable with signature `metric(y_true, y_pred)` to use a custom metric. Only available if `bootstrap=True`. For an illustration of out-of-bag (OOB) error estimation, see the example :ref:`sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py`.	False
	n_jobs n_jobs: int, default=None The number of jobs to run in parallel. :meth:`fit`, :meth:`predict`, :meth:`decision_path` and :meth:`apply` are all parallelized over the trees. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary <n_jobs>` for more details.	None
	verbose verbose: int, default=0 Controls the verbosity when fitting and predicting.	0
	warm_start warm_start: bool, default=False When set to ``True``, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new forest. See :term:`Glossary <warm_start>` and :ref:`tree_ensemble_warm_start` for details.	False
	class_weight class_weight: {"balanced", "balanced_subsample"}, dict or list of dicts, default=None Weights associated with classes in the form ``{class_label: weight}``. If not given, all classes are supposed to have weight one. For multi-output problems, a list of dicts can be provided in the same order as the columns of y. Note that for multioutput (including multilabel) weights should be defined for each class of every column in its own dict. For example, for four-class multilabel classification weights should be [{0: 1, 1: 1}, {0: 1, 1: 5}, {0: 1, 1: 1}, {0: 1, 1: 1}] instead of [{1:1}, {2:5}, {3:1}, {4:1}]. The "balanced" mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as ``n_samples / (n_classes * np.bincount(y))`` The "balanced_subsample" mode is the same as "balanced" except that weights are computed based on the bootstrap sample for every tree grown. For multi-output, the weights of each column of y will be multiplied. Note that these weights will be multiplied with sample_weight (passed through the fit method) if sample_weight is specified.	None
	ccp_alpha ccp_alpha: non-negative float, default=0.0 Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ``ccp_alpha`` will be chosen. By default, no pruning is performed. See :ref:`minimal_cost_complexity_pruning` for details. See :ref:`sphx_glr_auto_examples_tree_plot_cost_complexity_pruning.py` for an example of such pruning. .. versionadded:: 0.22	0.0
	max_samples max_samples: int or float, default=None If bootstrap is True, the number of samples to draw from X to train each base estimator. - If None (default), then draw `X.shape[0]` samples irrespective of `sample_weight`. - If int, then draw `max_samples` samples. - If float, then draw `max_samples * X.shape[0]` unweighted samples or `max_samples * sample_weight.sum()` weighted samples. .. versionadded:: 0.22 .. versionchanged:: 1.9 Float `max_samples` is relative to `sample_weight.sum()` instead of `X.shape[0]` for weighted samples.	None
	monotonic_cst monotonic_cst: array-like of int of shape (n_features), default=None Indicates the monotonicity constraint to enforce on each feature. - 1: monotonic increase - 0: no constraint - -1: monotonic decrease If monotonic_cst is None, no constraints are applied. Monotonicity constraints are not supported for: - multiclass classifications (i.e. when `n_classes > 2`), - multioutput classifications (i.e. when `n_outputs_ > 1`). The constraints hold over the probability of the positive class. Read more in the :ref:`User Guide <monotonic_cst_gbdt>`. .. versionadded:: 1.4	None

Fitted attributes

Name	Type	Value
classes_ classes_: ndarray of shape (n_classes,) or a list of such arrays The classes labels (single output problem), or a list of arrays of class labels (multi-output problem).	ndarray[int64](2,)	[0,1]
estimator_ estimator_: :class:`~sklearn.tree.DecisionTreeClassifier` The child estimator template used to create the collection of fitted sub-estimators. .. versionadded:: 1.2 `base_estimator_` was renamed to `estimator_`.	DecisionTreeClassifier	DecisionTreeClassifier()
estimators_ estimators_: list of DecisionTreeClassifier The collection of fitted sub-estimators.	list	[DecisionTreeC...te=1608637542), DecisionTreeC...te=1273642419), DecisionTreeC...te=1935803228), DecisionTreeC...ate=787846414), ...]
estimators_samples_ estimators_samples_: list of arrays The subset of drawn samples (i.e., the in-bag samples) for each base estimator. Each subset is defined by an array of the indices selected. .. versionadded:: 1.4	list	[array([2523, ..., dtype=int32), array([ 686, ..., dtype=int32), array([1366, ..., dtype=int32), array([1591, ..., dtype=int32), ...]
feature_importances_ feature_importances_: ndarray of shape (n_features,) The impurity-based feature importances. The higher, the more important the feature. The importance of a feature is computed as the (normalized) total reduction of the criterion brought by that feature. It is also known as the Gini importance. Warning: impurity-based feature importances can be misleading for high cardinality features (many unique values). See :func:`sklearn.inspection.permutation_importance` as an alternative.	ndarray[float64](10,)	[0.19,0.12,0.2 ,...,0.04,0.02,0.03]
feature_names_in_ feature_names_in_: ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0	ndarray[object](10,)	['CreditScore','Tenure','Balance',...,'Geography_Germany', 'Geography_Spain','Gender_Male']
n_classes_ n_classes_: int or list The number of classes (single output problem), or a list containing the number of classes for each output (multi-output problem).	int	2
n_features_in_ n_features_in_: int Number of features seen during :term:`fit`. .. versionadded:: 0.24	int	10
n_outputs_ n_outputs_: int The number of outputs when ``fit`` is performed.	int	1

Initialize the ValidMind models

In addition to the initialized datasets, you'll also need to initialize a ValidMind model object (vm_model) that can be passed to other functions for analysis and tests on the data for each of our two models:

# Initialize the champion logistic regression model
vm_log_model = vm.init_model(
    log_reg,
    input_id="log_model_champion",
)

# Initialize the challenger random forest classification model
vm_rf_model = vm.init_model(
    rf_model,
    input_id="rf_model",
)

# Assign predictions to Champion — Logistic regression model
vm_train_ds.assign_predictions(model=vm_log_model)
vm_test_ds.assign_predictions(model=vm_log_model)

# Assign predictions to Challenger — Random forest classification model
vm_train_ds.assign_predictions(model=vm_rf_model)
vm_test_ds.assign_predictions(model=vm_rf_model)

2026-07-31 16:51:38,608 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-07-31 16:51:38,610 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-07-31 16:51:38,610 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-07-31 16:51:38,614 - INFO(validmind.vm_models.dataset.utils): Done running predict()
2026-07-31 16:51:38,615 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-07-31 16:51:38,618 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-07-31 16:51:38,619 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-07-31 16:51:38,620 - INFO(validmind.vm_models.dataset.utils): Done running predict()
2026-07-31 16:51:38,623 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-07-31 16:51:38,646 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-07-31 16:51:38,648 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-07-31 16:51:38,670 - INFO(validmind.vm_models.dataset.utils): Done running predict()
2026-07-31 16:51:38,673 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-07-31 16:51:38,687 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-07-31 16:51:38,688 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-07-31 16:51:38,702 - INFO(validmind.vm_models.dataset.utils): Done running predict()

Implementing custom tests

Thanks to the documentation (Learn more: ValidMind for development), we know that the development team implemented a custom test to further evaluate the performance of the champion.

In a usual validation situation, you would load a saved custom test provided by the development team. In the following section, we'll have you implement the same custom test and make it available for reuse, to familiarize you with the processes.

Want to learn more about custom tests?

Refer to our in-depth introduction to custom tests: Implement custom tests

Implement a custom inline test

Let's implement the same custom inline test that calculates the confusion matrix for a binary classification model that the development team used in their performance evaluations.

An inline test refers to a test written and executed within the same environment as the code being tested — in this case, right in this Jupyter Notebook — without requiring a separate test file or framework.
You'll note that the custom test function is just a regular Python function that can include and require any Python library as you see fit.

Create a confusion matrix plot

Let's first create a confusion matrix plot using the confusion_matrix function from the sklearn.metrics module:

import matplotlib.pyplot as plt
from sklearn import metrics

# Get the predicted classes
y_pred = log_reg.predict(vm_test_ds.x)

confusion_matrix = metrics.confusion_matrix(y_test, y_pred)

cm_display = metrics.ConfusionMatrixDisplay(
    confusion_matrix=confusion_matrix, display_labels=[False, True]
)
cm_display.plot()

Next, create a @vm.test wrapper that will allow you to create a reusable test. Note the following changes in the code below:

The function confusion_matrix takes two arguments dataset and model. This is a VMDataset and VMModel object respectively.
- VMDataset objects allow you to access the dataset's true (target) values by accessing the .y attribute.
- VMDataset objects allow you to access the predictions for a given record (model) by accessing the .y_pred() method.
The function docstring provides a description of what the test does. This will be displayed along with the result in this notebook as well as in the ValidMind Platform.
The function body calculates the confusion matrix using the sklearn.metrics.confusion_matrix function as we just did above.
The function then returns the ConfusionMatrixDisplay.figure_ object — this is important as the ValidMind Library expects the output of the custom test to be a plot or a table.
The @vm.test decorator is doing the work of creating a wrapper around the function that will allow it to be run by the ValidMind Library. It also registers the test so it can be found by the ID my_custom_tests.ConfusionMatrix.

@vm.test("my_custom_tests.ConfusionMatrix")
def confusion_matrix(dataset, model):
    """The confusion matrix is a table that is often used to describe the performance of a classification model on a set of data for which the true values are known.

    The confusion matrix is a 2x2 table that contains 4 values:

    - True Positive (TP): the number of correct positive predictions
    - True Negative (TN): the number of correct negative predictions
    - False Positive (FP): the number of incorrect positive predictions
    - False Negative (FN): the number of incorrect negative predictions

    The confusion matrix can be used to assess the holistic performance of a classification model by showing the accuracy, precision, recall, and F1 score of the model on a single figure.
    """
    y_true = dataset.y
    y_pred = dataset.y_pred(model=model)

    confusion_matrix = metrics.confusion_matrix(y_true, y_pred)

    cm_display = metrics.ConfusionMatrixDisplay(
        confusion_matrix=confusion_matrix, display_labels=[False, True]
    )
    cm_display.plot()

    plt.close()  # close the plot to avoid displaying it

    return cm_display.figure_  # return the figure object itself

You can now run the newly created custom test on both the training and test datasets for both models using the run_test() function:

# Champion train and test
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:champion",
    input_grid={
        "dataset": [vm_train_ds,vm_test_ds],
        "model" : [vm_log_model]
    }
).log()

Confusion Matrix Champion

The ConfusionMatrix test evaluates classification performance by comparing predicted labels against true labels and reporting the counts of true positives, true negatives, false positives, and false negatives. The result is shown separately for the training dataset and the test dataset. For the training dataset, the matrix contains 862 true negatives, 451 false positives, 481 false negatives, and 791 true positives. For the test dataset, the matrix contains 189 true negatives, 114 false positives, 126 false negatives, and 218 true positives.

Key insights:

Correct classifications exceed errors: In both datasets, the diagonal cells are larger than the off-diagonal cells. Training results show 862 true negatives and 791 true positives versus 451 false positives and 481 false negatives, while test results show 189 true negatives and 218 true positives versus 114 false positives and 126 false negatives.
Error types are relatively balanced: The two misclassification counts are of similar magnitude in each dataset. On the training dataset, false positives (451) and false negatives (481) differ by 30, and on the test dataset, false positives (114) and false negatives (126) differ by 12.
Prediction mix is similar across datasets: The relative structure of the confusion matrices is comparable between training and test results. In both cases, true negatives are the largest count among the negative-class outcomes, and true positives are the largest count among the positive-class outcomes, with false negatives slightly exceeding false positives.

The confusion matrices show that the model produces more correct than incorrect classifications on both the training and test datasets. Misclassifications are distributed fairly evenly between false positives and false negatives, with a small tilt toward false negatives in both samples. The similarity in the count pattern between training and test results indicates consistent classification behavior across the two evaluated datasets.

Figures

ValidMind Figure my_custom_tests.ConfusionMatrix:champion:a3bd

ValidMind Figure my_custom_tests.ConfusionMatrix:champion:42e5

2026-07-31 16:51:45,437 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:champion does not exist in model's document

# Challenger train and test
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:challenger",
    input_grid={
        "dataset": [vm_train_ds,vm_test_ds],
        "model" : [vm_rf_model]
    }
).log()

Confusion Matrix Challenger

The Confusion Matrix test evaluates classification performance by comparing predicted labels with true labels across the training and test datasets. The results are presented as 2x2 confusion matrices with counts for true negatives, false positives, false negatives, and true positives. For the training dataset, the matrix contains 1,313 true negatives, 0 false positives, 1 false negative, and 1,271 true positives. For the test dataset, the matrix contains 210 true negatives, 93 false positives, 127 false negatives, and 217 true positives.

Key insights:

Near-perfect training classification: The training confusion matrix shows only 1 misclassified positive case and no misclassified negative cases, with 2,584 correct classifications out of 2,585 total observations.
Test errors in both directions: The test confusion matrix contains both false positives and false negatives, with 93 negative cases predicted as positive and 127 positive cases predicted as negative.
Higher false negatives than false positives: On the test dataset, false negatives exceed false positives by 34 cases, indicating more missed positive cases than incorrect positive predictions.
Correct predictions remain the largest cells: In the test matrix, true negatives (210) and true positives (217) are larger than the corresponding error counts, showing that correct classifications remain the most frequent outcomes in both classes.

The confusion matrix results show a sharp contrast between training and test classification outcomes. Training performance is almost exact, while the test dataset exhibits materially higher misclassification counts in both classes. The observed gap between the two datasets indicates that classification outcomes are substantially less accurate outside the training sample, with missed positive cases slightly more frequent than incorrect positive assignments.

Figures

ValidMind Figure my_custom_tests.ConfusionMatrix:challenger:8c55

ValidMind Figure my_custom_tests.ConfusionMatrix:challenger:9dc4

2026-07-31 16:51:52,595 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:challenger does not exist in model's document

Note the output returned indicating that a test-driven block doesn't currently exist in your documentation for some test IDs.

That's expected, as when we run validations tests the results logged need to be manually added to your report as part of your compliance assessment process within the ValidMind Platform.

Add parameters to custom tests

Custom tests can take parameters just like any other function. To demonstrate, let's modify the confusion_matrix function to take an additional parameter normalize that will allow you to normalize the confusion matrix:

@vm.test("my_custom_tests.ConfusionMatrix")
def confusion_matrix(dataset, model, normalize=False):
    """The confusion matrix is a table that is often used to describe the performance of a classification model on a set of data for which the true values are known.

    The confusion matrix is a 2x2 table that contains 4 values:

    - True Positive (TP): the number of correct positive predictions
    - True Negative (TN): the number of correct negative predictions
    - False Positive (FP): the number of incorrect positive predictions
    - False Negative (FN): the number of incorrect negative predictions

    The confusion matrix can be used to assess the holistic performance of a classification model by showing the accuracy, precision, recall, and F1 score of the model on a single figure.
    """
    y_true = dataset.y
    y_pred = dataset.y_pred(model=model)

    if normalize:
        confusion_matrix = metrics.confusion_matrix(y_true, y_pred, normalize="all")
    else:
        confusion_matrix = metrics.confusion_matrix(y_true, y_pred)

    cm_display = metrics.ConfusionMatrixDisplay(
        confusion_matrix=confusion_matrix, display_labels=[False, True]
    )
    cm_display.plot()

    plt.close()  # close the plot to avoid displaying it

    return cm_display.figure_  # return the figure object itself

Pass parameters to custom tests

You can pass parameters to custom tests by providing a dictionary of parameters to the run_test() function.

The parameters will override any default parameters set in the custom test definition. Note that dataset and model are still passed as inputs.
Since these are VMDataset or VMModel inputs, they have a special meaning.

Re-running and logging the custom confusion matrix with normalize=True for both models and our testing dataset looks like this:

# Champion with test dataset and normalize=True
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:test_normalized_champion",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_log_model]
    },
    params={"normalize": True}
).log()

Confusion Matrix Test Normalized Champion

The Confusion Matrix test evaluates classification outcomes by comparing predicted labels with true labels, and this normalized result shows the relative distribution of true negatives, false positives, false negatives, and true positives for the champion logistic model on the test dataset. The matrix is presented as a 2x2 heatmap with true labels on the rows and predicted labels on the columns. The four normalized cell values are 0.29 for true negatives, 0.18 for false positives, 0.19 for false negatives, and 0.34 for true positives.

Key insights:

True positives are the largest cell: The highest normalized value in the matrix is 0.34 in the true positive cell, indicating that correctly predicted positive cases represent the largest share of observations among the four outcome categories.
Correct classifications exceed errors: The diagonal cells sum to 0.63 (0.29 true negatives and 0.34 true positives), while the off-diagonal cells sum to 0.37 (0.18 false positives and 0.19 false negatives), showing more correct than incorrect classifications overall.
Error types are closely balanced: False positives account for 0.18 and false negatives account for 0.19, indicating that the two misclassification types occur at very similar normalized rates.
Positive-class outcomes are more prevalent: The true positive cell (0.34) exceeds the true negative cell (0.29), and the false negative cell (0.19) is slightly above the false positive cell (0.18), resulting in a larger combined share in the row for true positive cases.

The normalized confusion matrix shows that correct classifications make up the majority of outcomes, with the largest single share coming from true positives. Misclassification is distributed almost evenly between false positives and false negatives, with only a 0.01 difference between them. Overall, the observed pattern reflects a modestly stronger concentration in positive-class outcomes than in negative-class outcomes within this test result.

Parameters:

{
  "normalize": true
}

Figures

ValidMind Figure my_custom_tests.ConfusionMatrix:test_normalized_champion:f564

2026-07-31 16:52:00,358 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:test_normalized_champion does not exist in model's document

# Challenger with test dataset and normalize=True
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:test_normalized_challenger",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_rf_model]
    },
    params={"normalize": True}
).log()

Confusion Matrix Test Normalized Challenger

The ConfusionMatrix test evaluates classification performance by comparing predicted labels with true labels, and this result presents the normalized confusion matrix for test_dataset_final using rf_model. The matrix is shown as a 2x2 table with proportions rather than raw counts, with rows corresponding to true labels and columns corresponding to predicted labels. The displayed values are 0.32 for true negatives, 0.14 for false positives, 0.20 for false negatives, and 0.34 for true positives, allowing direct comparison of correct and incorrect classification outcomes across both classes.

Key insights:

Correct classifications dominate overall: The diagonal cells sum to 0.66, comprising 0.32 true negatives and 0.34 true positives. This exceeds the off-diagonal total of 0.34, indicating that correct predictions occur more frequently than misclassifications in the normalized result.
True positives slightly exceed true negatives: The true positive cell is 0.34, compared with 0.32 for the true negative cell. This indicates a marginally larger share of correctly identified positive cases than correctly identified negative cases.
False negatives exceed false positives: The false negative proportion is 0.20, while the false positive proportion is 0.14. Misclassification is therefore more concentrated in missed positive cases than in incorrect positive predictions.
Error distribution is asymmetric: The two error cells differ by 0.06, with the larger contribution coming from false negatives. This shows that the model’s incorrect classifications are not evenly distributed across the two error types.

The normalized confusion matrix shows that most observations fall into the correct-classification cells, with 0.66 of total outcomes on the diagonal and 0.34 off the diagonal. Correct identification is slightly higher for positive cases than for negative cases, as reflected by the 0.34 true positive share versus 0.32 true negative share. Among errors, false negatives occur more frequently than false positives, indicating that the model’s misclassification pattern is weighted more toward missed positive cases.

Parameters:

{
  "normalize": true
}

Figures

ValidMind Figure my_custom_tests.ConfusionMatrix:test_normalized_challenger:d0f5

2026-07-31 16:52:07,669 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:test_normalized_challenger does not exist in model's document

Use external test providers

Sometimes you may want to reuse the same set of custom tests across multiple records (models) and share them with others in your organization, like the development team would have done with you in this example workflow featured in this series of notebooks. In this case, you can create an external custom test provider that will allow you to load custom tests from a local folder or a Git repository.

In this section you will learn how to declare a local filesystem test provider that allows loading tests from a local folder following these high level steps:

Create a folder of custom tests from existing inline tests (tests that exist in your active Jupyter Notebook)
Save an inline test to a file
Define and register a LocalTestProvider that points to that folder
Run test provider tests
Add the test results to your documentation

Create custom tests folder

Let's start by creating a new folder that will contain reusable custom tests from your existing inline tests.

The following code snippet will create a new my_tests directory in the current working directory if it doesn't exist:

tests_folder = "my_tests"

import os

# create tests folder
os.makedirs(tests_folder, exist_ok=True)

# remove existing tests
for f in os.listdir(tests_folder):
    # remove files and pycache
    if f.endswith(".py") or f == "__pycache__":
        os.system(f"rm -rf {tests_folder}/{f}")

After running the command above, confirm that a new my_tests directory was created successfully. For example:

~/notebooks/tutorials/validation/my_tests/

Save an inline test

The @vm.test decorator we used in Implement a custom inline test above to register one-off custom tests also includes a convenience method on the function object that allows you to simply call <func_name>.save() to save the test to a Python file at a specified path.

While save() will get you started by creating the file and saving the function code with the correct name, it won't automatically include any imports, or other functions or variables, outside of the functions that are needed for the test to run. To solve this, pass in an optional imports argument ensuring necessary imports are added to the file.

The confusion_matrix test requires the following additional imports:

import matplotlib.pyplot as plt
from sklearn import metrics

Let's pass these imports to the save() method to ensure they are included in the file with the following command:

confusion_matrix.save(
    # Save it to the custom tests folder we created
    tests_folder,
    imports=["import matplotlib.pyplot as plt", "from sklearn import metrics"],
)

2026-07-31 16:52:08,297 - INFO(validmind.tests.decorator): Saved to /home/runner/work/documentation/documentation/site/notebooks/EXECUTED/validation/my_tests/ConfusionMatrix.py!Be sure to add any necessary imports to the top of the file.
2026-07-31 16:52:08,297 - INFO(validmind.tests.decorator): This metric can be run with the ID: <test_provider_namespace>.ConfusionMatrix

Confirm that the save() method saved the confusion_matrix function to a file named ConfusionMatrix.py in the my_tests folder.

Note that the new file provides some context on the origin of the test, which is useful for traceability:

# Saved from __main__.confusion_matrix
# Original Test ID: my_custom_tests.ConfusionMatrix
# New Test ID: <test_provider_namespace>.ConfusionMatrix

Additionally, the new test function has been stripped off its decorator, as it now resides in a file that will be loaded by the test provider:
```
def ConfusionMatrix(dataset, model, normalize=False):
```

Register a local test provider

Now that your my_tests folder has a sample custom test, let's initialize a test provider that will tell the ValidMind Library where to find your custom tests:

ValidMind offers out-of-the-box test providers for local tests (tests in a folder) or a Github provider for tests in a Github repository.
You can also create your own test provider by creating a class that has a load_test method that takes a test ID and returns the test function matching that ID.

Want to learn more about test providers?

An extended introduction to test providers can be found in: Integrate external test providers

Initialize a local test provider

For most use cases, using a LocalTestProvider that allows you to load custom tests from a designated directory should be sufficient.

The most important attribute for a test provider is its namespace. This is a string that will be used to prefix test IDs in documentation. This allows you to have multiple test providers with tests that can even share the same ID, but are distinguished by their namespace.

Let's go ahead and load the custom tests from our my_tests directory:

from validmind.tests import LocalTestProvider

# initialize the test provider with the tests folder we created earlier
my_test_provider = LocalTestProvider(tests_folder)

vm.tests.register_test_provider(
    namespace="my_test_provider",
    test_provider=my_test_provider,
)
# `my_test_provider.load_test()` will be called for any test ID that starts with `my_test_provider`
# e.g. `my_test_provider.ConfusionMatrix` will look for a function named `ConfusionMatrix` in `my_tests/ConfusionMatrix.py` file

Run test provider tests

Now that we've set up the test provider, we can run any test that's located in the tests folder by using the run_test() method as with any other test:

For tests that reside in a test provider directory, the test ID will be the namespace specified when registering the provider, followed by the path to the test file relative to the tests folder.
For example, the Confusion Matrix test we created earlier will have the test ID my_test_provider.ConfusionMatrix. You could organize the tests in subfolders, say classification and regression, and the test ID for the Confusion Matrix test would then be my_test_provider.classification.ConfusionMatrix.

Let's go ahead and re-run the confusion matrix test with our testing dataset for our two models by using the test ID my_test_provider.ConfusionMatrix. This should load the test from the test provider and run it as before.

# Champion with test dataset and test provider custom test
vm.tests.run_test(
    test_id="my_test_provider.ConfusionMatrix:champion",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_log_model]
    }
).log()

Confusion Matrix Champion

The Confusion Matrix test evaluates classification performance by comparing predicted labels against observed labels on the test dataset. The result is presented as a 2x2 matrix for log_model_champion, with counts shown for each actual/predicted class combination. The matrix contains 189 observations with true label False predicted as False, 114 with true label False predicted as True, 126 with true label True predicted as False, and 218 with true label True predicted as True.

Key insights:

True positives are the largest cell: The highest count in the matrix is 218 for observations with true label True predicted as True, indicating this is the most frequent prediction outcome in the test set.
Correct predictions exceed misclassifications: Diagonal counts total 407 (189 + 218), while off-diagonal counts total 240 (114 + 126), showing more correct classifications than errors overall.
False negatives exceed false positives slightly: Misclassified True cases predicted as False total 126, compared with 114 False cases predicted as True, indicating a modestly higher count of missed positive cases than incorrectly flagged positive cases.
Observed class support is slightly higher for True labels: The test set contains 344 observations with true label True (126 + 218) and 303 with true label False (189 + 114), indicating somewhat greater representation of the positive class in the evaluated sample.

The confusion matrix shows that the model produces more correct than incorrect classifications on the test dataset, with the largest concentration in correctly identified True cases. Errors are present in both directions and are relatively close in magnitude, with false negatives slightly higher than false positives. The evaluated sample also contains somewhat more True than False observations, which is reflected in the distribution of counts across the matrix.

Figures

ValidMind Figure my_test_provider.ConfusionMatrix:champion:046a

2026-07-31 16:52:14,549 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_test_provider.ConfusionMatrix:champion does not exist in model's document

# Challenger with test dataset  and test provider custom test
vm.tests.run_test(
    test_id="my_test_provider.ConfusionMatrix:challenger",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_rf_model]
    }
).log()

Confusion Matrix Challenger

The Confusion Matrix test evaluates classification performance by comparing predicted labels with true labels across the four outcome categories: true positives, true negatives, false positives, and false negatives. For the test_dataset_final results shown for rf_model, the matrix reports 210 true negatives, 93 false positives, 127 false negatives, and 217 true positives. The figure presents these counts by true label on the y-axis and predicted label on the x-axis, allowing direct comparison of correct and incorrect classifications for both classes.

Key insights:

Correct classifications are concentrated on the diagonal: The model records 210 true negatives and 217 true positives, compared with 93 false positives and 127 false negatives. This indicates that correct predictions exceed misclassifications for both classes.
False negatives exceed false positives: The number of false negatives is 127, while false positives total 93. Misclassification is therefore more pronounced for observations with true label True that were predicted as False.
Positive and negative classes are similarly represented: The true-label totals are 303 for False observations (210 + 93) and 344 for True observations (127 + 217). This indicates that both classes are materially represented in the evaluated test set.
True positive and true negative counts are similar: The model produces 217 true positives and 210 true negatives. The closeness of these counts shows comparable volumes of correct classification across the two predicted outcomes.

The confusion matrix shows that the model achieves more correct than incorrect classifications overall, with similar counts of true positives and true negatives. The main asymmetry in the error profile is the higher number of false negatives relative to false positives. Across the evaluated test set, both classes are represented at comparable scale, which supports direct interpretation of performance across positive and negative outcomes.

Figures

ValidMind Figure my_test_provider.ConfusionMatrix:challenger:8b17

2026-07-31 16:52:21,065 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_test_provider.ConfusionMatrix:challenger does not exist in model's document

Verify test runs

Our final task is to verify that all the tests provided by the development team were run and reported accurately. Note the appended result_ids to delineate which dataset we ran the test with for the relevant tests.

Here, we'll specify all the tests we'd like to independently rerun in a dictionary called test_config. Note here that inputs and input_grid expect the input_id of the dataset or model as the value rather than the variable name we specified:

test_config = {
    # Run with the raw dataset
    'validmind.data_validation.DatasetDescription:raw_data': {
        'inputs': {'dataset': 'raw_dataset'}
    },
    'validmind.data_validation.DescriptiveStatistics:raw_data': {
        'inputs': {'dataset': 'raw_dataset'}
    },
    'validmind.data_validation.MissingValues:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_percentage_threshold': 1}
    },
    'validmind.data_validation.ClassImbalance:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_percent_threshold': 10}
    },
    'validmind.data_validation.Duplicates:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_threshold': 1}
    },
    'validmind.data_validation.HighCardinality:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {
            'num_threshold': 100,
            'percent_threshold': 0.1,
            'threshold_type': 'percent'
        }
    },
    'validmind.data_validation.Skewness:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'max_threshold': 1}
    },
    'validmind.data_validation.UniqueRows:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_percent_threshold': 1}
    },
    'validmind.data_validation.TooManyZeroValues:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'max_percent_threshold': 0.03}
    },
    'validmind.data_validation.IQROutliersTable:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'threshold': 5}
    },
    # Run with the preprocessed dataset
    'validmind.data_validation.DescriptiveStatistics:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.TabularDescriptionTables:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.MissingValues:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'},
        'params': {'min_percentage_threshold': 1}
    },
    'validmind.data_validation.TabularNumericalHistograms:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.TargetRateBarPlots:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'},
        'params': {'default_column': 'loan_status'}
    },
    # Run with the training and test datasets
    'validmind.data_validation.DescriptiveStatistics:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.TabularDescriptionTables:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.ClassImbalance:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'min_percent_threshold': 10}
    },
    'validmind.data_validation.UniqueRows:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'min_percent_threshold': 1}
    },
    'validmind.data_validation.TabularNumericalHistograms:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.MutualInformation:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'min_threshold': 0.01}
    },
    'validmind.data_validation.PearsonCorrelationMatrix:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.HighPearsonCorrelation:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'max_threshold': 0.3, 'top_n_correlations': 10}
    },
    'validmind.model_validation.ModelMetadata': {
        'input_grid': {'model': ['log_model_champion', 'rf_model']}
    },
    'validmind.model_validation.sklearn.ModelParameters': {
        'input_grid': {'model': ['log_model_champion', 'rf_model']}
    },
    'validmind.model_validation.sklearn.ROCCurve': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final'], 'model': ['log_model_champion']}
    },
    'validmind.model_validation.sklearn.MinimumROCAUCScore': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final'], 'model': ['log_model_champion']},
        'params': {'min_threshold': 0.5}
    }
}

Then batch run and log our tests in test_config:

for t in test_config:
    print(t)
    try:
        # Check if test has input_grid
        if 'input_grid' in test_config[t]:
            # For tests with input_grid, pass the input_grid configuration
            if 'params' in test_config[t]:
                vm.tests.run_test(t, input_grid=test_config[t]['input_grid'], params=test_config[t]['params']).log()
            else:
                vm.tests.run_test(t, input_grid=test_config[t]['input_grid']).log()
        else:
            # Original logic for regular inputs
            if 'params' in test_config[t]:
                vm.tests.run_test(t, inputs=test_config[t]['inputs'], params=test_config[t]['params']).log()
            else:
                vm.tests.run_test(t, inputs=test_config[t]['inputs']).log()
    except Exception as e:
        print(f"Error running test {t}: {str(e)}")

validmind.data_validation.DatasetDescription:raw_data

Dataset Description Raw Data

The Dataset Description test evaluates column-level completeness, type classification, and value cardinality across the raw dataset. The results summarize 11 columns over 8,000 records, reporting each column’s inferred type, non-missing count, missingness, and number of distinct values. The dataset includes 5 numeric columns and 6 categorical columns, with all columns showing full population counts and no missing values. Distinct-value counts range from 2 for several binary categorical fields to 8,000 for EstimatedSalary.

Key insights:

No missing values observed: All 11 columns have a count of 8,000 and missingness of 0.0%, indicating complete population coverage for the analyzed dataset.
Mixed numeric and categorical structure: The dataset contains 5 numeric variables (CreditScore, Age, Tenure, Balance, NumOfProducts, EstimatedSalary includes six listed as numeric? Actually table shows 6 numeric) and 5 categorical predictors plus the categorical target Exited, reflecting a combination of continuous, discrete, and binary fields.
EstimatedSalary is fully unique: EstimatedSalary has 8,000 distinct values out of 8,000 records, corresponding to a distinct proportion of 1.0, making it the highest-cardinality field in the dataset.
Balance also shows high cardinality: Balance contains 5,088 distinct values, or 63.6% of all records, which is materially higher than the remaining non-unique fields aside from EstimatedSalary.
Several variables are low-cardinality: Geography has 3 distinct values, Gender, HasCrCard, IsActiveMember, and Exited each have 2, NumOfProducts has 4, and Tenure has 11, indicating multiple binary or limited-category fields.
CreditScore and Age have moderate diversity: CreditScore has 452 distinct values and Age has 69 distinct values, placing both between the highly continuous fields and the low-cardinality discrete variables.

The results show a complete raw dataset with no missing values across all 8,000 observations and 11 analyzed columns. Column structure combines high-cardinality numeric fields, moderate-diversity numeric attributes, and several binary or limited-category variables. The most prominent distributional characteristic in the summary is the contrast between fully unique or highly distinct numeric fields such as EstimatedSalary and Balance and the low-cardinality categorical and discrete variables such as Gender, HasCrCard, IsActiveMember, Exited, Geography, and Tenure.

Tables

Dataset Description

Name	Type	Count	Distinct	Distinct %
CreditScore	Numeric	8000.0	452	0.0565
Geography	Categorical	8000.0	3	0.0004
Gender	Categorical	8000.0	2	0.0002
Age	Numeric	8000.0	69	0.0086
Tenure	Numeric	8000.0	11	0.0014
Balance	Numeric	8000.0	5088	0.6360
NumOfProducts	Numeric	8000.0	4	0.0005
HasCrCard	Categorical	8000.0	2	0.0002
IsActiveMember	Categorical	8000.0	2	0.0002
EstimatedSalary	Numeric	8000.0	8000	1.0000
Exited	Categorical	8000.0	2	0.0002

2026-07-31 16:52:28,909 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DatasetDescription:raw_data does not exist in model's document

validmind.data_validation.DescriptiveStatistics:raw_data

Descriptive Statistics Raw Data

The Descriptive Statistics test evaluates the distributional characteristics of numerical and categorical variables in the raw dataset. The results summarize central tendency, dispersion, range, and percentile structure for eight numerical variables, and frequency concentration for two categorical variables. All reported variables have a count of 8,000 observations, and the categorical summary shows the number of unique values, the most frequent category, and its share of the sample.

Key insights:

Complete coverage across variables: Each numerical and categorical variable reports a count of 8,000, indicating that all summarized fields contain the same number of observations in this result set.
Balance shows strong concentration at zero: Balance has a minimum of 0, a 25th percentile of 0, and a median of 97,264, with mean 76,434.10 and standard deviation 62,612.25. This pattern indicates a substantial share of observations at zero alongside a wide spread among nonzero values.
CreditScore and Age are broadly distributed: CreditScore ranges from 350 to 850 with mean 650.16 and median 652, while Age ranges from 18 to 92 with mean 38.95 and median 37. The closeness of mean and median in CreditScore contrasts with a somewhat higher upper tail in Age, where the 95th percentile is 60 and the maximum is 92.
Tenure is bounded and centered mid-range: Tenure spans from 0 to 10, with mean 5.03 and median 5. Percentiles are concentrated within the bounded range, with the 90th and 95th percentiles both at 9.
Product holdings are concentrated at low counts: NumOfProducts has mean 1.53, median 1, 75th percentile 2, and maximum 4. The distribution is concentrated in the lower product counts, with most observations at 1 or 2 products.
Binary indicators are unevenly distributed: HasCrCard has mean 0.7026, indicating approximately 70.26% of observations in the value 1 category, while IsActiveMember has mean 0.5199, indicating a near-even split with a slight majority at 1.
Categorical concentration is moderate: Geography contains 3 unique values, with France as the top category at 4,010 observations or 50.12% of the sample. Gender contains 2 unique values, with Male as the top category at 4,396 observations or 54.95%.

The descriptive statistics show a dataset with consistent observation counts across all summarized variables and a mix of bounded, binary, continuous, and categorical fields. The most distinct distributional feature is Balance, where the lower quartile is zero despite a much higher median and a wide dispersion. Other variables such as CreditScore, Age, Tenure, and EstimatedSalary exhibit broad but clearly defined ranges, while categorical variables show moderate concentration in their most frequent classes rather than near-total dominance.

Tables

Numerical Variables

Name	Count	Mean	Std	Min	25%	50%	75%	90%	95%	Max
CreditScore	8000.0	650.1596	96.8462	350.0	583.0	652.0	717.0	778.0	813.0	850.0
Age	8000.0	38.9489	10.4590	18.0	32.0	37.0	44.0	53.0	60.0	92.0
Tenure	8000.0	5.0339	2.8853	0.0	3.0	5.0	8.0	9.0	9.0	10.0
Balance	8000.0	76434.0965	62612.2513	0.0	0.0	97264.0	128045.0	149545.0	162488.0	250898.0
NumOfProducts	8000.0	1.5325	0.5805	1.0	1.0	1.0	2.0	2.0	2.0	4.0
HasCrCard	8000.0	0.7026	0.4571	0.0	0.0	1.0	1.0	1.0	1.0	1.0
IsActiveMember	8000.0	0.5199	0.4996	0.0	0.0	1.0	1.0	1.0	1.0	1.0
EstimatedSalary	8000.0	99790.1880	57520.5089	12.0	50857.0	99505.0	149216.0	179486.0	189997.0	199992.0

Categorical Variables

Name	Count	Number of Unique Values	Top Value	Top Value Frequency	Top Value Frequency %
Geography	8000.0	3.0	France	4010.0	50.12
Gender	8000.0	2.0	Male	4396.0	54.95

2026-07-31 16:52:36,476 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DescriptiveStatistics:raw_data does not exist in model's document

validmind.data_validation.MissingValues:raw_data

✅ Missing Values Raw Data

The Missing Values test evaluates dataset completeness by measuring the percentage of missing values in each feature against the configured 1% threshold. The result table reports the number and percentage of missing values for each column in the raw dataset, along with a pass/fail outcome based on that threshold. Across the 11 evaluated columns, all entries show 0 missing values and 0.0% missingness, resulting in a Pass status for every feature.

Key insights:

No missing values detected: All 11 columns, including CreditScore, Geography, Gender, Age, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary, and Exited, show 0 missing values and 0.0% missingness.
All features pass threshold: Every evaluated feature received a Pass outcome under the configured minimum percentage threshold of 1%.
Uniform completeness across dataset: Missingness results are consistent across all columns, with no variation in missing value counts or percentages.

The test result shows complete observed data coverage across all evaluated raw-data features under the applied missingness criterion. No column exceeds the 1% threshold, and the missing-value profile is uniformly zero across the dataset. This indicates that, within the scope of this test, dataset completeness is consistent across all recorded variables.

Parameters:

{
  "min_percentage_threshold": 1
}

Tables

Column	Number of Missing Values	Percentage of Missing Values (%)	Pass/Fail
CreditScore	0	0.0	Pass
Geography	0	0.0	Pass
Gender	0	0.0	Pass
Age	0	0.0	Pass
Tenure	0	0.0	Pass
Balance	0	0.0	Pass
NumOfProducts	0	0.0	Pass
HasCrCard	0	0.0	Pass
IsActiveMember	0	0.0	Pass
EstimatedSalary	0	0.0	Pass
Exited	0	0.0	Pass

2026-07-31 16:52:41,202 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.MissingValues:raw_data does not exist in model's document

validmind.data_validation.ClassImbalance:raw_data

✅ Class Imbalance Raw Data

The Class Imbalance test evaluates the distribution of target classes in the dataset by measuring each class’s share of total records against the configured minimum threshold. For the Exited target, the results show two classes: class 0 represents 79.80% of rows and class 1 represents 20.20% of rows. The reported threshold is 10%, and the table records a pass/fail outcome for each class alongside its observed percentage.

Key insights:

Both classes pass threshold: Class 0 at 79.80% and class 1 at 20.20% both exceed the 10% minimum percentage threshold, and both are marked as Pass.
Majority class is class 0: The target distribution is concentrated in class 0, which accounts for 79.80% of observations, compared with 20.20% for class 1.
Minority class remains materially represented: Although class 1 is the smaller class, its observed share of 20.20% remains above the configured cutoff used in this test.

The result indicates that the target classes are not flagged by this test as falling below the minimum representation threshold. The observed distribution is uneven, with class 0 comprising the clear majority of records, while class 1 retains a substantial share of the dataset. Under the configured 10% criterion, both target classes satisfy the test condition.

Parameters:

{
  "min_percent_threshold": 10
}

Tables

Exited Class Imbalance

Exited	Percentage of Rows (%)	Pass/Fail
0	79.80%	Pass
1	20.20%	Pass

Figures

ValidMind Figure validmind.data_validation.ClassImbalance:raw_data:2d21

2026-07-31 16:52:49,533 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.ClassImbalance:raw_data does not exist in model's document

validmind.data_validation.Duplicates:raw_data

✅ Duplicates Raw Data

The Duplicates test evaluates whether the dataset contains exact duplicate rows that could affect data quality and downstream model training. The results table reports the count and share of duplicate rows identified in the raw dataset. For this test run, the table shows 0 duplicate rows, corresponding to 0.0% of all rows evaluated, against a configured minimum threshold parameter of 1.

Key insights:

No duplicate rows detected: The test identified 0 duplicate rows in the dataset, indicating that no exact row-level duplication was observed in the evaluated data.
Duplicate share is zero: The percentage of duplicate rows is reported as 0.0%, showing that duplicate observations did not contribute to the dataset composition in this test run.
Result is below threshold parameter: The observed duplicate count of 0 is below the configured min_threshold value of 1.

The duplicate row check indicates that exact row-level duplication was not present in the evaluated raw dataset. Both the absolute count and percentage measures are zero, and the reported duplicate count is below the configured threshold parameter. Collectively, the result documents an absence of observed exact duplicates in this test execution.

Parameters:

{
  "min_threshold": 1
}

Tables

Duplicate Rows Results for Dataset

Number of Duplicates	Percentage of Rows (%)
0	0.0

2026-07-31 16:52:55,607 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.Duplicates:raw_data does not exist in model's document

validmind.data_validation.HighCardinality:raw_data

✅ High Cardinality Raw Data

The High Cardinality test evaluates the number of unique values in categorical columns to identify features with large counts of distinct categories. In this result, the table reports the number and percentage of distinct values for each categorical column alongside the pass/fail outcome against the configured threshold. Two categorical columns were evaluated: Geography with 3 distinct values and 0.0375% distinctness, and Gender with 2 distinct values and 0.025% distinctness. Both columns are recorded as passing the test.

Key insights:

No categorical columns failed: Both evaluated categorical features, Geography and Gender, have a Pass status in the result table, indicating that neither exceeded the applied high-cardinality threshold.
Distinct counts are very low: Geography contains 3 distinct values and Gender contains 2 distinct values, reflecting a small number of categories in each evaluated column.
Distinct percentages remain minimal: The percentage of distinct values is 0.0375% for Geography and 0.025% for Gender, showing very limited uniqueness relative to the dataset size.

The result indicates that the categorical fields assessed in this test exhibit low cardinality under the configured threshold. Both the absolute distinct counts and the reported distinct-value percentages are small, and no categorical feature was flagged by the test. Overall, the evaluated categorical inputs do not show evidence of high-cardinality behavior in this result.

Parameters:

{
  "num_threshold": 100,
  "percent_threshold": 0.1,
  "threshold_type": "percent"
}

Tables

Column	Number of Distinct Values	Percentage of Distinct Values (%)	Pass/Fail
Geography	3	0.0375	Pass
Gender	2	0.0250	Pass

2026-07-31 16:53:00,344 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.HighCardinality:raw_data does not exist in model's document

validmind.data_validation.Skewness:raw_data

❌ Skewness Raw Data

The Skewness test evaluates the asymmetry of numerical feature distributions against the configured maximum threshold of 1. The results table reports skewness values and pass/fail status for nine numeric columns in the raw dataset. Seven columns pass the threshold, while two columns exceed it. Observed skewness values range from -0.8867 to 1.4847 across the evaluated fields.

Key insights:

Two columns exceed threshold: Age records skewness of 1.0245 and Exited records skewness of 1.4847, making them the only variables that fail the maximum threshold of 1.
Most variables show limited skewness: Seven of nine numeric columns pass the test, including CreditScore (-0.062), Tenure (0.0077), Balance (-0.1353), IsActiveMember (-0.0796), and EstimatedSalary (0.0095), all of which are close to zero.
Exited is the most skewed variable: Exited has the largest absolute skewness value at 1.4847, exceeding the threshold by the widest margin among all evaluated columns.
Negative skewness is present but within limit: HasCrCard shows the strongest negative skewness at -0.8867, but it remains within the configured threshold and therefore passes.
Moderate positive skewness appears in NumOfProducts: NumOfProducts has skewness of 0.7172, indicating asymmetry below the failure threshold but higher than most other passing variables.

The results indicate that skewness is limited across most numeric columns in the dataset, with seven variables remaining within the configured threshold. The primary exceptions are Age and Exited, both of which display positive skewness above 1, with Exited showing the largest deviation overall. Aside from these two fields, the remaining variables exhibit either near-symmetric distributions or moderate asymmetry that remains within the test limit.

Parameters:

{
  "max_threshold": 1
}

Tables

Skewness Results for Dataset

Column	Skewness	Pass/Fail
CreditScore	-0.0620	Pass
Age	1.0245	Fail
Tenure	0.0077	Pass
Balance	-0.1353	Pass
NumOfProducts	0.7172	Pass
HasCrCard	-0.8867	Pass
IsActiveMember	-0.0796	Pass
EstimatedSalary	0.0095	Pass
Exited	1.4847	Fail

2026-07-31 16:53:06,132 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.Skewness:raw_data does not exist in model's document

validmind.data_validation.UniqueRows:raw_data

❌ Unique Rows Raw Data

The UniqueRows test evaluates dataset diversity by comparing the proportion of unique values in each column against the configured minimum threshold of 1%. The results table reports the number and percentage of unique values for each column together with a pass/fail outcome. Across the 11 evaluated columns, 3 columns pass the test and 8 fail, with observed uniqueness percentages ranging from 0.025% to 100.0%.

Key insights:

Pass results are concentrated: Only CreditScore, Balance, and EstimatedSalary exceed the 1% minimum threshold. Their uniqueness levels are 5.65%, 63.6%, and 100.0%, respectively.
EstimatedSalary is fully unique: EstimatedSalary has 8,000 unique values out of 8,000 rows, corresponding to 100.0% uniqueness, which is the highest observed result in the dataset.
Balance shows substantial variation: Balance records 5,088 unique values, equal to 63.6% uniqueness, making it the second most diverse column in the test output.
Most columns fall materially below threshold: Eight columns fail the test, with uniqueness percentages of 0.8625% or lower. These include Age at 0.8625%, Tenure at 0.1375%, NumOfProducts at 0.05%, Geography at 0.0375%, and Gender, HasCrCard, IsActiveMember, and Exited each at 0.025%.
Very low-cardinality columns are prominent: Gender, HasCrCard, IsActiveMember, and Exited each contain 2 unique values, while Geography contains 3 and NumOfProducts contains 4. These columns contribute the lowest uniqueness ratios reported by the test.

The results indicate that uniqueness is unevenly distributed across the raw data columns. Diversity above the 1% threshold is observed in EstimatedSalary, Balance, and CreditScore, while the remaining columns exhibit lower-cardinality value structures and do not meet the configured threshold. Overall, the test output shows a dataset containing a small set of highly unique numerical fields alongside a larger set of columns with limited distinct values.

Parameters:

{
  "min_percent_threshold": 1
}

Tables

Column	Number of Unique Values	Percentage of Unique Values (%)	Pass/Fail
CreditScore	452	5.6500	Pass
Geography	3	0.0375	Fail
Gender	2	0.0250	Fail
Age	69	0.8625	Fail
Tenure	11	0.1375	Fail
Balance	5088	63.6000	Pass
NumOfProducts	4	0.0500	Fail
HasCrCard	2	0.0250	Fail
IsActiveMember	2	0.0250	Fail
EstimatedSalary	8000	100.0000	Pass
Exited	2	0.0250	Fail

2026-07-31 16:53:12,122 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.UniqueRows:raw_data does not exist in model's document

validmind.data_validation.TooManyZeroValues:raw_data

❌ Too Many Zero Values Raw Data

The TooManyZeroValues test evaluates numerical columns for zero-value prevalence above the configured threshold of 0.03%. The result table reports row count, number of zero values, percentage of zero values, and pass/fail status for each evaluated numerical variable. Four variables are listed in the output, and each exceeds the threshold, resulting in a fail status. Reported zero-value percentages range from 4.0375% to 48.0125% across the tested columns.

Key insights:

All evaluated variables failed: Each of the four reported numerical variables exceeded the 0.03% threshold and was flagged as Fail in the test output.
IsActiveMember has the highest zero share: IsActiveMember contains 3,841 zero values out of 8,000 rows, corresponding to 48.0125%, which is the highest proportion among the evaluated variables.
Balance shows substantial zero concentration: Balance contains 2,912 zero values, representing 36.4% of observations and indicating a large concentration of zeros in this field.
Zero prevalence varies materially across variables: Tenure has the lowest reported zero percentage at 4.0375% (323 zeros), while HasCrCard and IsActiveMember show materially higher rates at 29.7375% and 48.0125%, respectively.

The test output shows that zero values are present above the configured threshold in every evaluated numerical variable. The largest concentrations appear in IsActiveMember and Balance, followed by HasCrCard, while Tenure has the lowest zero proportion but still fails the test. Collectively, the results indicate that zero-value prevalence is widespread across the reported numerical fields rather than isolated to a single variable.

Parameters:

{
  "max_percent_threshold": 0.03
}

Tables

Variable	Row Count	Number of Zero Values	Percentage of Zero Values (%)	Pass/Fail
Tenure	8000	323	4.0375	Fail
Balance	8000	2912	36.4000	Fail
HasCrCard	8000	2379	29.7375	Fail
IsActiveMember	8000	3841	48.0125	Fail

2026-07-31 16:53:17,104 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TooManyZeroValues:raw_data does not exist in model's document

validmind.data_validation.IQROutliersTable:raw_data

IQR Outliers Table Raw Data

The Interquartile Range Outliers Table test evaluates numerical features for observations outside the IQR-based bounds and summarizes any detected outliers. In this test run, the result table titled Summary of Outliers Detected by IQR Method contains no rows. The output therefore indicates that no outlier summary entries were produced in the reported results.

Key insights:

No outlier rows reported: The outlier summary table is empty, with no numerical features listed and no corresponding outlier statistics shown.
No feature-level outlier detail available: Because the table contains no entries, there are no reported counts or percentile summaries for individual features.

The reported result consists of an empty outlier summary table. Based on the provided output, the test does not present any feature-level outlier findings or summary statistics for this run.

Parameters:

{
  "threshold": 5
}

Tables

Summary of Outliers Detected by IQR Method

2026-07-31 16:53:20,067 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.IQROutliersTable:raw_data does not exist in model's document

validmind.data_validation.DescriptiveStatistics:preprocessed_data

Descriptive Statistics Preprocessed Data

The Descriptive Statistics test evaluates the distributional characteristics of numerical and categorical variables in the preprocessed dataset. The results summarize 3,232 observations across seven numerical variables and two categorical variables using counts, central tendency, dispersion, percentiles, and category frequency information. The numerical table shows the range and percentile structure for variables such as CreditScore, Balance, Tenure, and EstimatedSalary, while the categorical table reports the number of unique values and the most frequent category for Geography and Gender.

Key insights:

Full population coverage across variables: All reported variables have a count of 3,232, indicating consistent record coverage in both the numerical and categorical summaries.
Balance shows pronounced lower-tail concentration: Balance has a minimum of 0 and a 25th percentile of 0, while the median is 103,081 and the mean is 81,530.139. This indicates that at least one quarter of observations are at zero, alongside a substantial concentration of higher positive balances.
NumOfProducts is concentrated at low values: NumOfProducts has a mean of 1.5121, a median of 1, a 75th percentile of 2, a 95th percentile of 3, and a maximum of 4. The distribution is concentrated in the lower product counts, with most observations between 1 and 2 products.
Binary indicators are unevenly distributed: HasCrCard has a mean of 0.7064, indicating a larger share of records with value 1 than 0, while IsActiveMember has a mean of 0.4666 and a median of 0, indicating a slightly larger share of records with value 0 than 1.
Categorical variables show moderate concentration: Geography contains 3 unique values, with France as the top category at 1,456 observations (45.05%). Gender contains 2 unique values, with Male as the top category at 1,651 observations (51.08%), indicating limited dominance by the leading category in both fields.

The descriptive summary shows complete coverage across all reported variables and a mix of continuous, discrete, and binary feature distributions. The most notable concentration appears in Balance, where zero values account for at least the lowest quartile despite a substantially higher median and upper-percentile values. Other variables, including NumOfProducts, HasCrCard, IsActiveMember, Geography, and Gender, display bounded and moderately concentrated distributions without a single category dominating the categorical fields.

Tables

Numerical Variables

Name	Count	Mean	Std	Min	25%	50%	75%	90%	95%	Max
CreditScore	3232.0	646.2995	98.1349	350.0	580.0	647.0	714.0	777.0	813.0	850.0
Tenure	3232.0	5.0019	2.8869	0.0	3.0	5.0	7.0	9.0	10.0	10.0
Balance	3232.0	81530.1390	61461.2329	0.0	0.0	103081.0	129044.0	149699.0	163704.0	250898.0
NumOfProducts	3232.0	1.5121	0.6707	1.0	1.0	1.0	2.0	2.0	3.0	4.0
HasCrCard	3232.0	0.7064	0.4555	0.0	0.0	1.0	1.0	1.0	1.0	1.0
IsActiveMember	3232.0	0.4666	0.4990	0.0	0.0	0.0	1.0	1.0	1.0	1.0
EstimatedSalary	3232.0	100465.9283	57714.9846	12.0	50936.0	101089.0	150043.0	179704.0	190182.0	199909.0

Categorical Variables

Name	Count	Number of Unique Values	Top Value	Top Value Frequency	Top Value Frequency %
Geography	3232.0	3.0	France	1456.0	45.05
Gender	3232.0	2.0	Male	1651.0	51.08

2026-07-31 16:53:26,930 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DescriptiveStatistics:preprocessed_data does not exist in model's document

validmind.data_validation.TabularDescriptionTables:preprocessed_data

Tabular Description Tables Preprocessed Data

The Descriptive Statistics test evaluates the composition, completeness, and basic distributional properties of the preprocessed dataset across numerical and categorical variables. The results summarize eight numerical variables and two categorical variables over 3,232 observations, reporting counts, means, minimum and maximum values, missingness, data types, and categorical cardinality. All listed variables show 0.0% missing values, and the tables provide the observed ranges for continuous and indicator fields together with the unique category values for Geography and Gender.

Key insights:

No missing values observed: All numerical and categorical variables report 0.0% missing values across 3,232 observations, indicating complete population coverage in the summarized preprocessed dataset.
Binary indicators are consistently encoded numerically: HasCrCard, IsActiveMember, and Exited are stored as int64 with observed ranges from 0 to 1. Their means are 0.7064, 0.4666, and 0.5000 respectively, reflecting the observed class proportions in the dataset.
Exited is evenly balanced: The target variable Exited has a mean of 0.5 with minimum 0 and maximum 1, indicating an equal split between the two observed outcome classes in this sample.
Continuous variables span wide ranges: Balance ranges from 0.0 to 250,898.09 with a mean of 81,530.139, while EstimatedSalary ranges from 11.58 to 199,909.32 with a mean of 100,465.9283. CreditScore spans 350 to 850 with a mean of 646.2995.
Categorical fields have low cardinality: Geography contains 3 unique values (Germany, Spain, France) and Gender contains 2 unique values (Female, Male), with both fields stored as object and showing no missing values.

The descriptive statistics indicate that the preprocessed dataset is fully populated for the variables reported and uses consistent data types across numerical and categorical fields. The numerical features include both bounded indicator variables and wider-range continuous variables, while the categorical structure is limited to two low-cardinality fields. The target variable is balanced in the summarized sample, and the tables provide a clear baseline view of feature coverage, encoding, and observed value ranges.

Tables

Numerical Variable	Num of Obs	Mean	Min	Max	Data Type
CreditScore	3232	646.2995	350.00	850.00	int64
Tenure	3232	5.0019	0.00	10.00	int64
Balance	3232	81530.1390	0.00	250898.09	float64
NumOfProducts	3232	1.5121	1.00	4.00	int64
HasCrCard	3232	0.7064	0.00	1.00	int64
IsActiveMember	3232	0.4666	0.00	1.00	int64
EstimatedSalary	3232	100465.9283	11.58	199909.32	float64
Exited	3232	0.5000	0.00	1.00	int64

Categorical Variable	Num of Obs	Num of Unique Values	Unique Values	Missing Values (%)	Data Type
Geography	3232.0	3.0	['Germany' 'Spain' 'France']	0.0	object
Gender	3232.0	2.0	['Female' 'Male']	0.0	object

2026-07-31 16:53:32,863 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularDescriptionTables:preprocessed_data does not exist in model's document

validmind.data_validation.MissingValues:preprocessed_data

✅ Missing Values Preprocessed Data

The Missing Values test evaluates dataset completeness by measuring the proportion of missing values in each feature against the configured 1% threshold. The results table reports the number and percentage of missing values for each column in the preprocessed dataset, along with a pass/fail outcome for the threshold check. Ten features are included in the output: CreditScore, Geography, Gender, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary, and Exited. For each of these features, the recorded missing-value count is 0 and the missing-value percentage is 0.0%.

Key insights:

No missing values detected: All 10 reported features show 0 missing values and 0.0% missingness in the test output.
All features passed threshold: Every column received a Pass result under the configured 1% missing-value threshold.
Completeness is uniform across variables: Missingness does not vary by feature in the reported dataset; each listed variable has the same observed result of zero missing values.

The test results show complete absence of missing values across all reported features in the preprocessed dataset. Because every column records 0.0% missingness, all features pass the configured threshold without exception. Collectively, the output indicates uniform dataset completeness for the variables included in this test.

Parameters:

{
  "min_percentage_threshold": 1
}

Tables

Column	Number of Missing Values	Percentage of Missing Values (%)	Pass/Fail
CreditScore	0	0.0	Pass
Geography	0	0.0	Pass
Gender	0	0.0	Pass
Tenure	0	0.0	Pass
Balance	0	0.0	Pass
NumOfProducts	0	0.0	Pass
HasCrCard	0	0.0	Pass
IsActiveMember	0	0.0	Pass
EstimatedSalary	0	0.0	Pass
Exited	0	0.0	Pass

2026-07-31 16:53:37,075 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.MissingValues:preprocessed_data does not exist in model's document

validmind.data_validation.TabularNumericalHistograms:preprocessed_data

Tabular Numerical Histograms Preprocessed Data

The TabularNumericalHistograms test evaluates the distribution of numerical input features by plotting a histogram for each variable. The results show histograms for CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, and EstimatedSalary. These plots provide a visual view of the concentration, spread, and discreteness of each feature, including continuous variables such as CreditScore, Balance, and EstimatedSalary, as well as integer and binary variables such as Tenure, NumOfProducts, HasCrCard, and IsActiveMember.

Key insights:

CreditScore is broadly bell-shaped: CreditScore values are concentrated in the mid-range, with the highest density around the low-600s to high-600s. Frequencies taper on both sides toward the lower and upper ends of the observed range.
Tenure is concentrated across integer levels: Tenure appears as a discrete distribution over values from 0 to 10, with most interior tenure values showing similar frequencies. The edge values, particularly 0 and 10, are visibly lower than the central tenure levels.
Balance shows a strong zero spike: Balance includes a very large concentration at or near zero, alongside a separate broad concentration centered roughly between 100k and 140k. This produces a distinctly non-uniform distribution with a dominant mass at zero and a second elevated region at positive balances.
NumOfProducts is concentrated at lower counts: NumOfProducts is discrete and heavily concentrated at 1 and 2, with substantially fewer observations at 3 and only a small number at 4. The distribution is strongly weighted toward the lowest two product counts.
Binary features are imbalanced to different degrees: HasCrCard shows a higher count at 1 than at 0, while IsActiveMember appears more balanced, though still with somewhat more observations at 0 than at 1. Both variables are represented as two-point distributions at 0 and 1.
EstimatedSalary is approximately even across its range: EstimatedSalary is distributed across the full displayed range up to about 200k with relatively similar bar heights across bins. No single region dominates the distribution to the same extent seen in Balance or NumOfProducts.

Overall, the histograms show that the numerical inputs include a mix of continuous, discrete, and binary distributions with materially different shapes across features. The most prominent distributional characteristics are the pronounced mass at zero in Balance, the concentration of NumOfProducts at 1 and 2, and the relatively even spread of EstimatedSalary across its range. CreditScore is centered in the middle of its observed range, while Tenure and the binary indicators exhibit discrete support with varying degrees of class imbalance.

Figures

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:c0c5

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:887e

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:27f8

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:549c

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:389d

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:2529

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:preprocessed_data:f534

2026-07-31 16:54:10,502 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularNumericalHistograms:preprocessed_data does not exist in model's document

validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data

Tabular Categorical Bar Plots Preprocessed Data

The TabularCategoricalBarPlots test evaluates the composition of categorical features by displaying category counts for each categorical variable in the preprocessed dataset. The result consists of separate bar plots for Geography and Gender, with bar heights representing the number of records in each category. In Geography, the categories shown are France, Germany, and Spain, while Gender contains Male and Female. The plots provide a direct view of relative category frequencies across these categorical inputs.

Key insights:

France is the largest geography category: In Geography, France has the highest count, at approximately 1,450 observations, compared with about 1,000 for Germany and about 800 for Spain.
Geography distribution is uneven: The three Geography categories are not represented uniformly, with France materially more frequent than Spain and moderately more frequent than Germany.
Gender distribution is nearly balanced: In Gender, Male and Female counts are close in magnitude, at approximately 1,650 and 1,580 observations respectively, indicating limited imbalance between the two categories.
Category cardinality remains low: The displayed categorical features contain a small number of categories, with three levels for Geography and two levels for Gender.

The categorical composition shown in the preprocessed dataset is characterized by moderate imbalance across Geography and near parity across Gender. The most pronounced concentration appears in the France category, while Gender remains evenly distributed between its two levels. Overall, the plotted features exhibit low category cardinality with clearly interpretable category counts.

Figures

ValidMind Figure validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data:ee5a

ValidMind Figure validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data:9967

2026-07-31 16:54:36,024 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data does not exist in model's document

validmind.data_validation.TargetRateBarPlots:preprocessed_data

Target Rate Bar Plots Preprocessed Data

The TargetRateBarPlots test evaluates categorical feature distributions and their associated positive-class rates. The result displays paired bar plots for the categorical features Geography and Gender, with one plot showing category counts and the other showing target rates by category. For Geography, the categories shown are France, Germany, and Spain; for Gender, the categories shown are Male and Female. The plots allow direct comparison of category prevalence alongside the corresponding target-rate differences within each feature.

Key insights:

Germany has the highest geography target rate: Among geographic categories, Germany shows the highest target rate at approximately 0.66, while France and Spain are both materially lower at roughly 0.43 and 0.42, respectively.
France is the most frequent geography: France has the largest observation count at about 1,450, followed by Germany at around 1,000 and Spain at roughly 800, indicating uneven category representation across Geography.
Female has higher target rate: Within Gender, Female shows a target rate of approximately 0.56 compared with about 0.43 for Male, indicating a visible separation in positive-class incidence between the two categories.
Gender counts are relatively balanced: Male and Female counts are close in size, at roughly 1,650 and 1,580 respectively, so the observed target-rate difference is not accompanied by a large difference in category frequency.

The results show observable variation in target rates across both categorical features included in the test. Geography exhibits the largest separation, with Germany materially above France and Spain despite having a smaller count than France. Gender shows a clearer count balance, with Female having a higher target rate than Male. Collectively, the plots indicate that category-level target incidence differs across the displayed categorical segments rather than remaining uniform across groups.

Figures

ValidMind Figure validmind.data_validation.TargetRateBarPlots:preprocessed_data:7384

ValidMind Figure validmind.data_validation.TargetRateBarPlots:preprocessed_data:f58d

2026-07-31 16:54:52,784 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TargetRateBarPlots:preprocessed_data does not exist in model's document

validmind.data_validation.DescriptiveStatistics:development_data

Descriptive Statistics Development Data

The Descriptive Statistics test evaluates the distributional characteristics of numerical variables in the dataset. The results present summary statistics for seven variables across train_dataset_final and test_dataset_final, including count, central tendency, dispersion, and percentile values from the minimum through the maximum. The training dataset contains 2,585 records and the test dataset contains 647 records, with each reported variable populated to the full dataset count. The tables allow direct comparison of variable ranges and distribution summaries between the two data splits.

Key insights:

Train and test summaries are closely aligned: For CreditScore, Tenure, Balance, HasCrCard, IsActiveMember, and EstimatedSalary, the train and test means, medians, and interquartile ranges are similar. For example, CreditScore medians are 646 in training and 649 in testing, while EstimatedSalary medians are 100,637 and 102,530 respectively.
Balance shows substantial dispersion: Balance has a standard deviation of 61,642 in training and 60,741 in testing, relative to means of 81,043 and 83,478. The variable also spans from 0 at the minimum to 250,898 in training and 214,347 in testing, indicating a wide spread of observed values.
Balance includes a zero lower quartile: The 25th percentile for Balance is 0 in both training and testing, while the medians are 102,365 and 105,603 respectively. This indicates that at least one quarter of observations have zero balance in both datasets, with the distribution shifting upward above that point.
NumOfProducts is concentrated at lower values: NumOfProducts has a median of 1 and a 75th percentile of 2 in both datasets. The means are 1.5269 in training and 1.4529 in testing, with observed values ranging from 1 to 4.
Binary indicators show uneven class proportions: HasCrCard has means of 0.7044 in training and 0.7141 in testing, while IsActiveMember has means of 0.4708 and 0.4498. Since both variables are bounded between 0 and 1 with medians of 1 for HasCrCard and 0 for IsActiveMember, the summaries show different prevalence levels for the two indicators.
CreditScore and salary appear broadly distributed: CreditScore spans 350 to 850 in both datasets with standard deviations near 98 to 99, and EstimatedSalary spans nearly the full reported range up to about 200,000 with standard deviations around 57,000 to 58,000. Median values for both variables remain close to their corresponding means across train and test.

The descriptive statistics show that the numerical feature distributions in the training and test datasets are generally consistent at the level of means, medians, quartiles, and standard deviations. The most pronounced spread is observed in Balance, which combines a zero first quartile with large upper-percentile values and high dispersion. NumOfProducts is concentrated in the lower observed values, while the binary indicators reflect different underlying prevalence rates. Overall, the reported summaries indicate similar distributional structure across the two dataset splits for the variables tested.

Tables

dataset	Name	Count	Mean	Std	Min	25%	50%	75%	90%	95%	Max
train_dataset_final	CreditScore	2585.0	646.4685	97.9222	350.0	580.0	646.0	714.0	778.0	814.0	850.0
train_dataset_final	Tenure	2585.0	5.0240	2.8793	0.0	3.0	5.0	7.0	9.0	10.0	10.0
train_dataset_final	Balance	2585.0	81042.5266	61642.1743	0.0	0.0	102365.0	129109.0	150052.0	163749.0	250898.0
train_dataset_final	NumOfProducts	2585.0	1.5269	0.6759	1.0	1.0	1.0	2.0	2.0	3.0	4.0
train_dataset_final	HasCrCard	2585.0	0.7044	0.4564	0.0	0.0	1.0	1.0	1.0	1.0	1.0
train_dataset_final	IsActiveMember	2585.0	0.4708	0.4992	0.0	0.0	0.0	1.0	1.0	1.0	1.0
train_dataset_final	EstimatedSalary	2585.0	100194.1192	57792.4063	12.0	50369.0	100637.0	149946.0	179508.0	190290.0	199909.0
test_dataset_final	CreditScore	647.0	645.6244	99.0537	350.0	580.0	649.0	718.0	771.0	808.0	850.0
test_dataset_final	Tenure	647.0	4.9134	2.9176	0.0	2.0	5.0	7.0	9.0	9.0	10.0
test_dataset_final	Balance	647.0	83478.3276	60741.0889	0.0	0.0	105603.0	128842.0	148275.0	159741.0	214347.0
test_dataset_final	NumOfProducts	647.0	1.4529	0.6469	1.0	1.0	1.0	2.0	2.0	3.0	4.0
test_dataset_final	HasCrCard	647.0	0.7141	0.4522	0.0	0.0	1.0	1.0	1.0	1.0	1.0
test_dataset_final	IsActiveMember	647.0	0.4498	0.4979	0.0	0.0	0.0	1.0	1.0	1.0	1.0
test_dataset_final	EstimatedSalary	647.0	101551.9042	57436.2946	123.0	53115.0	102530.0	150438.0	180423.0	189126.0	199506.0

2026-07-31 16:55:05,291 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DescriptiveStatistics:development_data does not exist in model's document

validmind.data_validation.TabularDescriptionTables:development_data

Tabular Description Tables Development Data

The Descriptive Statistics test evaluates the composition, central tendency, range, completeness, and recorded data types of variables in the development data. The results summarize numerical and categorical fields for both train_dataset_final and test_dataset_final, including observation counts, means, minimum and maximum values, missing-value percentages, and data types. The numerical summary covers CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary, and Exited, while the categorical summary covers the boolean indicator variables Geography_Germany, Geography_Spain, and Gender_Male.

Key insights:

No missing values observed: All reported numerical and categorical variables in both train_dataset_final and test_dataset_final show Missing Values (%) of 0.0, indicating complete observed coverage across the summarized fields.
Train and test means are closely aligned: Several numerical variables have similar mean values across the two datasets, including CreditScore (646.4685 train vs. 645.6244 test), Tenure (5.0240 vs. 4.9134), HasCrCard (0.7044 vs. 0.7141), IsActiveMember (0.4708 vs. 0.4498), and EstimatedSalary (100194.1192 vs. 101551.9042).
Outcome rate differs between splits: The mean of Exited is 0.4921 in train_dataset_final and 0.5317 in test_dataset_final, showing a higher observed proportion of positive outcomes in the test split.
Balance is higher in test data: Balance has a mean of 81042.5266 in train_dataset_final and 83478.3276 in test_dataset_final. The maximum balance is also higher in train data (250898.09) than in test data (214346.96), indicating different upper-end observed ranges across splits.
Categorical fields are binary boolean indicators: All listed categorical variables have exactly 2 unique values, recorded as True and False, with data type bool in both datasets. This applies consistently to Geography_Germany, Geography_Spain, and Gender_Male.
Numerical types are consistently encoded: Integer fields (CreditScore, Tenure, NumOfProducts, HasCrCard, IsActiveMember, Exited) are recorded as int64, while continuous-valued fields (Balance, EstimatedSalary) are recorded as float64 in both train and test datasets.

The descriptive statistics indicate that the summarized development data is complete for all reported fields and consistently typed across the train and test splits. Central tendency is broadly similar across most numerical variables, with the largest visible split-level differences occurring in Exited, Balance, and NumOfProducts. The categorical inputs are represented uniformly as binary boolean indicators, and the reported value ranges provide the observed bounds for each numerical field in both datasets.

Tables

dataset	Numerical Variable	Num of Obs	Mean	Min	Max	Data Type
train_dataset_final	CreditScore	2585	646.4685	350.00	850.00	int64
train_dataset_final	Tenure	2585	5.0240	0.00	10.00	int64
train_dataset_final	Balance	2585	81042.5266	0.00	250898.09	float64
train_dataset_final	NumOfProducts	2585	1.5269	1.00	4.00	int64
train_dataset_final	HasCrCard	2585	0.7044	0.00	1.00	int64
train_dataset_final	IsActiveMember	2585	0.4708	0.00	1.00	int64
train_dataset_final	EstimatedSalary	2585	100194.1192	11.58	199909.32	float64
train_dataset_final	Exited	2585	0.4921	0.00	1.00	int64
test_dataset_final	CreditScore	647	645.6244	350.00	850.00	int64
test_dataset_final	Tenure	647	4.9134	0.00	10.00	int64
test_dataset_final	Balance	647	83478.3276	0.00	214346.96	float64
test_dataset_final	NumOfProducts	647	1.4529	1.00	4.00	int64
test_dataset_final	HasCrCard	647	0.7141	0.00	1.00	int64
test_dataset_final	IsActiveMember	647	0.4498	0.00	1.00	int64
test_dataset_final	EstimatedSalary	647	101551.9042	123.07	199505.53	float64
test_dataset_final	Exited	647	0.5317	0.00	1.00	int64

dataset	Categorical Variable	Num of Obs	Num of Unique Values	Unique Values	Data Type
train_dataset_final	Geography_Germany	2585.0	2.0	[False True]	bool
train_dataset_final	Geography_Spain	2585.0	2.0	[False True]	bool
train_dataset_final	Gender_Male	2585.0	2.0	[ True False]	bool
test_dataset_final	Geography_Germany	647.0	2.0	[ True False]	bool
test_dataset_final	Geography_Spain	647.0	2.0	[False True]	bool
test_dataset_final	Gender_Male	647.0	2.0	[False True]	bool

2026-07-31 16:55:12,723 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularDescriptionTables:development_data does not exist in model's document

validmind.data_validation.ClassImbalance:development_data

✅ Class Imbalance Development Data

The Class Imbalance test evaluates the distribution of target classes in the development data by measuring the percentage of records in each class against the configured minimum threshold of 10%. The results are reported separately for train_dataset_final and test_dataset_final for the target Exited, with each class assigned a pass/fail outcome based on its observed share. The accompanying bar charts visualize the class proportions in each dataset, while the result table provides the corresponding percentages and threshold assessment.

Key insights:

Both classes pass the threshold: In both train_dataset_final and test_dataset_final, each Exited class exceeds the 10% minimum percentage threshold and is marked as Pass.
Training data is nearly balanced: In train_dataset_final, Exited=0 represents 50.79% of rows and Exited=1 represents 49.21%, indicating a difference of 1.58 percentage points between classes.
Test data shows a modest class skew: In test_dataset_final, Exited=1 accounts for 53.17% of rows and Exited=0 accounts for 46.83%, corresponding to a 6.34 percentage point difference between classes.
Class ordering differs between splits: The majority class is Exited=0 in the training dataset, while Exited=1 is the majority class in the test dataset.

The observed class distributions indicate that both development data splits satisfy the configured class frequency threshold, with no class falling below 10%. The training dataset is close to evenly distributed across the two target classes, while the test dataset remains balanced but with a somewhat larger separation between class shares. Across the two splits, the majority class changes from Exited=0 in training to Exited=1 in testing.

Parameters:

{
  "min_percent_threshold": 10
}

Tables

dataset	Exited	Percentage of Rows (%)	Pass/Fail
train_dataset_final	0	50.79%	Pass
train_dataset_final	1	49.21%	Pass
test_dataset_final	1	53.17%	Pass
test_dataset_final	0	46.83%	Pass

Figures

ValidMind Figure validmind.data_validation.ClassImbalance:development_data:1ed0

ValidMind Figure validmind.data_validation.ClassImbalance:development_data:d551

2026-07-31 16:55:25,331 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.ClassImbalance:development_data does not exist in model's document

validmind.data_validation.UniqueRows:development_data

❌ Unique Rows Development Data

The UniqueRows test evaluates column-level data diversity by comparing the percentage of unique values in each column against the configured minimum threshold of 1%. Results are reported for both train_dataset_final and test_dataset_final, showing the number and percentage of unique values for each column together with a pass/fail outcome. In the training dataset, 3 of 11 columns pass the threshold, while in the test dataset 4 of 11 columns pass. The reported uniqueness percentages range from 0.0774% to 100.0% in training and from 0.3091% to 100.0% in testing.

Key insights:

EstimatedSalary is fully unique: EstimatedSalary records 2,585 unique values out of 2,585 rows in train_dataset_final and 647 unique values out of 647 rows in test_dataset_final, corresponding to 100.0% uniqueness in both datasets.
Balance and CreditScore pass in both datasets: Balance shows 67.3888% unique values in training and 68.9335% in testing, while CreditScore shows 16.1702% in training and 46.8315% in testing. Both columns exceed the 1% threshold in each dataset.
Most low-cardinality columns fail: NumOfProducts, HasCrCard, IsActiveMember, Geography_Germany, Geography_Spain, Gender_Male, and Exited fail in both datasets. Their uniqueness percentages range from 0.0774% to 0.6182%, all below the configured threshold.
Tenure differs between training and testing: Tenure fails in train_dataset_final with 11 unique values and 0.4255% uniqueness, but passes in test_dataset_final with the same 11 unique values and 1.7002% uniqueness.

The results show that uniqueness is concentrated in a small subset of columns, with EstimatedSalary, Balance, and CreditScore consistently exceeding the threshold across both datasets. Most binary or otherwise low-cardinality columns fall below the 1% cutoff in both samples, producing repeated failures under this test configuration. Tenure is the only column with a different pass/fail outcome between datasets, driven by a higher uniqueness percentage in the test sample despite the same number of distinct values.

Parameters:

{
  "min_percent_threshold": 1
}

Tables

dataset	Column	Number of Unique Values	Percentage of Unique Values (%)	Pass/Fail
train_dataset_final	CreditScore	418	16.1702	Pass
train_dataset_final	Tenure	11	0.4255	Fail
train_dataset_final	Balance	1742	67.3888	Pass
train_dataset_final	NumOfProducts	4	0.1547	Fail
train_dataset_final	HasCrCard	2	0.0774	Fail
train_dataset_final	IsActiveMember	2	0.0774	Fail
train_dataset_final	EstimatedSalary	2585	100.0000	Pass
train_dataset_final	Geography_Germany	2	0.0774	Fail
train_dataset_final	Geography_Spain	2	0.0774	Fail
train_dataset_final	Gender_Male	2	0.0774	Fail
train_dataset_final	Exited	2	0.0774	Fail
test_dataset_final	CreditScore	303	46.8315	Pass
test_dataset_final	Tenure	11	1.7002	Pass
test_dataset_final	Balance	446	68.9335	Pass
test_dataset_final	NumOfProducts	4	0.6182	Fail
test_dataset_final	HasCrCard	2	0.3091	Fail
test_dataset_final	IsActiveMember	2	0.3091	Fail
test_dataset_final	EstimatedSalary	647	100.0000	Pass
test_dataset_final	Geography_Germany	2	0.3091	Fail
test_dataset_final	Geography_Spain	2	0.3091	Fail
test_dataset_final	Gender_Male	2	0.3091	Fail
test_dataset_final	Exited	2	0.3091	Fail

2026-07-31 16:55:35,182 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.UniqueRows:development_data does not exist in model's document

validmind.data_validation.TabularNumericalHistograms:development_data

Tabular Numerical Histograms Development Data

The TabularNumericalHistograms test evaluates the univariate distributions of numerical features through histograms to visualize concentration, spread, and tail behavior in the development data. The results present separate histograms for the train and test datasets across CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary, Geography_Germany, Geography_Spain, and Gender_Male. The figures show the shape of each feature distribution, including continuous variables with broad ranges and encoded discrete variables with mass concentrated at a small number of values. The train and test panels allow direct visual comparison of the same features across both development splits.

Key insights:

CreditScore is broadly bell-shaped: CreditScore in both train and test is concentrated in the mid-range, with the highest bar density around roughly 600 to 700 and thinner tails toward the lower and upper ends of the range.
Balance includes a large zero-valued mass: Balance shows a prominent spike at 0 in both train and test, alongside a separate concentration centered approximately around 100k to 140k, indicating a mixed distribution with one mass at zero and another across positive balances.
EstimatedSalary is approximately uniform: EstimatedSalary appears relatively flat across the full range up to about 200k in both train and test, with no strong central peak and only moderate bin-to-bin variation.
NumOfProducts is heavily concentrated at lower counts: NumOfProducts takes discrete values from 1 to 4, with the largest concentration at 1, followed by 2, and substantially fewer observations at 3 and 4 in both train and test.
Binary features show class imbalance or near-balance: HasCrCard is concentrated at 1 more than 0 in both datasets, while IsActiveMember is closer to balanced, with only a modestly higher count at 0 than at 1 in the test set and a similar near-balanced pattern in train.
Tenure is discrete with broad coverage: Tenure spans integer values from 0 to 10 in both datasets, with frequencies distributed across most levels rather than concentrated in only a few bins, although 0 and 10 appear lower than several interior values.
One-hot geography indicators are imbalanced: Geography_Germany and Geography_Spain both show more false than true observations in train and test, indicating that the positive class for each indicator is the minority category.
Gender indicator is near balanced: Gender_Male is close to evenly split in both train and test, with only a small difference between true and false counts.

The histogram results show that the development data contains a mix of continuous, discrete, and binary-valued inputs with distinct distributional forms. The most pronounced shape features are the zero-inflated Balance distribution, the lower-count concentration in NumOfProducts, and the relatively flat EstimatedSalary distribution. Across train and test, the corresponding feature shapes appear visually similar, with no obvious split-specific changes in the overall form of the displayed distributions.

Figures

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:2585

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:8ceb

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:9ff0

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:8b45

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:eebe

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:299c

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:4d02

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:745b

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:f123

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:6704

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:4e47

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:e944

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:57ca

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:d12c

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:68ec

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:0f4a

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:7972

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:1f8e

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:d760

ValidMind Figure validmind.data_validation.TabularNumericalHistograms:development_data:82a6

2026-07-31 16:56:55,280 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularNumericalHistograms:development_data does not exist in model's document

validmind.data_validation.MutualInformation:development_data

Mutual Information Development Data

The Mutual Information test evaluates the statistical dependency between each feature and the target to quantify feature relevance. The results are presented as mutual information scores for the training and test datasets, with a minimum threshold of 0.01 indicated by a dashed reference line. In the training dataset, feature scores range from approximately 0.123 to 0, while in the test dataset they range from approximately 0.044 to 0. Features above and below the threshold are visually separated, showing the relative concentration of predictive signal across the available inputs.

Key insights:

NumOfProducts is the strongest feature: NumOfProducts has the highest mutual information score in both datasets, at approximately 0.123 in training and 0.044 in test, making it the most informative feature relative to the target in both views.
Feature relevance is concentrated in a subset: In the training dataset, five features exceed the 0.01 threshold: NumOfProducts, Balance, IsActiveMember, Geography_Germany, and Gender_Male. In the test dataset, six features exceed the threshold: NumOfProducts, Geography_Germany, Geography_Spain, IsActiveMember, HasCrCard, and Balance.
Several features show zero information score: CreditScore, Tenure, HasCrCard, and EstimatedSalary are at 0 in the training plot, while CreditScore, Tenure, EstimatedSalary, and Gender_Male are at 0 in the test plot.
Some features differ across datasets: Gender_Male is above threshold in training at approximately 0.012 but 0 in test. Conversely, Geography_Spain and HasCrCard are below threshold in training, at approximately 0.004 and 0 respectively, but above threshold in test at approximately 0.028 and 0.022.
Training scores show a sharper top feature gap: The highest training score for NumOfProducts is substantially above the next highest feature, with Balance at approximately 0.035. In the test dataset, the distribution among the top six features is more compressed, with scores ranging from approximately 0.021 to 0.044.

The mutual information results show that predictive signal is not evenly distributed across the feature set and is concentrated in a limited group of variables. NumOfProducts is the dominant feature in both datasets, while several variables register no measurable mutual information in at least one dataset. The set of features above the 0.01 threshold differs between training and test, indicating variation in measured feature-target dependency across the two samples.

Parameters:

{
  "min_threshold": 0.01
}

Figures

ValidMind Figure validmind.data_validation.MutualInformation:development_data:5f46

ValidMind Figure validmind.data_validation.MutualInformation:development_data:6228

2026-07-31 16:57:48,608 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.MutualInformation:development_data does not exist in model's document

validmind.data_validation.PearsonCorrelationMatrix:development_data

Pearson Correlation Matrix Development Data

The PearsonCorrelationMatrix test evaluates linear dependency among numerical variables using pairwise Pearson correlation coefficients. The result is presented as correlation heat maps for the development dataset’s train and test partitions, with coefficients ranging from -1 to 1 and the diagonal equal to 1.0. Across both heat maps, most off-diagonal correlations are close to zero, with a limited number of moderate positive or negative relationships visible among the encoded geography indicators, balance, number of products, and the target variable Exited.

Key insights:

Most pairwise correlations are weak: In both train and test partitions, the majority of off-diagonal coefficients are small in magnitude and cluster near zero, indicating limited linear dependency across most variable pairs.
No correlations exceed the 0.7 threshold: The largest observed absolute correlations are 0.43 in the train partition and 0.41 in the test partition, both well below the stated high-correlation threshold of 0.7.
Geography and balance show the strongest positive relationships: Geography_Germany is positively correlated with Balance at 0.43 in train and 0.40 in test, representing the strongest positive association visible in the matrices.
Encoded geography indicators are moderately negatively related: Geography_Germany and Geography_Spain show correlations of -0.37 in train and -0.41 in test, indicating a consistent inverse relationship between these two encoded variables.
Exited has only modest linear associations: In the train partition, Exited is most correlated with Geography_Germany (0.22), IsActiveMember (-0.19), and Balance (0.16). In the test partition, the largest associations with Exited are IsActiveMember (-0.23), Geography_Germany (0.19), Gender_Male (-0.14), and Balance (0.13).

The correlation structure in the development data indicates generally low linear dependency among the available numerical variables. The most pronounced relationships are moderate and concentrated in a small subset of features, particularly the encoded geography variables and Balance. The train and test partitions exhibit similar overall patterns, with only modest differences in coefficient magnitude across corresponding variable pairs.

Figures

ValidMind Figure validmind.data_validation.PearsonCorrelationMatrix:development_data:554d

ValidMind Figure validmind.data_validation.PearsonCorrelationMatrix:development_data:22bb

2026-07-31 16:58:06,787 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.PearsonCorrelationMatrix:development_data does not exist in model's document

validmind.data_validation.HighPearsonCorrelation:development_data

❌ High Pearson Correlation Development Data

The High Pearson Correlation test evaluates pairwise linear relationships among features to identify highly correlated variable pairs that may indicate redundancy or multicollinearity. The results list the top correlations for both train_dataset_final and test_dataset_final, using a threshold of 0.3 to assign Pass or Fail status. In the training data, the reported coefficients range from -0.3687 to 0.4304, and in the test data they range from -0.4074 to 0.4003. Two feature pairs exceed the threshold in each dataset, while the remaining reported correlations are below the threshold and marked as Pass.

Key insights:

Two failed pairs in each dataset: train_dataset_final contains two failed correlations: (Balance, Geography_Germany) at 0.4304 and (Geography_Germany, Geography_Spain) at -0.3687. test_dataset_final also contains two failed correlations: (Geography_Germany, Geography_Spain) at -0.4074 and (Balance, Geography_Germany) at 0.4003.
Same strongest relationships recur: The same two feature pairs exceed the 0.3 threshold in both datasets. This repetition indicates that the highest-magnitude linear relationships reported are consistent between training and test samples.
Balance relates most strongly to Germany indicator: (Balance, Geography_Germany) is the largest positive reported correlation in both datasets, at 0.4304 in training and 0.4003 in test. No other positive correlation in the reported results exceeds these values.
Geography indicator pair is moderately negative: (Geography_Germany, Geography_Spain) is the strongest negative reported correlation in both datasets, with coefficients of -0.3687 in training and -0.4074 in test. This pair exceeds the threshold in absolute value in both samples.
Correlations with Exited remain below threshold: All reported correlations involving Exited are marked Pass. The largest absolute correlations with Exited are (Geography_Germany, Exited) at 0.2151 in training and (IsActiveMember, Exited) at -0.2286 in test, both below the 0.3 threshold.

The reported correlation structure is concentrated in two recurring feature pairs that exceed the configured threshold across both training and test datasets. Outside these pairs, the listed correlations are smaller in magnitude and remain within the pass criterion, including all reported relationships involving Exited. Overall, the results show a limited set of stronger linear relationships alongside a broader set of lower-magnitude pairwise associations.

Parameters:

{
  "max_threshold": 0.3,
  "top_n_correlations": 10
}

Tables

dataset	Columns	Coefficient	Pass/Fail
train_dataset_final	(Balance, Geography_Germany)	0.4304	Fail
train_dataset_final	(Geography_Germany, Geography_Spain)	-0.3687	Fail
train_dataset_final	(Geography_Germany, Exited)	0.2151	Pass
train_dataset_final	(IsActiveMember, Exited)	-0.1858	Pass
train_dataset_final	(Balance, NumOfProducts)	-0.1682	Pass
train_dataset_final	(Balance, Exited)	0.1600	Pass
train_dataset_final	(Balance, Geography_Spain)	-0.1586	Pass
train_dataset_final	(Gender_Male, Exited)	-0.1285	Pass
train_dataset_final	(Geography_Spain, Exited)	-0.0799	Pass
train_dataset_final	(NumOfProducts, IsActiveMember)	0.0594	Pass
test_dataset_final	(Geography_Germany, Geography_Spain)	-0.4074	Fail
test_dataset_final	(Balance, Geography_Germany)	0.4003	Fail
test_dataset_final	(IsActiveMember, Exited)	-0.2286	Pass
test_dataset_final	(Balance, NumOfProducts)	-0.2128	Pass
test_dataset_final	(Balance, Geography_Spain)	-0.1947	Pass
test_dataset_final	(Geography_Germany, Exited)	0.1892	Pass
test_dataset_final	(Gender_Male, Exited)	-0.1391	Pass
test_dataset_final	(Balance, Exited)	0.1322	Pass
test_dataset_final	(Geography_Spain, Exited)	-0.1223	Pass
test_dataset_final	(Tenure, Gender_Male)	0.1035	Pass

2026-07-31 16:58:17,766 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.HighPearsonCorrelation:development_data does not exist in model's document

validmind.model_validation.ModelMetadata

Model Metadata

The ModelMetadata test compares metadata across models to summarize their implementation characteristics, including modeling technique, framework, framework version, and programming language. The result table presents two models, log_model_champion and rf_model, with each row showing the recorded metadata fields used for comparison. The table enables a direct side-by-side view of the metadata values captured for each model.

Key insights:

Metadata is fully aligned across models: log_model_champion and rf_model share the same recorded values for all reported fields in the summary table.
Common modeling technique recorded: Both models are listed with the modeling technique SKlearnModel, indicating a consistent metadata classification for implementation type.
Framework and version are identical: Both models use the sklearn framework with framework version 1.9.0, showing no version differences in the reported metadata.
Programming language is consistent: Both models are recorded as using Python, with no language variation across the compared models.

The metadata comparison shows complete consistency across the reported fields for the two models included in the test. No differences are present in modeling technique, framework, framework version, or programming language within the provided results. This indicates that the compared models are documented with the same high-level implementation metadata.

Tables

model	Modeling Technique	Modeling Framework	Framework Version	Programming Language
log_model_champion	SKlearnModel	sklearn	1.9.0	Python
rf_model	SKlearnModel	sklearn	1.9.0	Python

2026-07-31 16:58:23,384 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.ModelMetadata does not exist in model's document

validmind.model_validation.sklearn.ModelParameters

Model Parameters

The Model Parameters test extracts and displays the configuration values that define model behavior for transparency and reproducibility. The results present parameter tables for two models, log_model_champion and rf_model, with each row listing a parameter name and its recorded value. For log_model_champion, the output includes regularization, solver, iteration, and intercept settings, while for rf_model it includes tree construction, sampling, impurity, ensemble size, and reproducibility settings.

Key insights:

Two model configurations documented: The results capture parameter sets for both log_model_champion and rf_model, providing structured visibility into the configuration of each model object included in the test output.
Logistic model uses L1 regularization: log_model_champion is configured with penalty = l1, solver = liblinear, and C = 1. The model also records fit_intercept = True, max_iter = 100, and tol = 0.0001.
Random forest uses 50 trees: rf_model is configured with n_estimators = 50, criterion = gini, and max_features = sqrt. The table also shows bootstrap = True, indicating resampled tree construction within the ensemble.
Random forest split thresholds are minimal: For rf_model, min_samples_split = 2 and min_samples_leaf = 1 are recorded, alongside min_impurity_decrease = 0.0 and ccp_alpha = 0.0. These values indicate that no additional pruning or impurity-based split restriction is encoded in the extracted parameters.
Reproducibility settings are explicit: rf_model includes random_state = 42, and both models record verbose = 0 and warm_start = False. The parameter tables therefore include explicit values for run-control settings rather than leaving them undocumented in the result.

The extracted parameter tables provide a clear record of the configured settings for both the logistic regression champion model and the random forest model. The logistic configuration is defined by L1 regularization with the liblinear solver, while the random forest configuration is defined by a 50-tree ensemble with bootstrap sampling, gini splitting, and explicit randomness control through random_state = 42. Collectively, the results document the parameterization necessary to identify and reproduce the model configurations represented in this test output.

Tables

model	Parameter	Value
log_model_champion	C	1
log_model_champion	dual	False
log_model_champion	fit_intercept	True
log_model_champion	intercept_scaling	1
log_model_champion	max_iter	100
log_model_champion	penalty	l1
log_model_champion	solver	liblinear
log_model_champion	tol	0.0001
log_model_champion	verbose	0
log_model_champion	warm_start	False
rf_model	bootstrap	True
rf_model	ccp_alpha	0.0
rf_model	criterion	gini
rf_model	max_features	sqrt
rf_model	min_impurity_decrease	0.0
rf_model	min_samples_leaf	1
rf_model	min_samples_split	2
rf_model	min_weight_fraction_leaf	0.0
rf_model	n_estimators	50
rf_model	oob_score	False
rf_model	random_state	42
rf_model	verbose	0
rf_model	warm_start	False

2026-07-31 16:58:30,249 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.sklearn.ModelParameters does not exist in model's document

validmind.model_validation.sklearn.ROCCurve

ROC Curve

The ROCCurve test evaluates classification performance by plotting the receiver operating characteristic curve and calculating the area under the curve (AUC) to measure class discrimination across threshold levels. Results are shown for log_model_champion on both train_dataset_final and test_dataset_final, with each figure comparing the model ROC curve against the random-classification reference line. The reported AUC values are 0.68 on the training dataset and 0.69 on the test dataset, and in both plots the ROC curve remains above the diagonal reference line across the threshold range.

Key insights:

Similar train and test AUC: The AUC is 0.68 on train_dataset_final and 0.69 on test_dataset_final. This indicates that the measured discrimination level is nearly unchanged between the two evaluated datasets.
Discrimination exceeds random baseline: In both figures, the ROC curve lies above the random reference line and the AUC values are above 0.5. The observed results show discrimination better than chance on both datasets.
Performance is moderate in magnitude: The reported AUC values remain below 0.70 in both datasets. The ROC curves show separation from the diagonal, but not a large margin across the full false-positive-rate range.

Overall, the ROC results show that log_model_champion achieves consistent discriminatory performance on the training and test datasets, with AUC values of 0.68 and 0.69 respectively. The closeness of these values indicates limited change in measured ROC performance between datasets. At the same time, the magnitude of the AUC values reflects moderate rather than strong separation between classes in this evaluation.

Figures

ValidMind Figure validmind.model_validation.sklearn.ROCCurve:181c

ValidMind Figure validmind.model_validation.sklearn.ROCCurve:6278

2026-07-31 16:58:42,842 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.sklearn.ROCCurve does not exist in model's document

validmind.model_validation.sklearn.MinimumROCAUCScore

✅ Minimum ROCAUC Score

The Minimum ROC AUC Score test evaluates whether the model’s ROC AUC score meets or exceeds a predefined minimum threshold. Results are reported for both train_dataset_final and test_dataset_final, with each dataset showing the observed ROC AUC score alongside the threshold value of 0.5 and the corresponding pass/fail outcome. The recorded scores are 0.6814 for the training dataset and 0.6865 for the test dataset, and both results are marked as passing.

Key insights:

Both datasets passed threshold: The ROC AUC score exceeded the minimum threshold of 0.5 on both evaluated datasets. train_dataset_final recorded 0.6814 and test_dataset_final recorded 0.6865.
Test performance is slightly higher: The test dataset ROC AUC score of 0.6865 is marginally above the training dataset score of 0.6814. The difference between the two results is 0.0051.
Consistent discrimination across splits: The proximity of the train and test ROC AUC values indicates similar measured discrimination performance across the two evaluated datasets within this test.

The test results show that the model met the minimum ROC AUC requirement on both the training and test datasets. Observed performance is closely aligned across the two dataset splits, with the test score slightly exceeding the training score. Collectively, the results indicate that the measured classification discrimination under this test remained above the specified threshold in both cases.

Parameters:

{
  "min_threshold": 0.5
}

Tables

dataset	Score	Threshold	Pass/Fail
train_dataset_final	0.6814	0.5	Pass
test_dataset_final	0.6865	0.5	Pass

2026-07-31 16:58:51,974 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.sklearn.MinimumROCAUCScore does not exist in model's document

In summary

In this final notebook, you learned how to:

Implement a custom inline test
Run and log your custom inline tests
Use external custom test providers
Run and log tests from your custom test providers
Re-run tests provided by your development team to verify that they were run and reported accurately

With our ValidMind for validation series of notebooks, you learned how to validate a record (model) end-to-end with the ValidMind Library by running through some common scenarios in a typical validation setting:

Verifying the data quality steps performed by the development team
Independently replicating the champion's results and conducting additional tests to assess performance, stability, and robustness
Setting up test inputs and a challenger for comparative analysis
Running validation tests, analyzing results, and logging artifacts to ValidMind

Next steps

Work with your validation report

Now that you've logged all your test results and verified the work done by the development team, head to the ValidMind Platform to wrap up your validation report. Continue to work on your validation report by:

Inserting additional test results: Click Link Evidence under any Evidence panel of 2. Validation in your validation report. (Learn more: Link evidence to reports)
Making qualitative edits to your test descriptions: Expand any linked evidence under Validator Evidence and click See evidence details to review and edit the ValidMind-generated test descriptions for quality and accuracy. (Learn more: Preparing validation reports)
Adding more findings: Click Link Finding to Report in any validation report section, then click + Create New Finding. (Learn more: Add and manage artifacts)
Adding risk assessment notes: Click under Risk Assessment Notes in any validation report section to access the text editor and content editing toolbar, including an option to generate a draft with AI. Once generated, edit your ValidMind-generated test descriptions to adhere to your organization's requirements. (Learn more: Work with content blocks)
Assessing compliance: Under the Guideline for any validation report section, click Assessment and select the compliance status from the drop-down menu. (Learn more: Assign compliance assessments)
Collaborate with other stakeholders: Use the ValidMind Platform's real-time collaborative features to work seamlessly together with the rest of your organization, including developers. Propose suggested changes in the documentation, work with versioned history, and use comments to discuss specific portions of the documentation. (Learn more: Collaborate with others)

When your validation report is complete and ready for review, submit it for approval from the same ValidMind Platform where you made your edits and collaborated with the rest of your organization, ensuring transparency and a thorough validation history. (Learn more: Submit documents)

Learn more

Now that you're familiar with the basics, you can explore the following notebooks to get a deeper understanding on how the ValidMind Library assists you in streamlining validation:

Use cases

Validate an application scorecard model

Discover more learning resources

Learn more about the ValidMind Library tools we used in this notebook:

We also offer many interactive notebooks to help you use the ValidMind Library to streamline your work:

Or, visit our documentation to learn more about ValidMind.