Validator
Fundamentals

docs.validmind.ai/training

Click to start

Learning objectives

“As a validator who is new to ValidMind, I want to learn how to review model documentation, prepare my validation report, track issues, and submit my report for approval.”

In this course

PART 1

• Review documentation
• Analyze test results
• Prepare validation reports

PART 2

• Track issue resolution
• Submit for review and approval

First, let’s make sure you can log in to ValidMind.

Training is interactive — you explore ValidMind live. Try it!

→ , ↓ , SPACE , N — next slide     ← , ↑ , P , H — previous slide     ? — all keyboard shortcuts

Can you log in?

To try out this course, you need to have been onboarded onto ValidMind Academy with the Validator role.

Log in to check your access:

Log in to ValidMind

Be sure to return to this page afterwards.

You’re in — let’s show you around ValidMind.

This is the ValidMind Platform.

From here, you have access to:

• Model inventory
• Documentation & test results
• Validation reports
• Model findings
• Analytics

Try it live on the next page.

From the Inventory,
you access:

• Model documentation
• Validation reports
• Getting started for
  developers

Explore some models, then
click to continue.

From Model Findings,
you track:

• Major or minor issues
• Deficiencies
• Model limitations
• Robustness concerns
• And more!

Explore existing findings,
then click to continue.

Under Analytics, you can
find:

• Executive summaries
• Model tracking
• Risk assessments
• Compliance review
• Remediation tracking

Explore existing reports,
then click to continue.

PART 1

Review documentation

Evaluate the conceptual soundness, data preparation, model development, and ongoing monitoring and governance plans for the model.

Learn more…

Review model documentation

The Document Overview shows a section-by-section outline of your model’s documentation, as well as summaries of:

• Any unresolved conversations
• The number of model findings
• The completion status for your model’s documentation
• Recent activity on your model

To locate your document overview for a model:

1. In the left sidebar, click Inventory.

2. Select a model or find your model by applying a filter or searching for it.

3. In the left sidebar that appears for your model, click Documentation.

Try it live on the next page.

Review the documentation for the [Quickstart] Customer Churn Model.

When you are done, click to continue.

Have a question about the model? Collaborate with your developer right in the model documentation.

Learn more …

Try it live on the next page.

While working with content blocks in documentation, you can comment directly on specific portions of the text.

All users associated with a model, such as model developers and model validators, will see a notification that a comment has been posted in their Recent Activity feed, accessible via the Dashboard.

• Post comments
• Reply to comments
• Resolve comments

1. In any section of the model documentation, select the portion of text you want to comment on, then click the button that appears.

2. Enter your comment and click Comment.

3. You can view the comment by clicking the highlighted text. Comments will also appear in the right sidebar.

1. Click the highlighted text to view the comment thread.

2. Enter your comment and click Reply.

3. You can view the comment thread by clicking the highlighted text.

1. Click the highlighted text portion to view the thread, then click to resolve the thread.

2. To view the resolved comment thread, click the Comments archive button in the toolbar.

   You can view a history of all archived comments in the Comment archive.

To reopen a comment thread, reply to the comment thread in the Comment archive or click the Reopen button that appears next to the highlighted text portion.

In the documentation, post a comment, reply to it, and then resolve the comment thread. Review your comments in the Recent Activity feed on the front page.

When you are done, click to continue.

Analyze test results

Locate the test results in the documentation, review the data, and identify issues with the model.

Learn more …

To locate your document overview for a model:

1. In the left sidebar, click Inventory.

2. Select a model or find your model by applying a filter or searching for it.

3. In the left sidebar that appears for your model, click Documentation.

1. Review the sections:

   • 2. Data Preparation
   • 3. Model Development

Try it live on the next page.

In the documentation, review 2. Data Preparation and 3. Model Development.

When you are done, click to continue.

Prepare validation reports

Based on your review of the documentation, add some findings for your validation report.

Learn more …

Try it live on the next page.

Add findings

As part of the validation process, you may find issues with the model documentation that must be resolved. To indicate that there is an issue and to track the resolution later on, you add a new finding.

• Findings are logged with your model documentation.
• You can add findings both on the main documentation overview page and in each documentation section.

1. Select a model or find your model by applying a filter or searching for it.

2. In the left sidebar that appears for your model, click Documentation.

   You can now either log a finding on this overview page, or via a specific documentation section. Both methods will allow you to associate a finding with a documentation section.

Add some findings for your validation report.

When you are done, click to continue.

Link your findings and the evidence from the test results you analyzed to the validation report.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

• Link finding
• Link evidence

1. Select a model or find your model by applying a filter or searching for it.

2. In the left sidebar that appears for your model, click Validation Report.

   You can now expand any subsection of the validation report you would like to work with.

   For example, locate 2. Validation then select 2.1.1. Assumptions.

3. In any section of the report where the button is available, click Link Finding to Report.

   Validation report section 2.1.1. that shows a compliance assessment with the option to link to finding

4. On the Link Finding to Report modal that opens, select from the list of available findings, or click Create New Finding.

5. Click Update Linked Findings.

   Confirm that newly linked finding shown under Findings is accurate..

To link evidence logged by developers to your validation report:

1. Select a model or find your model by applying a filter or searching for it.

2. In the left sidebar that appears for your model, click Validation Report.

   You can now expand any subsection of the validation report you would like to work with.

   For example, locate 2. Validation then select 2.1.1. Assumptions.

3. In any section of the report where the button is available, click Link Developer Evidence.

   Validation report section 2.1.1. that shows a compliance assessment with the option to link to developer evidence

4. On the Link Developer Evidence to Validation Report page that opens, select the evidence that is related to your assessment.

   • Available evidence is associated with the relevant documentation section.
   • If you are not sure if something is relevant, you can expand the section by clicking for more details.

5. Click Update Linked Evidence.

   • Confirm that the newly linked-to evidence shown under Developer Evidence is accurate and is connected to the correct documentation section.
   • Evidence that needs your attention gets flagged with
     Requires Attention. These sections get flagged automatically whenever a test result is above or below a certain threshold.

Link findings and link evidence to your validation report.

When you are done, click to continue.

Based on the evidence you analyzed and your model, with your guidelines.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

Assess compliance

1. Select a model or find your model by applying a filter or searching for it.

2. In the left sidebar that appears for your model, click Validation Report.

   You can now expand any subsection of the validation report you would like to work with.

   For example, locate 2. Validation then select 2.1.1. Assumptions.

3. In any section of the documentation where the assessment dropdown menu is available, select one of the available options:

   • No compliance — No tests have been run and no other evidence has been provided.
   • Some compliance — Evidence provided but not properly documented or verifiable.
   • Strong compliance — Evidence provided and documented, but not validated.
   • Full compliance — Evidence provided, documented, and validated.

   For example, to indicate that there is some compliance based on the evidence or findings you linked to:

   Validation report section 2.1.1. that shows a compliance assessment with the option to link to finding

   Confirm that the compliance summary shown for each subsection under 2. Validation provides an accurate overview for current qualitative and quantitative risk assessments:

   Risk assessment compliance summary

Assess compliance for your validation report.

When you are done, click to continue.

PART 2

Track issue resolution

As you prepare your report, review open or past due findings, close resolved ones, or add a mitigation plan.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

Track issue resolution

1. Select a model or find your model by applying a filter or searching for it.

2. In the left sidebar that appears for your model, click Model Findings.

3. Go through the open findings one-by-one:

   • If the finding has been addressed:

     • Add any final comments to assess how the issue was resolved.
     • Under status, select Closed.

   • If part or all of the finding remains to be addressed:

     Assess if the issue identified by the finding prevents the model from being approved.

     Criteria include:

     1. High-severity findings — Must be resolved before approval.

     2. Medium-severity findings — May allow conditional approval with a mitigation plan.

     3. Low-severity findings — Don’t prevent approval and are typically resolved post-approval with a mitigation plan.

4. Track open issues until all findings are resolved or a remediation plan is in place for post-approval issues.

To track issue resolution, review open or past due findings, close resolved ones, or add a mitigation plan.

When you are done, click to continue.

Submit for review and approval

When you’re ready, verify the approval workflow, and then submit your validation report for approval.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

• Verify workflow
• Submit for approval

Workflow states and transitions are configured by an administrator in advance, but you should verify that the expected people are included in the approval process.

1. Select a model or find your model by applying a filter or searching for it.

2. On the landing page of your model, locate the model status section:

   • Click See Workflow to open the detailed workflow associated with that model.
   • The current workflow state will be highlighted on this detail view.

While your lifecycle statuses and workflows are custom to your organization, some examples are:

• In Documentation — The model is currently being documented and can be submitted for validation next.
• In Validation — The model is currently being validated and can be submitted for review and then approval.

To transition through the approval workflow, all required workflow steps must be completed. For example, you cannot submit a validation report for review until the model documentation itself has been submitted.

1. Select a model or find your model by applying a filter or searching for it.

2. If an action is available to your role, you’ll see it listed under your model status on the model’s landing page.

   • Click to open up the transition panel for your selected action. This arrow will be followed by the action name.
   • Enter your notes and any other additional inventory fields then click Submit.

While your lifecycle statuses and workflows are custom to your organization, some examples are:

• To submit model documentation for validation — Click Ready for Validation to indicate that a model developer has completed the initial model documentation and is ready to have it validated. Add any notes that need to be included and then click Ready for Validation.
• To submit validation reports for review and approval — Click Ready for Review to indicate that you have completed your initial model validation report, fill in the mandatory notes, and submit.
• To request revisions to model documentation or validation reports — Click Request Revision, fill in the mandatory notes to explain the changes that are required, and submit.
• To have your model documentation and validation report approved — Click Ready for Approval, fill in the mandatory notes, and submit.

Verify the workflow, then submit your validation report for review & approval.

When you are done, click to continue.

About validation reports

There is more that ValidMind can do to help you prepare validation reports, from using your own template to configuring the full approval workflow.

All model validation guides

Or, find your next learning resource on ValidMind Academy.