Developer
Fundamentals

docs.validmind.ai/training

Click to start

Learning objectives

“As a developer who is new to ValidMind, I want to learn how to generate model documentation, add my own tests, edit the content, and then submit my documentation for approval.”

In this course

PART 1

• Initialize the ValidMind Library
• Start the model development process
• Edit model documentation
• Collaborate with others

PART 2

• Train a model
• Implement custom tests and integrate external test providers
• Finalize testing and documentation

PART 3

• View documentation activity
• Submit for approval

First, let’s make sure you can log in to ValidMind.

Training is interactive — you explore ValidMind live. Try it!

→ , ↓ , SPACE , N — next slide     ← , ↑ , P , H — previous slide     ? — all keyboard shortcuts

Can you log in?

To try out this course, you need to have been onboarded onto ValidMind Academy with the Developer role.

Log in to check your access:

Log in to JupyterHub

Log in to ValidMind

Be sure to return to this page afterwards.

You’re in — let’s show you around.

This introductory notebook includes sample code and how-to information, all in one place.

When run on JupyterHub, this notebook will generate model documentation and upload it to ValidMind. After you finish this course, your own training notebook will look similar, complete with output.

For now, scroll through this notebook to explore. When you are done, click to continue.

This is the ValidMind Platform.

From here, you can:

• Register models in the model inventory.
• Review and edit model documentation
  generated with the introductory notebook.
• Collaborate with model validators to get
  your documentation approved.
• And much more!

To start the documentation process, you register a
new model in the model inventory or select one that
has already been registered.

Explore ValidMind live on the next page.

From the Inventory:

1. Open a model, such as:

 [Quickstart] Customer Churn Model

2. Explore Documentation for
   model documentation.
3. Check Getting Started for
   the code snippet.

Did you find the code snippet?
You will copy and paste a similar
snippet into your own notebook
later to upload documentation.

When you’re done, click to
continue.

PART 1

Initialize the library

On JupyterHub: Run the cells in 1. Initializing the ValidMind Library.

When you are done, return to this page and click to continue.

Start the model development process

On JupyterHub: Run the cells in 2. Start the model development process with raw data, run out-of-the box tests, and add evidence to model documentation.

When you reach Add individual test results to model documentation, return to this page and click to continue.

Edit model documentation

Now that you have generated documentation, edit it on ValidMind to add text or test-driven content blocks.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

Add a test-driven content block

Content blocks provide sections that are part of your model documentation — you can always add more, as required, and fill them with text or test results.

1. Select a model or find your model by applying a filter or searching for it.

2. In the left sidebar that appears for your model, click Documentation.

3. Navigate to the 2.3. Correlations and Interactions section.

4. Hover the cursor after the Pearson Correlation Matrix content block until a horizontal dashed line with a button appears that indicates you can insert a new block:

   Screenshot showing the insert button for test-driven blocks

5. Click and then select Test-Driven Block:

   • In the search bar, type in HighPearsonCorrelation.
   • Select HighPearsonCorrelation:balanced_raw_dataset as the test.

   A preview of the test gets shown:

   Screenshot showing the selected test result in the dialog

6. Click Insert 1 Test Result to Document.

After you have completed these steps, the new content block becomes a part of your model documentation. You will now see two individual results for the high-correlation test in the 2.3. Correlations and Interactions section of the documentation.

To finalize the documentation, you can also edit the description of the test result to explain the changes made to the raw data and the reasons behind them. For example:

Screenshot showing description added to the new content block

Locate the model documentation you generated and add a test-driven block to the 2.3 Correlations and Interactions section.

When you are done, click to continue.

Collaborate with others

Have a question? Collaborate with other developers or with your validator right in the model documentation.

Learn more …

Try it live on the next page.

While working with content blocks in documentation, you can comment directly on specific portions of the text.

All users associated with a model, such as model developers and model validators, will see a notification that a comment has been posted in their Recent Activity feed, accessible via the Dashboard.

• Post comments
• Reply to comments
• Resolve comments

1. In any section of the model documentation, select the portion of text you want to comment on, then click the button that appears.

2. Enter your comment and click Comment.

3. You can view the comment by clicking the highlighted text. Comments will also appear in the right sidebar.

1. Click the highlighted text to view the comment thread.

2. Enter your comment and click Reply.

3. You can view the comment thread by clicking the highlighted text.

1. Click the highlighted text portion to view the thread, then click to resolve the thread.

2. To view the resolved comment thread, click the Comments archive button in the toolbar.

   You can view a history of all archived comments in the Comment archive.

To reopen a comment thread, reply to the comment thread in the Comment archive or click the Reopen button that appears next to the highlighted text portion.

In the documentation, post a comment, reply to it, and then resolve the thread.
Review your comments in the Recent Activity feed on the front page.

When you are done, click to continue.

PART 2

Train a model

On JupyterHub: Run the cells under Model Testing.

When you are done, return to this page and click to continue.

Implement custom tests and integrate external test providers

On JupyterHub: Run the cells in 3. Implementing custom tests.

When you are done, return to this page and click to continue.

Finalize testing and documentation

On JupyterHub: Run the cells in 4. Finalize testing and documentation.

When you are done, return to this page and click to continue.

PART 3

View documentation activity

Track changes and updates made to model documentation over time.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

View documentation activity

The Model Activity page shows a record of all activities, including actions performed by users in your organization and updates generated via the library such as:

• Inventory fields updated on the model
• Model or documentation status transitions
• Updates to model documentation, validation reports, or ongoing monitoring plans
• Test results added to your model
• Findings added, updated, or removed
• Comment creation and replies

To view model activity:

1. In the left sidebar, click Inventory.

2. Select a model or find your model by applying a filter or searching for it.

3. In the expanded sidebar that appears for your model, click Model Activity.

Review the Recent Activity feed on the front page:

• Comments

• Status updates

• Model updates

• Test results

When you are done, click to continue.

Submit for approval

When you’re ready, verify the approval workflow, and then submit your model documentation for approval.

(Scroll down for the full instructions.)

Learn more …

Try it live on the next page.

• Verify workflow
• Submit for approval

Workflow states and transitions are configured by an administrator in advance, but you should verify that the expected people are included in the approval process.

1. Select a model or find your model by applying a filter or searching for it.

2. On the landing page of your model, locate the model status section:

   • Click See Workflow to open the detailed workflow associated with that model.
   • The current workflow state will be highlighted on this detail view.

While your lifecycle statuses and workflows are custom to your organization, some examples are:

• In Documentation — The model is currently being documented and can be submitted for validation next.
• In Validation — The model is currently being validated and can be submitted for review and then approval.

To transition through the approval workflow, all required workflow steps must be completed. By default, a model must be in the
In Documentation state before you can submit it for validation.

1. Select a model or find your model by applying a filter or searching for it.

2. If an action is available to your role, you’ll see it listed under your model status on the model’s landing page.

   • Click to open up the transition panel for your selected action. This arrow will be followed by the action name.
   • Enter your notes and any other additional inventory fields then click Submit.

While your lifecycle statuses and workflows are custom to your organization, some examples are:

• To submit model documentation for validation — Click Ready for Validation to indicate that a model developer has completed the initial model documentation and is ready to have it validated. Add any notes that need to be included and then click Ready for Validation.
• To submit validation reports for review and approval — Click Ready for Review to indicate that you have completed your initial model validation report, fill in the mandatory notes, and submit.
• To request revisions to model documentation or validation reports — Click Request Revision, fill in the mandatory notes to explain the changes that are required, and submit.
• To have your model documentation and validation report approved — Click Ready for Approval, fill in the mandatory notes, and submit.

Verify the approval workflow and then submit your model documentation for approval.

When you are done, click to continue.

About model documentation

There is more that ValidMind can do to help you create model documentation, from using your own template to code samples you can adapt for your own use case.

All model documentation guides

Or, find your next learning resource on ValidMind Academy.