Enable monitoring

To start uploading ongoing monitoring results for a model to ​ValidMind, enable monitoring in your code snippet within the ValidMind Platform and then select a monitoring template. You can enable monitoring for both new and existing models.

NoneTo try out monitoring, check out the code sample for ongoing monitoring of models.

Quickstart for ongoing monitoring of models with ​ValidMind

Prerequisites

• You are logged in to ValidMind.
• You are a Developer or assigned another role with sufficient permissions to perform the tasks in this guide.¹
• The model is in the pre-approval stage for performance testing or the model has been approved and is in production.

¹ Manage permissions

Steps

1. Get monitoring code snippet

Initialize the ValidMind Library with the code snippet unique to each model per document to connect your environment to the ValidMind Platform, ensuring test results are uploaded to the correct model and automatically populated in the right document.

To enable ongoing monitoring for a model, add monitoring=True to your code snippet:

1. In the ValidMind Platform, either register a new model² or locate an existing model:

   • In the left sidebar, click Inventory.
   • Select a model or find your model by applying a filter or searching for it.³

2. Copy the code snippet for the model:

   • In the left sidebar that appears for your model, click Getting Started.
   • Select a Monitoring document type from the document drop-down menu.⁴
   • Set enable monitoring? to true.
   • Click Copy snippet to clipboard.

² Register models in the inventory

³ Working with the model inventory

⁴ Selecting a document to connect to requires ValidMind Library version >=2.12.0.

2. Select monitoring template

Before you can start sending ongoing monitoring data using the ValidMind Library to the ValidMind Platform, you must select a monitoring template:

1. In the ValidMind Platform, click Inventory in the left sidebar.

2. Select a model or find your model by applying a filter or searching for it.⁵

3. In the left sidebar that appears for your model, click Documents and select the Latest tab.⁶

4. Click on a Monitoring type file.⁷

5. From the template drop-down, select one of the available monitoring templates.

6. Click Use Template to apply your desired template.

   • After you select a template, the initial document where your ongoing monitoring results will be stored becomes available.
   • Review this documents structure to confirm that it meets your requirements, then start running your code to populate the template with monitoring results.

⁵ Working with the model inventory

⁶ Work with document versions

⁷ Working with model documents

3. Run code snippet

1. Paste the code snippet into your Jupyter Notebook or other development environment.

2. Confirm that monitoring=True is present in the vm.init method, similar to this example:

   import validmind as vm
   
   vm.init(
       api_host="https://api.prod.validmind.ai/api/v1/tracking",
       api_key="...",
       api_secret="...",
       model="...",
       document="monitoring", # Unique key for your monitoring document
       monitoring=True
   )

   
   

   NoneHow do I enable monitoring when I am storing credentials in an .env file?

   Load your model credentials from your .env file, then remove the inline credentials from vm.init() after importing the library.

   You’ll still need to include your document key and the monitoring flag.

   Store model credentials in .env files

   %load_ext dotenv
   %dotenv .env
   
   import validmind as vm
   
   vm.init(
      document="monitoring",
      monitoring=True
   )

3. Run the code snippet to initialize the ValidMind Library and connect to the selected monitoring document for your model in the ValidMind Platform.⁸

⁸ Install and initialize the ValidMind Library

NoneA template must already be applied to your selected document to populate monitoring test results in the ValidMind Platform.

Attempting to initialize the library with a document that does not have a template applied will result in an error.

What’s next

After you have enabled ongoing monitoring and run your code to generate some output, you can start reviewing the monitoring results.⁹

⁹ Review monitoring results