Enable monitoring

To start uploading ongoing monitoring results for a model to ValidMind, you enable monitoring in your code snippet and then select a monitoring template. You can enable monitoring for both new and existing models.

Enable monitoring with two steps:

1. Add monitoring=True to your code snippet
2. Select a monitoring template

To try out monitoring, check out the code sample for ongoing monitoring of models.

Quickstart for ongoing monitoring of models with ValidMind

Prerequisites

• You are logged in to ValidMind.
• You are a Developer or assigned another role with sufficient permissions to perform the tasks in this guide.¹
• The model is in the pre-approval stage for performance testing or the model has been approved and is in production.

¹ Manage permissions

Add monitoring=True to your code snippet

ValidMind generates a unique code snippet for each registered model to connect with your developer environment. You initialize the ValidMind Library with this code snippet, which ensures that your documentation and tests are uploaded to the correct model when you run the notebook.

To enable ongoing monitoring for a model, add monitoring=True to your code snippet:

1. In the ValidMind Platform, either register a new model² or locate an existing model:

   • In the left sidebar, click Model Inventory.
   • Select a model or find your model by applying a filter or searching for it.³

2. Copy the code snippet for the model:

   • In the left sidebar that appears for your model, click Getting Started.
   • Locate the code snippet and set enable ongoing monitoring to true.
   • Click Copy snippet to clipboard.

3. Paste the snippet into your development source code.⁴

   Confirm that monitoring=True is present in the vm.init method, similar to this example:

   import validmind as vm
   
   vm.init(
       api_host="https://api.prod.validmind.ai/api/v1/tracking",
       api_key="...",
       api_secret="...",
       model="...",
       monitoring=True
   )

² Register models in the inventory

³ Working with the model inventory

⁴ Install and initialize the ValidMind Library

Select a monitoring template

Before you can start sending ongoing monitoring data from your developer environment to ValidMind, you must select a monitoring template:

1. In the ValidMind Platform, click Model Inventory in the left sidebar.

2. Select a model or find your model by applying a filter or searching for it.⁵

3. In the left sidebar, click Ongoing Monitoring.

4. From the template drop-down, select one of the available monitoring templates.

5. Click Use Template to apply your desired template.

   

   Template that gets populated with monitoring results

   • After you select a template, the initial document where your ongoing monitoring results will be stored becomes available.
   • Review this documents structure to confirm that it meets your requirements, then start running your code to populate the template with monitoring results.

⁵ Working with the model inventory

Troubleshooting

If you see errors similar to this one when running your code, make sure that you have selected your monitoring template and then re-run your code:

APIRequestError: Please select an ongoing monitoring template on the ValidMind Platform first

What’s next

After you have enabled ongoing monitoring and run your code to generate some output, you can start reviewing the monitoring results.⁶

⁶ Review monitoring results