Enable monitoring

Published

October 3, 2024

To start uploading ongoing monitoring results for a model to ValidMind, you enable monitoring in your code snippet and then select a monitoring template. You can enable monitoring for both new and existing models.

Enable monitoring with two steps:

  1. Add monitoring=True to your code snippet
  2. Select a monitoring template
To try out monitoring, check out the code sample for ongoing monitoring of models.

Prerequisites

Add monitoring=True to your code snippet

ValidMind generates a unique code snippet for each registered model to connect with your developer environment. You initialize the client library with this code snippet, which ensures that your documentation and tests are uploaded to the correct model when you run the notebook.

To enable ongoing monitoring for a model, add monitoring=True to your code snippet:

  1. In the ValidMind Platform UI, either register a new model2 or locate an existing model:

    • In the left sidebar, click Model Inventory.
    • Select a model by clicking on it or find your model by applying a filter or searching for it.3
  2. Copy the code snippet for the model:

    • In the left sidebar that appears for your model, click Getting Started.
    • Locate the code snippet and set enable ongoing monitoring to true.
    • Click Copy snippet to clipboard.
  3. Paste the snippet into your development source code.4

    Confirm that monitoring=True is present in the vm.init method, similar to this example:

    import validmind as vm
    
    vm.init(
        api_host="https://api.prod.validmind.ai/api/v1/tracking",
        api_key="...",
        api_secret="...",
        project="...",
        monitoring=True
    )

Select a monitoring template

Before you can start sending ongoing monitoring data from your developer environment to ValidMind, you must select a monitoring template:

  1. In the ValidMind Platform UI, click Model Inventory in the left sidebar.

  2. Select a model by clicking on it or find your model by applying a filter or searching for it.5

  3. In the left sidebar, click Ongoing Monitoring.

  4. From the template drop-down, select one of the available monitoring templates.

  5. Click Use Template to apply your desired template.

    An image showing the template that gets populated with monitoring results

    Template that gets populated with monitoring results
    • After you select a template, the initial document where your ongoing monitoring results will be stored becomes available.
    • Review this documents structure to confirm that it meets your requirements, then start running your code to populate the template with monitoring results.

Troubleshooting

If you see errors similar to this one when running your code, make sure that you have selected your monitoring template and then re-run your code:

APIRequestError: Please select an ongoing monitoring template on the ValidMind platform first

What’s next

After you have enabled ongoing monitoring and run your code to generate some output, you can start reviewing the monitoring results.6