validmind.RegressionModelForecastPlotLevels
integrate_diff
defintegrate_diff(series_diff,start_value):
RegressionModelForecastPlotLevels
@tags('time_series_data', 'forecasting', 'visualization')
@tasks('regression')
defRegressionModelForecastPlotLevels(model:validmind.vm_models.VMModel,dataset:validmind.vm_models.VMDataset):
Assesses the alignment between forecasted and observed values in regression models through visual plots
Purpose
This test aims to visually assess the performance of a regression model by comparing its forecasted values against the actual observed values for both the raw and transformed (integrated) data. This helps determine the accuracy of the model and can help identify overfitting or underfitting. The integration is applied to highlight the trend rather than the absolute level.
Test Mechanism
This test generates two plots:
- Raw data vs forecast
- Transformed data vs forecast
The transformed data is created by performing a cumulative sum on the raw data.
Signs of High Risk
- Significant deviation between forecasted and observed values.
- Patterns suggesting overfitting or underfitting.
- Large discrepancies in the plotted forecasts, indicating potential issues with model generalizability and precision.
Strengths
- Provides an intuitive, visual way to assess multiple regression models, aiding in easier interpretation and evaluation of forecast accuracy.
Limitations
- Relies heavily on visual interpretation, which may vary between individuals.
- Does not provide a numerical metric to quantify forecast accuracy, relying solely on visual assessment.