validmind.RegressionR2Square

RegressionR2Square

@tags('sklearn', 'model_performance')

@tasks('regression')

defRegressionR2Square(dataset,model):

Assesses the overall goodness-of-fit of a regression model by evaluating R-squared (R2) and Adjusted R-squared (Adj R2) scores to determine the model's explanatory power over the dependent variable.

Purpose

The purpose of the RegressionR2Square Metric test is to measure the overall goodness-of-fit of a regression model. Specifically, this Python-based test evaluates the R-squared (R2) and Adjusted R-squared (Adj R2) scores, which are statistical measures used to assess the strength of the relationship between the model's predictors and the response variable.

Test Mechanism

The test deploys the r2_score method from the Scikit-learn metrics module to measure the R2 score on both training and test sets. This score reflects the proportion of the variance in the dependent variable that is predictable from the independent variables. The test also calculates the Adjusted R2 score, which accounts for the number of predictors in the model to penalize model complexity and reduce overfitting. The Adjusted R2 score will be smaller if unnecessary predictors are included in the model.

Signs of High Risk

  • Low R2 or Adjusted R2 scores, suggesting that the model does not explain much variation in the dependent variable.
  • Significant discrepancy between R2 scores on the training set and test set, indicating overfitting and poor generalization to unseen data.

Strengths

  • Widely-used measure in regression analysis, providing a sound general indication of model performance.
  • Easy to interpret and understand, as it represents the proportion of the dependent variable's variance explained by the independent variables.
  • Adjusted R2 score helps control overfitting by penalizing unnecessary predictors.

Limitations

  • Sensitive to the inclusion of unnecessary predictors even though Adjusted R2 penalizes complexity.
  • Less reliable in cases of non-linear relationships or when the underlying assumptions of linear regression are violated.
  • Does not provide insight on whether the correct regression model was used or if key assumptions have been met.