Package {PenalReg}

Automated Penalized Regression Analysis

Description

Provides an automated framework for penalized regression analysis using Ridge Regression, Lasso Regression and Elastic Net Regression. The package performs data standardization, training-testing data partitioning, cross-validation for hyperparameter tuning, model fitting, coefficient estimation, variable importance assessment, prediction, and performance evaluation. It simplifies regularized regression analysis by integrating the complete modeling workflow into a single function suitable for researchers for better understanding of the data.

Usage

PenalReg(
  data,
  response,
  train_ratio = 0.8,
  standardize = TRUE,
  cv = 10,
  lambda = seq(0.01, 5, length.out = 10),
  alpha = seq(0, 1, 0.01),
  verbose = TRUE
)

Arguments

Details

The function automatically fits three penalized regression models:

• Ridge Regression

• Lasso Regression

• Elastic Net Regression

Predictor variables that are not numeric are converted into a numeric design matrix using model.matrix. Predictor variables can optionally be standardized using the training-set mean and standard deviation. The same training-set scaling parameters are applied to the testing data.

The data are partitioned into training and testing subsets according to train_ratio. Hyperparameters are optimized using k-fold cross-validation implemented through the caret package and the glmnet modeling method.

Model performance is evaluated using:

• Coefficient of Determination (R-squared)

• Mean Squared Error (MSE)

• Root Mean Squared Error (RMSE)

• Mean Absolute Error (MAE)

• Mean Absolute Percentage Error (MAPE)

• Root Mean Squared Percentage Error (RMSPE)

The model with the highest testing-set R-squared is identified as the best-performing model and displayed when verbose = TRUE.

Value

A list of class "PenalReg" containing:

importance

Variable importance measures for Ridge, Lasso, and Elastic Net models.

coefficients

Estimated regression coefficients for each penalized regression model.

Fitted_Values

A data frame containing observed and fitted values for the training data from Ridge, Lasso, and Elastic Net models.

Predicted_Values

A data frame containing observed and predicted values for the testing data from Ridge, Lasso, and Elastic Net models.

Selected_Parameters

A data frame containing the selected lambda and alpha values for each fitted model.

Training_Performance

A data frame containing R-squared, MSE, RMSE, MAE, MAPE, and RMSPE values calculated on the training data.

Testing_Performance

A data frame containing R-squared, MSE, RMSE, MAE, MAPE, and RMSPE values calculated on the testing data.

References

Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55–67. doi:10.1080/00401706.1970.10488634

Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B, 67(2), 301–320. doi:10.1111/j.1467-9868.2005.00503.x

Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1–22. doi:10.18637/jss.v033.i01

See Also

train, glmnet

Examples

data(mtcars)

mtcars_subset <- data.frame(
  mpg = mtcars$mpg,
  cyl = mtcars$cyl,
  disp = mtcars$disp,
  hp = mtcars$hp,
  drat = mtcars$drat,
  wt = mtcars$wt,
  qsec = mtcars$qsec,
  gear = mtcars$gear,
  carb = mtcars$carb
)

fit <- PenalReg(
  data = mtcars_subset,
  response = "mpg",
  cv = 5,
  verbose = TRUE
)

fit