PLSCLUSTER: A Hybrid Approach Combining Partial Least Squares and Graph Clustering to Address Multicollinearity in High-Dimensional Data

Abstract

Multicollinearity continues to pose significant challenges in statistical modelling, especially in high-dimensional datasets where predictors exhibit strong linear dependencies. Traditional approaches such as ridge regression, principal components regression (PCR), and partial least squares (PLS) each address the issue in part but struggle to balance predictive performance and interpretability. This study presents and evaluates PLSCLUSTER, a novel hybrid technique that integrates graph-based clustering of correlated predictors with supervised dimension reduction using PLS regression. The method first partitions highly correlated variables into clusters based on pairwise correlation networks, selects representative variables or latent cluster components, and then applies PLS to derive stable, interpretable models. Through extensive simulations varying sample size (n = 20, 50, 100, 500), number of predictors (p = 5, 10, 15, 20), and correlation strength (ρ = 0.5, 0.7, 0.9), PLSCLUSTER is benchmarked against Ridge, Lasso, and PCA/PCR using RMSE, MAE, R², AIC, BIC, and model stability indices such as VIF and condition number. Results demonstrate that PLSCLUSTER consistently outperforms competing methods under moderate to strong multicollinearity (ρ ≥ 0.7), achieving lower prediction errors and greater coefficient stability while retaining interpretability through cluster representatives. The method is robust across dimensions and benefits from larger sample sizes, while preserving interpretability via cluster representatives. Practical implementation details (SAS macros used in the thesis) and recommended hyper parameter choices (cluster threshold, number of PLS components) are provided to guide replication and adoption.