This review explores XAI methods and multicollinearity issues.
problem Multicollinearity affects the interpretability of AI models.
method A review of current XAI approaches and their limitations.
result Current XAI methods struggle with multicollinearity.
Improved Fréchet regression tackles noise and multicollinearity.
problem Addressing noise and multicollinearity in multi-label regression.
method Implicit regularization framework for explicit modeling of relationships.
result Effective modeling of complex dependencies without introducing biases.
Interpretable machine-learning models can be unstable under multicollinearity, leading to oscillatory weights that do not reflect meaningful contributions.
problem Interpretable machine-learning models can be unstable under multicollinearity.
method Theoretical analysis of eigenmodes of the feature correlation matrix.
result Small-eigenvalue modes associated with multicollinearity amplify fluctuations in the weights and generate oscillatory patterns that do not necessarily reflect meaningful contributions.
Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.
problem Multicollinearity in Poisson regression models with experts.
method Ridge and Liu-type shrinkage methods.
result Shrinkage methods offer more reliable estimates for coefficients in multicollinearity.
The paper identifies redundant columns in matrices for feature selection and clustering.
problem Identifying redundant columns in matrices for feature selection and clustering.
method Proves that after re-ordering columns, a matrix can be block-diagonalized revealing linearly dependent columns.
result Identifies redundant columns in matrices, aiding in feature selection and clustering.
Automates model selection for GLMs using optimization.
problem Automating model selection for generalized linear models.
method Mixed-integer conic optimization for feature subset selection and model constraints.
result Optimization of AIC and BIC criteria with multicollinearity constraints.
A new method for high-dimensional functional regression reduces multicollinearity and improves interpretability.
problem Multicollinearity, overfitting, and interpretability in high-dimensional functional linear models.
method Partition-based functional ridge regression framework.
result Improved numerical stability and enhanced interpretability without explicit variable selection.
Method introduces topological regularization using information filtering networks.
problem Sparse probabilistic modeling and multicollinear regression.
method Topological regularization via information filtering network.
result Direct application to L0-norm regularized problems. SPPCSO addresses multicollinearity in high-dimensional data, improving model stability and predictive accuracy.
problem Multicollinearity in high-dimensional data leads to unstable estimation and reduced predictive accuracy.
method SPPCSO integrates principal component regression and L1 regularization to adaptively adjust shrinkage factors.
result SPPCSO achieves stable and reliable estimation in high-noise settings, distinguishing signal variables from noise.
This paper evaluates and improves metrics for identifying important features in machine learning models.
problem Evaluation metrics for explainable AI are limited by multicollinearity and model accuracy.
method Proposes Expected Accuracy Interval (EAI) to predict model accuracy with multicollinearity.
result EAI is a useful metric for identifying important features in models with multicollinearity.
We propose a new scalable algorithm for holistic linear regression building on Bertsimas & King (2016). Specifically, we develop new theory to model significance and multicollinearity as lazy constraints rather than checking the conditions iteratively. The resulting algorithm scales with the number of samples n in th…
The paper compares traditional regression with modern neural network methods for financial hedging and risk compression.
problem Finding optimal hedge ratios and managing portfolio risk using traditional regression methods has limitations.
method The paper introduces regularization techniques and common factor analyses using neural networks to improve upon regression methods.
result Neural network methods provide better performance in hedge ratio estimation and risk compression compared to traditional regression.
Improved equation learning accuracy via comprehensive R²-elimination and Bayesian model selection.
problem Challenges in exhaustive equation learning due to multicollinearity and greedy steps.
method Combines R2 and Bayesian model evidence for a comprehensive yet efficient search. result Our approach surpasses all other methods in identification accuracy, especially in exact equation recovery.
Solar improves variable selection in high-dimensional data with complicated dependence structures.
problem Variable selection in ultrahigh dimensional data with severe multicollinearity and grouping effect issues.
method Subsample-ordered least angle regression (Solar) for ultrahigh dimensional data.
result Solar yields substantial improvements in sparsity, stability, and accuracy of variable selection compared to traditional methods.
Unified framework for calculating Shapley values with correlated features.
problem Inaccurate Shapley values due to multicollinearity.
method Unified framework with matrix formulation and mathematical proof.
result Shapley values become independent of feature correlations.
This paper proposes a method to select relevant features for multi-label learning.
problem Feature selection in multi-label learning to retain important information with minimal features.
method Random manifold sampling and joint sparse regularization to solve multicollinearity and obtain sparse feature sets.
result The proposed method outperforms other methods in selecting relevant features for multi-label learning.
Despite the success of neural networks (NNs), there is still a concern among many over their "black box" nature. Why do they work? Here we present a simple analytic argument that NNs are in fact essentially polynomial regression models. This view will have various implications for NNs, e.g. providing an explanation for…
In recent years, real estate industry has captured government and public attention around the world. The factors influencing the prices of real estate are diversified and complex. However, due to the limitations and one-sidedness of their respective views, they did not provide enough theoretical basis for the fluctuati…
In this article, we propose a new algorithm for supervised learning methods, by which one can both capture the non-linearity in data and also find the best subset model. To produce an enhanced subset of the original variables, an ideal selection method should have the potential of adding a supplementary level of regres…
Motivation: Radiomics refers to the high-throughput mining of quantitative features from radiographic images. It is a promising field in that it may provide a non-invasive solution for screening and classification. Standard machine learning classification and feature selection techniques, however, tend to display infer…
Understanding the causes of crime is a longstanding issue in researcher's agenda. While it is a hard task to extract causality from data, several linear models have been proposed to predict crime through the existing correlations between crime and urban metrics. However, because of non-Gaussian distributions and multic…
Study evaluates scikit-learn regularization frameworks for machine learning models.
problem Choosing the right regularization framework for applied machine learning models.
method Empirical evaluation of four canonical frameworks (Ridge, Lasso, ElasticNet, Post-Lasso OLS) across 134,400 simulations.
result Lasso recall is highly fragile under multicollinearity; at high condition numbers (kappa) and low SNR, Lasso recall collapses to 0.18 while ElasticNet maintains 0.93.
New methods improve portfolio risk minimization by estimating covariance matrix more accurately.
problem Uncertainty in estimating covariance matrix leads to unreliable hedge trades.
method Proposes two new estimators of the inverse covariance matrix using l2 and l1 norms.
result Portfolio formed using proposed estimators achieves substantial risk reduction and improved returns.
Linear regression models are not as interpretable as commonly believed.
problem Interpretability of linear regression models is often overlooked.
method Analysis of common XAI metrics and challenges faced by linear regression models.
result Linear regression models are not inherently interpretable and require careful consideration.
Penalized likelihood approaches are widely used for high-dimensional regression. Although many methods have been proposed and the associated theory is now well-developed, the relative efficacy of different approaches in finite-sample settings, as encountered in practice, remains incompletely understood. There is theref…
Enhanced regime shifts detection using unstructured text and financial data.
problem Detecting regime shifts in financial markets is challenging due to noisy and multicollinear data.
method Combines LLM reasoning on unstructured text and statistical validation on financial time series.
result Framework achieves F1 score of 0.82, outperforming pure data-driven methods.
The paper derives theoretical foundations for two common machine learning variable importance measures.
problem Understanding variable importance in machine learning problems.
method The paper derives closed-form expressions for Permute-and-Predict (PaP) and Leave-One-Covariate-Out (LOCO) methods.
result Theoretical derivations explain the behavior of PaP and LOCO under collinearity, linking them to coefficients and predictor variability.
The paper uses graph learning to detect valid instruments in high-dimensional data for house pricing.
problem Endogeneity bias and invalid instrument validation in high-dimensional data.
method Merge variable selection algorithms and probabilistic graphs to estimate house prices and causal structure.
result Efficient data-driven instrument selection and invalid instrument purge in high-dimensional data.
ROOFS helps researchers select robust biomarker features from complex data.
problem Challenges in feature selection for biomarker discovery and clinical models.
method ROOFS is a Python package that benchmarks multiple feature selection methods on user data.
result ROOFS identifies a filter method as optimal for identifying predictors of lung cancer resistance.
A new algorithm for missing data imputation with low RMSE and explainability.
problem Missing data in various domains, especially in critical applications requiring low RMSE and explainability.
method DIMV algorithm that uses conditional distribution of features based on fully observed features.
result DIMV provides low RMSE, scalability, and explainability for imputed values.
Renet improves Elastic Net by dynamically selecting between convex blending and refitting, enhancing prediction accuracy.
problem Elastic Net's shrinkage bias limits its prediction accuracy in high-dimensional settings.
method Adaptive relaxation procedure that dynamically dispatches between convex blending and efficient sub-path refitting.
result Renet consistently outperforms standard Elastic Net and Adaptive Elastic Net in high-dimensional, low signal-to-noise ratio, and high-multicollinearity scenarios.
The study finds that firm membership in flagship indices and TCFD endorsement are strong predictors of a wider Disclosure-Performance Gap.
problem The Aggregate Confusion hypothesis and the measurement of greenwashing in environmental disclosures.
method The study uses a Disclosure-Performance Gap (DPG) model to measure the divergence between voluntary environmental disclosures and realised emissions performance for 200 large European firms. The model selection process involved multiple stages and robust standard errors.
result Firm membership in flagship indices and TCFD endorsement are strong predictors of a wider gap, while renewable energy use and environmental capital expenditure significantly narrow the gap.