Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

54109163217 · Jun 202019922001200920172026
48 results for multicollinear regression

Interpretable machine-learning models can be unstable under multicollinearity, leading to oscillatory weights that do not reflect meaningful contributions.

problem Interpretable machine-learning models can be unstable under multicollinearity.
method Theoretical analysis of eigenmodes of the feature correlation matrix.
result Small-eigenvalue modes associated with multicollinearity amplify fluctuations in the weights and generate oscillatory patterns that do not necessarily reflect meaningful contributions.

A new method for high-dimensional functional regression reduces multicollinearity and improves interpretability.

problem Multicollinearity, overfitting, and interpretability in high-dimensional functional linear models.
method Partition-based functional ridge regression framework.
result Improved numerical stability and enhanced interpretability without explicit variable selection.

SPPCSO addresses multicollinearity in high-dimensional data, improving model stability and predictive accuracy.

problem Multicollinearity in high-dimensional data leads to unstable estimation and reduced predictive accuracy.
method SPPCSO integrates principal component regression and L1 regularization to adaptively adjust shrinkage factors.
result SPPCSO achieves stable and reliable estimation in high-noise settings, distinguishing signal variables from noise.

The paper compares traditional regression with modern neural network methods for financial hedging and risk compression.

problem Finding optimal hedge ratios and managing portfolio risk using traditional regression methods has limitations.
method The paper introduces regularization techniques and common factor analyses using neural networks to improve upon regression methods.
result Neural network methods provide better performance in hedge ratio estimation and risk compression compared to traditional regression.

We propose a new scalable algorithm for holistic linear regression building on Bertsimas & King (2016). Specifically, we develop new theory to model significance and multicollinearity as lazy constraints rather than checking the conditions iteratively. The resulting algorithm scales with the number of samples nn in th…

2019-02-08abs ↗pdf ↗

Improved equation learning accuracy via comprehensive R²-elimination and Bayesian model selection.

problem Challenges in exhaustive equation learning due to multicollinearity and greedy steps.
method Combines R2R^2 and Bayesian model evidence for a comprehensive yet efficient search.
result Our approach surpasses all other methods in identification accuracy, especially in exact equation recovery.

Solar improves variable selection in high-dimensional data with complicated dependence structures.

problem Variable selection in ultrahigh dimensional data with severe multicollinearity and grouping effect issues.
method Subsample-ordered least angle regression (Solar) for ultrahigh dimensional data.
result Solar yields substantial improvements in sparsity, stability, and accuracy of variable selection compared to traditional methods.

The paper identifies redundant columns in matrices for feature selection and clustering.

problem Identifying redundant columns in matrices for feature selection and clustering.
method Proves that after re-ordering columns, a matrix can be block-diagonalized revealing linearly dependent columns.
result Identifies redundant columns in matrices, aiding in feature selection and clustering.

Despite the success of neural networks (NNs), there is still a concern among many over their "black box" nature. Why do they work? Here we present a simple analytic argument that NNs are in fact essentially polynomial regression models. This view will have various implications for NNs, e.g. providing an explanation for…

2018-06-13abs ↗pdf ↗

In this article, we propose a new algorithm for supervised learning methods, by which one can both capture the non-linearity in data and also find the best subset model. To produce an enhanced subset of the original variables, an ideal selection method should have the potential of adding a supplementary level of regres…

2017-01-19abs ↗pdf ↗

This paper evaluates and improves metrics for identifying important features in machine learning models.

problem Evaluation metrics for explainable AI are limited by multicollinearity and model accuracy.
method Proposes Expected Accuracy Interval (EAI) to predict model accuracy with multicollinearity.
result EAI is a useful metric for identifying important features in models with multicollinearity.

The paper derives theoretical foundations for two common machine learning variable importance measures.

problem Understanding variable importance in machine learning problems.
method The paper derives closed-form expressions for Permute-and-Predict (PaP) and Leave-One-Covariate-Out (LOCO) methods.
result Theoretical derivations explain the behavior of PaP and LOCO under collinearity, linking them to coefficients and predictor variability.

Motivation: Radiomics refers to the high-throughput mining of quantitative features from radiographic images. It is a promising field in that it may provide a non-invasive solution for screening and classification. Standard machine learning classification and feature selection techniques, however, tend to display infer…

2019-03-27abs ↗pdf ↗

Study evaluates scikit-learn regularization frameworks for machine learning models.

problem Choosing the right regularization framework for applied machine learning models.
method Empirical evaluation of four canonical frameworks (Ridge, Lasso, ElasticNet, Post-Lasso OLS) across 134,400 simulations.
result Lasso recall is highly fragile under multicollinearity; at high condition numbers (kappa) and low SNR, Lasso recall collapses to 0.18 while ElasticNet maintains 0.93.

The paper uses graph learning to detect valid instruments in high-dimensional data for house pricing.

problem Endogeneity bias and invalid instrument validation in high-dimensional data.
method Merge variable selection algorithms and probabilistic graphs to estimate house prices and causal structure.
result Efficient data-driven instrument selection and invalid instrument purge in high-dimensional data.

ROOFS helps researchers select robust biomarker features from complex data.

problem Challenges in feature selection for biomarker discovery and clinical models.
method ROOFS is a Python package that benchmarks multiple feature selection methods on user data.
result ROOFS identifies a filter method as optimal for identifying predictors of lung cancer resistance.

This paper proposes a method to select relevant features for multi-label learning.

problem Feature selection in multi-label learning to retain important information with minimal features.
method Random manifold sampling and joint sparse regularization to solve multicollinearity and obtain sparse feature sets.
result The proposed method outperforms other methods in selecting relevant features for multi-label learning.

A new algorithm for missing data imputation with low RMSE and explainability.

problem Missing data in various domains, especially in critical applications requiring low RMSE and explainability.
method DIMV algorithm that uses conditional distribution of features based on fully observed features.
result DIMV provides low RMSE, scalability, and explainability for imputed values.

Understanding the causes of crime is a longstanding issue in researcher's agenda. While it is a hard task to extract causality from data, several linear models have been proposed to predict crime through the existing correlations between crime and urban metrics. However, because of non-Gaussian distributions and multic…

2017-12-08abs ↗pdf ↗

New methods improve portfolio risk minimization by estimating covariance matrix more accurately.

problem Uncertainty in estimating covariance matrix leads to unreliable hedge trades.
method Proposes two new estimators of the inverse covariance matrix using l2 and l1 norms.
result Portfolio formed using proposed estimators achieves substantial risk reduction and improved returns.

Enhanced regime shifts detection using unstructured text and financial data.

problem Detecting regime shifts in financial markets is challenging due to noisy and multicollinear data.
method Combines LLM reasoning on unstructured text and statistical validation on financial time series.
result Framework achieves F1 score of 0.82, outperforming pure data-driven methods.

Renet improves Elastic Net by dynamically selecting between convex blending and refitting, enhancing prediction accuracy.

problem Elastic Net's shrinkage bias limits its prediction accuracy in high-dimensional settings.
method Adaptive relaxation procedure that dynamically dispatches between convex blending and efficient sub-path refitting.
result Renet consistently outperforms standard Elastic Net and Adaptive Elastic Net in high-dimensional, low signal-to-noise ratio, and high-multicollinearity scenarios.

The study finds that firm membership in flagship indices and TCFD endorsement are strong predictors of a wider Disclosure-Performance Gap.

problem The Aggregate Confusion hypothesis and the measurement of greenwashing in environmental disclosures.
method The study uses a Disclosure-Performance Gap (DPG) model to measure the divergence between voluntary environmental disclosures and realised emissions performance for 200 large European firms. The model selection process involved multiple stages and robust standard errors.
result Firm membership in flagship indices and TCFD endorsement are strong predictors of a wider gap, while renewable energy use and environmental capital expenditure significantly narrow the gap.

This paper studies robust regression in the settings of Huber's εε-contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of εε-contamination models for various regression problems including nonpa…

2017-02-15abs ↗pdf ↗

This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…

2017-02-20abs ↗pdf ↗

Survey of SDR methods for high-dimensional regression and embedding.

problem Reducing dimensionality in high-dimensional data.
method Involves both statistical and machine learning approaches, covering inverse and forward regression methods.
result Supervised Kernel Dimension Reduction is equivalent to supervised PCA.

This paper reviews SDR methods for multivariate response regression.

problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.

Paper introduces semi-supervised linear extremile regression for high-dimensional data.

problem Challenges in high-dimensional extremile regression due to data sparsity and overfitting.
method Proposes semi-supervised learning for linear extremile regression, achieving n\sqrt{n}-consistency.
result Demonstrates improved estimation efficiency and performance in high-dimensional settings.

Prevalidated ridge regression simplifies logistic regression for high-dimensional data.

problem Efficient probabilistic classification in high-dimensional data with logistic regression.
method Developed a prevalidated ridge regression model that matches logistic regression's performance but is more computationally efficient.
result Prevalidated ridge regression achieves similar classification error and log-loss to logistic regression for high-dimensional data.