Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on factor models and shrinkage. Recent work of Ledoit and Wolf has extended the sh…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “nonlinear shrinkage”
This paper considers improved forecasting in possibly nonlinear dynamic settings, with high-dimension predictors ("big data" environments). To overcome the curse of dimensionality and manage data and model complexity, we examine shrinkage estimation of a back-propagation algorithm of a deep neural net with skip-layer c…
Improved portfolio optimization method reduces risk and improves performance.
problem Minimizing risk in large portfolios with limited data.
method Combines Tikhonov regularization and direct shrinkage of portfolio weights.
result Significantly reduces out-of-sample variance and Sharpe ratio compared to existing methods.
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
Bayesian neural networks improve macroeconomic forecasting and model nonlinearities.
problem Handling small T, big K macroeconomic datasets with temporal dependence.
method Developed Bayesian neural networks with mixture activation functions, shrinkage priors, and stochastic volatility.
result BNNs produce precise density forecasts, often better than other methods.
SCOPE: Spectral Concentration by Distributionally Robust Joint Covariance-Precision Estimationstat.ML
SCOPE estimator improves covariance and precision matrix estimation.
problem Estimating covariance and precision matrices accurately.
method Distributionally robust optimization with convex spectral divergence.
result SCOPE estimator reduces spectral bias and improves condition number.
Spectral deconfounding improves machine learning models by reducing hidden confounding effects.
problem Machine learning models can be misled by hidden confounders, leading to unreliable predictions.
method Develops a nonlinear spectral deconfounding framework for gradient boosting that modifies boosting dynamics to slow down in confounding-aligned directions.
result Spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is more scalable.
New covariance estimator for financial portfolios.
problem Estimating large financial covariances in non-stationary environments.
method Exponentially weighted averages and cross-validation for nonlinearly shrinking sample eigenvalues.
result Our estimator performs well in large dimensions compared to existing estimators.
We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically …
The paper uses machine learning to forecast macroeconomic outcomes with high-dimensional data.
problem Forecasting the full conditional distribution of macroeconomic outcomes.
method Systematically integrating three key principles: high-dimensional data with regularization, rigorous out-of-sample validation, and incorporating nonlinearities.
result Regularization via shrinkage is essential to control model complexity, while nonlinearities yield limited improvements in predictive accuracy.
New estimators improve sparse semiparametric additive modeling.
problem Sparse semiparametric additive modeling with structured sparsity.
method Combines group subset selection with shrinkage for nonconvex optimization.
result New estimators outperform alternatives in synthetic and real-world data.
We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a -dimensional Gaussian random vector from independent samples. The proposed model minimizes the worst case (maximum) of Stein's loss across all normal reference d…
This paper deals with the problem of nonparametric independence testing, a fundamental decision-theoretic problem that asks if two arbitrary (possibly multivariate) random variables are independent or not, a question that comes up in many fields like causality and neuroscience. While quantities like correlation o…
Bayesian GAMs improve predictive performance for high-dimensional data.
problem Sparse regularization in GAMs leads to excess shrinkage and difficulty in selecting nonlinear effects.
method Developed a novel spike-and-slab LASSO prior and scalable EM-Coordinate Descent algorithm.
result Improved predictive and computational performance compared to existing models.
We propose a nonparametric method for detecting nonlinear causal relationship within a set of multidimensional discrete time series, by using sparse additive models (SpAMs). We show that, when the input to the SpAM is a -mixing time series, the model can be fitted by first approximating each unknown function with a …
We consider machine learning techniques to develop low-latency approximate solutions to a class of inverse problems. More precisely, we use a probabilistic approach for the problem of recovering sparse stochastic signals that are members of the -balls. In this context, we analyze the Bayesian mean-square-error …
Generalized Cumulative Shrinkage Process Priors with Applications to Sparse Bayesian Factor Analysisstat.ME
The paper extends and applies a new shrinkage prior in Bayesian factor analysis.
problem Estimating the number of factors in sparse Bayesian factor analysis.
method Introduces and extends a generalized cumulative shrinkage process (CUSP) prior.
result Exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index increases.
Since the advent of the horseshoe priors for regularization, global-local shrinkage methods have proved to be a fertile ground for the development of Bayesian methodology in machine learning, specifically for high-dimensional regression and classification problems. They have achieved remarkable success in computation, …
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simpl…
In this paper we estimate the mean-variance portfolio in the high-dimensional case using the recent results from the theory of random matrices. We construct a linear shrinkage estimator which is distribution-free and is optimal in the sense of maximizing with probability the asymptotic out-of-sample expected utilit…
Unified framework for fast large-scale portfolio optimization.
problem Efficient portfolio optimization for large-scale financial data.
method Incorporates shrinkage and regularization techniques, addressing multiple objectives.
result AP-Trees and PCA-based factor models consistently outperform other approaches in out-of-sample portfolio performance.
WeSpeR speeds up non-linear shrinkage for high-dimensional weighted covariance.
problem Computing non-linear shrinkage formulas for high-dimensional weighted sample covariance.
method Derive extit{WeSpeR} algorithm using asymptotic sample spectrum properties.
result Significantly speeds up non-linear shrinkage in dimensions higher than 1000.
Extends covariance estimation with multiple targets for better performance.
problem Improving covariance estimation for multiple targets.
method Combines multiple constant matrices with sample covariance matrix, derives estimators and proves convergence.
result The multi-target linear shrinkage estimator outperforms other estimators in various situations.
We propose a new Bayesian model for flexible nonlinear regression and classification using tree ensembles. The model is based on the RuleFit approach in Friedman and Popescu (2008) where rules from decision trees and linear terms are used in a L1-regularized regression. We modify RuleFit by replacing the L1-regularizat…
PAS improves estimation of multiple means using ML predictions and shrinkage.
problem Improving statistical estimates with limited gold-standard data and noisy ML predictions.
method Prediction-Powered Adaptive Shrinkage (PAS) that combines PPI with empirical Bayes shrinkage.
result PAS adapts to the reliability of ML predictions and outperforms traditional methods in large-scale applications.
Transformations of macroeconomic data affect machine learning forecasts, especially with regularization and nonlinearity.
problem The impact of data transformations on machine learning forecasts in macroeconomic contexts.
method Review and propose new data transformations, empirically evaluate their effects, and compare traditional and moving average rotations.
result Traditional factors should almost always be included as predictors, and moving average rotations can provide important gains.
A popular regularized (shrinkage) covariance estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward its grand mean. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter ma…
Estimates dependent parameters using Markovian dependence with shrinkage.
problem Estimating dependent parameters from a hidden Markov model.
method Developed a novel non-parametric shrinkage algorithm combining Tweedie-based ideas and efficient state estimation.
result Superior performance compared to non-shrinkage methods in hidden Markov models.
Improved estimation of higher order integrals using shrinkage techniques.
problem Estimating higher order Bochner integrals in non-parametric settings.
method Shrinkage of U-statistic towards a target element, considering kernel degeneracy.
result Consistent shrinkage estimators with fast rates of convergence, even for non-degenerate kernels.
New method improves covariance estimation for weighted samples.
problem Improving covariance estimation for weighted sample data.
method Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
result Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.
problem Large dimensional covariance matrix estimation with unknown mean under Kolmogorov asymptotics.
method Extending Ledoit-Wolf linear shrinkage to translation-invariant estimators, proving their convergence properties.
result A new estimator outperforms other standard estimators empirically.
Improved stochastic gradient estimation for deep learning in high dimensions.
problem Inadmissibility of mini-batch gradients in high-dimensional settings.
method Stein-rule shrinkage applied to gradient computation.
result The proposed SR-Adam outperforms Adam in large-batch settings.
Self-distillation optimally improves model performance in spiked covariance models.
problem Improving model performance in spiked covariance models.
method Developed spectral shrinkage estimators and analyzed self-distillation.
result Self-distillation achieves optimal performance among spectral shrinkage estimators for spiked covariance matrices.
Admissibility of Stein Shrinkage for Batch Normalization in the Presence of Adversarial Attacksstat.ML
Stein shrinkage improves BN robustness against adversarial attacks.
problem Improving BN robustness against adversarial attacks.
method Applying Stein shrinkage to BN mean and variance estimates.
result Stein shrinkage outperforms vanilla BN in adversarial settings.
Guiding adaptive shrinkage by co-data to improve regression-based prediction and feature selectionstat.ME
Guided adaptive shrinkage uses co-data to improve feature selection in genomic studies.
problem Feature selection challenges in high-dimensional genomics data, especially in clinical settings.
method Guided adaptive shrinkage methods that use co-data to adapt shrinkage parameters.
result Improves feature selection in genomic studies, demonstrated through comparisons and examples.
Physics-Informed Singular-Value Learning for Cross-Covariances Forecasting in Financial Marketsq-fin.ST
New method cleans cross-covariance matrices for better financial forecasting.
problem Asymptotically optimal cross-covariance cleaners fail in real-world, time-varying markets.
method Physics-informed neural network that learns from empirical singular values.
result Trained model outperforms analytical cleaners in out-of-sample cross-covariance prediction.
New regularization method corrects over-shrinkage in small data regression.
problem Over-shrinkage in small data regression leading to underfitting.
method Negative-capable ridge family that permits negative regularization.
result Negative regularization acts as controlled anti-shrinkage, increasing effective complexity.
Proposes an efficient shrinkage path for ridge regression.
problem Ill-conditioned data in linear models.
method A new generalized ridge regression shrinkage path that minimizes MSE risk.
result The path is as short as possible while maintaining optimal trade-off.
This study evaluates shrinkage estimators for improving mean and covariance in portfolio optimization.
problem Estimation errors in expected returns and covariance matrix in mean-variance model.
method Examined five shrinkage estimators for expected returns and eleven for covariance matrix across six datasets.
result GMV model with Ledoit Wolf COV2 outperforms traditional methods in most scenarios.
Multi-Target Shrinkagestat.ME
Stein showed that the multivariate sample mean is outperformed by "shrinking" to a constant target vector. Ledoit and Wolf extended this approach to the sample covariance matrix and proposed a multiple of the identity as shrinkage target. In a general framework, independent of a specific estimator, we extend the shrink…
GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.
problem Regression with grouped predictors and adaptive shrinkage.
method Normal Beta Prime (NBP) prior with tunable hyperparameters for flexible sparsity control.
result Empirical validation of robust and versatile GRASP across various sparsity and signal-to-noise ratios.
Variance or Standard Deviation? Shell Geometry and Global-Scale Priors in High-Dimensional Shrinkagestat.ME
High-dimensional shrinkage risk depends on the default prior for the common scale.
problem Choosing the default prior for the common scale in high-dimensional shrinkage.
method Using radial-power benchmark to compare variance-flat and standard deviation-flat priors.
result The standard deviation-flat prior has a one-unit asymptotic risk advantage near the origin.
Additive principal components (APCs for short) are a nonlinear generalization of linear principal components. We focus on smallest APCs to describe additive nonlinear constraints that are approximately satisfied by the data. Thus APCs fit data with implicit equations that treat the variables symmetrically, as opposed t…
Improved covariance matrix forecasting for S&P 500 using factor models and shrinkage.
problem Forecasting large covariance matrices of returns in finance.
method Decompose covariance matrix into firm-level factors and sectoral restrictions. Estimate using VHAR models with LASSO.
result Significantly improved forecasting precision compared to benchmarks.
In this work we construct an optimal linear shrinkage estimator for the covariance matrix in high dimensions. The recent results from the random matrix theory allow us to find the asymptotic deterministic equivalents of the optimal shrinkage intensities and estimate them consistently. The developed distribution-free es…
Liu-type Shrinkage Estimators for Mixture of Poisson Regressions with Experts: A Heart Disease Studystat.ME
Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.
problem Multicollinearity in Poisson regression models with experts.
method Ridge and Liu-type shrinkage methods.
result Shrinkage methods offer more reliable estimates for coefficients in multicollinearity.
Model for dynamic relational data with regime changes.
problem Handling abrupt changes in dynamic relational data.
method Factorized fusion shrinkage model with global-local shrinkage priors.
result Posterior distribution attains minimax optimal rate up to logarithmic factors.
Convolutional neural networks (CNNs) often perform well, but their stability is poorly understood. To address this problem, we consider the simple prototypical problem of signal denoising, where classical approaches such as nonlinear diffusion, wavelet-based methods and regularisation offer provable stability guarantee…