Proposes a hierarchical model for learning discrete Bayesian networks with shrinkage.
problem Learning discrete Bayesian networks with high-order interactions and cell probabilities.
method Hierarchical Dirichlet shrinkage model with Metropolis-adjusted Langevin algorithm for sampling.
result Efficiently learns graph structure and selects between DAGs from sparse count data.
The paper extends and applies a new shrinkage prior in Bayesian factor analysis.
problem Estimating the number of factors in sparse Bayesian factor analysis.
method Introduces and extends a generalized cumulative shrinkage process (CUSP) prior.
result Exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index increases.
Proposes DSM priors for Bayesian neural networks to improve interpretability and robustness.
problem Bayesian neural networks struggle with interpretability, overconfidence, and adversarial attacks.
method Introduces Dirichlet scale mixture (DSM) priors to address these issues.
result DSM priors lead to sparse networks, robustness against adversarial attacks, and competitive predictive performance.
The edge partition model (EPM) is a fundamental Bayesian nonparametric model for extracting an overlapping structure from binary matrix. The EPM adopts a gamma process (ΓP) prior to automatically shrink the number of active atoms. However, we empirically found that the model shrinkage of the EPM does not typically wo…
R2D2-Net improves Bayesian neural networks by preventing over-shrinkage of important weights.
problem Bayesian neural networks struggle with choosing appropriate priors, leading to over-shrinkage or poor predictive performance.
method Proposes R2D2-Net with an R^2-induced Dirichlet Decomposition prior and variational Gibbs inference algorithm.
result R2D2-Net effectively shrinks irrelevant coefficients while preventing key features from over-shrinkage.
WeSpeR speeds up non-linear shrinkage for high-dimensional weighted covariance.
problem Computing non-linear shrinkage formulas for high-dimensional weighted sample covariance.
method Derive extit{WeSpeR} algorithm using asymptotic sample spectrum properties.
result Significantly speeds up non-linear shrinkage in dimensions higher than 1000.
Improved portfolio optimization method reduces risk and improves performance.
problem Minimizing risk in large portfolios with limited data.
method Combines Tikhonov regularization and direct shrinkage of portfolio weights.
result Significantly reduces out-of-sample variance and Sharpe ratio compared to existing methods.
Extends covariance estimation with multiple targets for better performance.
problem Improving covariance estimation for multiple targets.
method Combines multiple constant matrices with sample covariance matrix, derives estimators and proves convergence.
result The multi-target linear shrinkage estimator outperforms other estimators in various situations.
PAS improves estimation of multiple means using ML predictions and shrinkage.
problem Improving statistical estimates with limited gold-standard data and noisy ML predictions.
method Prediction-Powered Adaptive Shrinkage (PAS) that combines PPI with empirical Bayes shrinkage.
result PAS adapts to the reliability of ML predictions and outperforms traditional methods in large-scale applications.
Generating user interpretable multi-class predictions in data rich environments with many classes and explanatory covariates is a daunting task. We introduce Diagonal Orthant Latent Dirichlet Allocation (DOLDA), a supervised topic model for multi-class classification that can handle both many classes as well as many co…
Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on factor models and shrinkage. Recent work of Ledoit and Wolf has extended the sh…
A popular regularized (shrinkage) covariance estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward its grand mean. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter ma…
Improved estimation of higher order integrals using shrinkage techniques.
problem Estimating higher order Bochner integrals in non-parametric settings.
method Shrinkage of U-statistic towards a target element, considering kernel degeneracy.
result Consistent shrinkage estimators with fast rates of convergence, even for non-degenerate kernels.
New method improves covariance estimation for weighted samples.
problem Improving covariance estimation for weighted sample data.
method Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
result Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.
problem Large dimensional covariance matrix estimation with unknown mean under Kolmogorov asymptotics.
method Extending Ledoit-Wolf linear shrinkage to translation-invariant estimators, proving their convergence properties.
result A new estimator outperforms other standard estimators empirically.
Improved stochastic gradient estimation for deep learning in high dimensions.
problem Inadmissibility of mini-batch gradients in high-dimensional settings.
method Stein-rule shrinkage applied to gradient computation.
result The proposed SR-Adam outperforms Adam in large-batch settings.
Self-distillation optimally improves model performance in spiked covariance models.
problem Improving model performance in spiked covariance models.
method Developed spectral shrinkage estimators and analyzed self-distillation.
result Self-distillation achieves optimal performance among spectral shrinkage estimators for spiked covariance matrices.
Stein shrinkage improves BN robustness against adversarial attacks.
problem Improving BN robustness against adversarial attacks.
method Applying Stein shrinkage to BN mean and variance estimates.
result Stein shrinkage outperforms vanilla BN in adversarial settings.
Guided adaptive shrinkage uses co-data to improve feature selection in genomic studies.
problem Feature selection challenges in high-dimensional genomics data, especially in clinical settings.
method Guided adaptive shrinkage methods that use co-data to adapt shrinkage parameters.
result Improves feature selection in genomic studies, demonstrated through comparisons and examples.
New regularization method corrects over-shrinkage in small data regression.
problem Over-shrinkage in small data regression leading to underfitting.
method Negative-capable ridge family that permits negative regularization.
result Negative regularization acts as controlled anti-shrinkage, increasing effective complexity.
Proposes an efficient shrinkage path for ridge regression.
problem Ill-conditioned data in linear models.
method A new generalized ridge regression shrinkage path that minimizes MSE risk.
result The path is as short as possible while maintaining optimal trade-off.
This study evaluates shrinkage estimators for improving mean and covariance in portfolio optimization.
problem Estimation errors in expected returns and covariance matrix in mean-variance model.
method Examined five shrinkage estimators for expected returns and eleven for covariance matrix across six datasets.
result GMV model with Ledoit Wolf COV2 outperforms traditional methods in most scenarios.
We consider the problem of simultaneous estimation of a sequence of dependent parameters that are generated from a hidden Markov model. Based on observing a noise contaminated vector of observations from such a sequence model, we consider simultaneous estimation of all the parameters irrespective of their hidden states…
Stein showed that the multivariate sample mean is outperformed by "shrinking" to a constant target vector. Ledoit and Wolf extended this approach to the sample covariance matrix and proposed a multiple of the identity as shrinkage target. In a general framework, independent of a specific estimator, we extend the shrink…
GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.
problem Regression with grouped predictors and adaptive shrinkage.
method Normal Beta Prime (NBP) prior with tunable hyperparameters for flexible sparsity control.
result Empirical validation of robust and versatile GRASP across various sparsity and signal-to-noise ratios.
High-dimensional shrinkage risk depends on the default prior for the common scale.
problem Choosing the default prior for the common scale in high-dimensional shrinkage.
method Using radial-power benchmark to compare variance-flat and standard deviation-flat priors.
result The standard deviation-flat prior has a one-unit asymptotic risk advantage near the origin.
Improved covariance matrix forecasting for S&P 500 using factor models and shrinkage.
problem Forecasting large covariance matrices of returns in finance.
method Decompose covariance matrix into firm-level factors and sectoral restrictions. Estimate using VHAR models with LASSO.
result Significantly improved forecasting precision compared to benchmarks.
In this work we construct an optimal linear shrinkage estimator for the covariance matrix in high dimensions. The recent results from the random matrix theory allow us to find the asymptotic deterministic equivalents of the optimal shrinkage intensities and estimate them consistently. The developed distribution-free es…
Developed shrinkage methods for Poisson regression models with experts to handle multicollinearity.
problem Multicollinearity in Poisson regression models with experts.
method Ridge and Liu-type shrinkage methods.
result Shrinkage methods offer more reliable estimates for coefficients in multicollinearity.
Model for dynamic relational data with regime changes.
problem Handling abrupt changes in dynamic relational data.
method Factorized fusion shrinkage model with global-local shrinkage priors.
result Posterior distribution attains minimax optimal rate up to logarithmic factors.
SCOPE estimator improves covariance and precision matrix estimation.
problem Estimating covariance and precision matrices accurately.
method Distributionally robust optimization with convex spectral divergence.
result SCOPE estimator reduces spectral bias and improves condition number.
Unified model combines shrinkage, views, and factor models for better portfolio selection.
problem Limitations of mean-variance analysis, estimation errors, and reliance on historical data.
method Bayesian approach integrating shrinkage estimation and Black-Litterman model with Fama-French factor models.
result The model outperforms simple and sample-based optimal portfolios in US equity market.
Non-linear shrinkage isn't optimal for portfolio optimization, especially when asset dependence is non-stationary.
problem Optimizing portfolios with non-stationary asset dependence structures.
method Derived and compared non-linear shrinkage with an optimal target for covariance matrix estimation.
result Non-linear shrinkage can be significantly improved for portfolio optimization.
Estimates growth loss in fund models and proposes a shrinkage method.
problem Estimating growth loss in fund models under frequentist and Bayesian estimation.
method Proposes a shrinkage method to target maximal growth with minimal deviation.
result Empirical evidence shows shrinkage gives a stable estimate closer to growth potential.
New shrinkage estimator for GMV portfolio reduces risk in high-dimensional asset settings.
problem Estimating the global minimum variance portfolio in high-dimensional settings with limited data.
method Dynamic shrinkage of the GMV portfolio using previous data as a target.
result The new estimator outperforms traditional methods in high-dimensional asset settings.
Covariance shrinkage via stochastic interpolation
problem High-dimensional covariance estimation
method Recasting shrinkage as empirical risk minimization
result Reduces statistical risk through scheduling, flow maps, and early stopping
Optimizes high-dimensional portfolios using joint shrinkage.
problem Optimizing portfolios with many assets where classical methods fail.
method Regression-based joint shrinkage method for estimating partial correlations.
result Superior performance in variance, weight, and risk estimation compared to other methods.
Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.
problem Estimation of normal mean in multivariate settings with correlated observations.
method Approximate risk minimization over a functional class of shrinkage-thresholding rules.
result Unified estimator NOMAD for shrinkage, thresholding, and regularization.
In this on-going work, I explore certain theoretical and empirical implications of data transformations under the PCA. In particular, I state and prove three theorems about PCA, which I paraphrase as follows: 1). PCA without discarding eigenvector rows is injective, but looses this injectivity when eigenvector rows are…
MLShrink integrates machine learning with wavelet shrinkage for denoising.
problem Denoising signals with uncertain magnitudes
method Combines wavelet shrinkage with machine learning
result Preserves simplicity for signal coefficients while allowing data-adaptive decisions for ambiguous coefficients
The paper calibrates shrinkage covariance estimators for spectral functionals in high dimensions.
problem Calibrating shrinkage covariance estimators for spectral functionals in high dimensions.
method Derives first-order null laws, distribution-free Davis-Kahan bands, and calibrated tests for spectral functionals under shrinkage.
result Calibrated tests and intervals for spectral functionals are provided, addressing the issue of estimation noise and shrinkage bias.
Efficiently estimates shrinkage coefficient for RTME using LOOCV approximation.
problem Estimating optimal shrinkage coefficient for Regularized Tyler's M-estimator.
method Proposes an approximate LOOCV method to estimate α efficiently. result Significant speedup and accuracy improvement over existing methods.
A new method for linear regression using feature graphs and hierarchical shrinkage.
problem Estimating robust parameters for linear regression models.
method Hierarchical Feature Regression (HFR) estimator that constructs a supervised feature graph to shrink parameters towards group targets.
result Demonstrates good predictive accuracy and versatility compared to other regularization techniques.
Proposes a tail-adaptive shrinkage method for robust sparse estimation.
problem Robust Bayesian methods for high-dimensional regression under diverse sparse regimes.
method Global-local-tail (GLT) Gaussian mixture distribution with tail-adaptive shrinkage.
result GLT posterior contracts at minimax optimal rate for sparse normal mean models.
We test the price momentum effect in the Korean stock markets under the momentum universe shrinkage to subuniverses of the KOSPI 200. Performance of the momentum strategy is not homogeneous with respect to change of the momentum universe. It is found that some submarkets generate the higher momentum returns than other …
We propose a new framework for designing estimators for off-policy evaluation in contextual bandits. Our approach is based on the asymptotically optimal doubly robust estimator, but we shrink the importance weights to minimize a bound on the mean squared error, which results in a better bias-variance tradeoff in finite…
In this work we construct an optimal shrinkage estimator for the precision matrix in high dimensions. We consider the general asymptotics when the number of variables p→∞ and the sample size n→∞ so that p/n→c∈(0,+∞). The precision matrix is estimated directly, wit…
We forecast S&P 500 excess returns using a flexible Bayesian econometric state space model with non-Gaussian features at several levels. More precisely, we control for overparameterization via novel global-local shrinkage priors on the state innovation variances as well as the time-invariant part of the state space mod…