PAS improves estimation of multiple means using ML predictions and shrinkage.
problem Improving statistical estimates with limited gold-standard data and noisy ML predictions.
method Prediction-Powered Adaptive Shrinkage (PAS) that combines PPI with empirical Bayes shrinkage.
result PAS adapts to the reliability of ML predictions and outperforms traditional methods in large-scale applications.
GRASP simplifies Bayesian regression with grouped predictors using an adaptive NBP prior.
problem Regression with grouped predictors and adaptive shrinkage.
method Normal Beta Prime (NBP) prior with tunable hyperparameters for flexible sparsity control.
result Empirical validation of robust and versatile GRASP across various sparsity and signal-to-noise ratios.
Guided adaptive shrinkage uses co-data to improve feature selection in genomic studies.
problem Feature selection challenges in high-dimensional genomics data, especially in clinical settings.
method Guided adaptive shrinkage methods that use co-data to adapt shrinkage parameters.
result Improves feature selection in genomic studies, demonstrated through comparisons and examples.
Proposes a tail-adaptive shrinkage method for robust sparse estimation.
problem Robust Bayesian methods for high-dimensional regression under diverse sparse regimes.
method Global-local-tail (GLT) Gaussian mixture distribution with tail-adaptive shrinkage.
result GLT posterior contracts at minimax optimal rate for sparse normal mean models.
Improved stochastic gradient estimation for deep learning in high dimensions.
problem Inadmissibility of mini-batch gradients in high-dimensional settings.
method Stein-rule shrinkage applied to gradient computation.
result The proposed SR-Adam outperforms Adam in large-batch settings.
EigenBayes: A fast, adaptive Bayesian shrinkage approach for high-dimensional matrix factorization
problem Choosing the latent dimension k in factor models method Adaptive spectral shrinkage and empirical Bayes calibration
result Adapts to signal-to-noise ratio and shrinks superfluous components
Nash integrates covariate-specific side info into sparse regression via neural networks.
problem Sparse linear regression struggles with covariates exhibiting structure or coming from heterogeneous sources.
method Neural Adaptive Shrinkage (Nash) framework that integrates side information into sparse regression via neural networks. Uses split variational empirical Bayes algorithm.
result Nash improves accuracy and adaptability over existing methods in real data experiments.
A popular regularized (shrinkage) covariance estimator is the shrinkage sample covariance matrix (SCM) which shares the same set of eigenvectors as the SCM but shrinks its eigenvalues toward its grand mean. In this paper, a more general approach is considered in which the SCM is replaced by an M-estimator of scatter ma…
HS-MoE selects sparse experts using adaptive priors and data-adaptive gating.
problem Sparse expert selection in mixture-of-experts architectures.
method Combines horseshoe prior with input-dependent gating for data-adaptive sparsity.
result Data-adaptive sparsity in expert usage.
Paper introduces slow kill for efficient large-scale variable screening.
problem Challenges in variable selection and parameter estimation for big data.
method Nonconvex constrained optimization, adaptive \(\ell_2\)-shrinkage, and increasing learning rates.
result Slow kill outperforms state-of-the-art algorithms in various situations.
The paper decouples shrinkage and selection in Bayesian Quantile Regression.
problem Improving prediction accuracy in high-dimensional Bayesian Quantile Regression.
method Two-step procedure: shrinkage through continuous priors, sparsification through SAVS.
result The method reduces bias and provides interpretable variable selection.
We propose a new framework for designing estimators for off-policy evaluation in contextual bandits. Our approach is based on the asymptotically optimal doubly robust estimator, but we shrink the importance weights to minimize a bound on the mean squared error, which results in a better bias-variance tradeoff in finite…
KG-WDRO optimizes transfer learning with external knowledge.
problem Over-pessimism in WDRO for small target samples.
method KG-WDRO incorporates multiple sources of external knowledge to construct smaller Wasserstein ambiguity sets.
result KG-WDRO improves transfer learning performance and adaptivity.
New damping technique improves deep learning models by reducing noise in flat directions.
problem Improving generalization in deep learning models by reducing estimation noise in flat directions.
method Developed a novel random matrix theory based damping learner to reduce the shrinkage coefficient and improve generalization.
result Significant generalization improvements in logistic regression and deep neural networks experiments.
We derive new theoretical results on the properties of the adaptive least absolute shrinkage and selection operator (adaptive lasso) for time series regression models. In particular, we investigate the question of how to conduct finite sample inference on the parameters given an adaptive lasso model for some fixed valu…
Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.
problem Estimation of normal mean in multivariate settings with correlated observations.
method Approximate risk minimization over a functional class of shrinkage-thresholding rules.
result Unified estimator NOMAD for shrinkage, thresholding, and regularization.
Flexible co-data learning improves clinical prediction models.
problem High-dimensional clinical data challenges prediction accuracy.
method Combining domain knowledge and external studies to estimate adaptive multi-group ridge penalties.
result Improves prediction performance and variable selection stability.
Novel framework predicts brain biomarker trajectories with superior performance.
problem Challenges in estimating longitudinal brain biomarker trajectories due to variability, inconsistencies, and irregular measurements.
method Personalized deep kernel regression with Adaptive Shrinkage Estimation.
result Superior predictive performance compared to state-of-the-art models.
An efficient algorithm selects the correct number of latent dimensions in multidimensional probit models.
problem Determining the correct number of latent dimensions in multidimensional probit graded response models.
method Adaptive Bayesian dimension selection framework using cumulative ordered spike-and-slab (COSS) prior and Albert--Chib latent response augmentation.
result The proposed method accurately recovers latent structures and avoids repeated model fitting.
MLShrink integrates machine learning with wavelet shrinkage for denoising.
problem Denoising signals with uncertain magnitudes
method Combines wavelet shrinkage with machine learning
result Preserves simplicity for signal coefficients while allowing data-adaptive decisions for ambiguous coefficients
Dropout regularization of deep neural networks has been a mysterious yet effective tool to prevent overfitting. Explanations for its success range from the prevention of "co-adapted" weights to it being a form of cheap Bayesian inference. We propose a novel framework for understanding multiplicative noise in neural net…
Many real-world problems, including multi-speaker text-to-speech synthesis, can greatly benefit from the ability to meta-learn large models with only a few task-specific components. Updating only these task-specific modules then allows the model to be adapted to low-data tasks for as many steps as necessary without ris…
C-SURE improves complex-valued deep learning models by shrinking estimates, outperforming MLE and SurReal.
problem Improving accuracy and robustness of complex-valued deep learning models.
method Proposes a Stein's unbiased risk estimate (SURE) for complex-valued data and integrates it into a prototype CNN classifier.
result C-SURE outperforms SurReal and MLE in accuracy and robustness on complex-valued datasets.
In this paper, a new definition of tensor p-shrinkage nuclear norm (p-TNN) is proposed based on tensor singular value decomposition (t-SVD). In particular, it can be proved that p-TNN is a better approximation of the tensor average rank than the tensor nuclear norm when p < 1. Therefore, by employing the p-shrinkage nu…
Bayesian method for dynamic correlation matrices improves accuracy and responsiveness.
problem Challenges in estimating time-varying correlation matrices, including slow adaptation, insufficient regularization, and diffuse uncertainty.
method Low-rank factor representation with dynamic shrinkage prior and multivariate factor stochastic volatility model.
result Improved accuracy and responsiveness compared to competing methods in various challenging scenarios.
Networks are a natural representation of complex systems across the sciences, and higher-order dependencies are central to the understanding and modeling of these systems. However, in many practical applications such as online social networks, networks are massive, dynamic, and naturally streaming, where pairwise inter…
New nonconvex regularizer speeds up low-rank matrix completion.
problem Low-rank matrix completion with good theoretical and empirical performance.
method Proposes a new nonconvex regularizer with adaptive shrinkage, scalable, and fast optimization.
result Proposed method achieves state-of-the-art recovery performance and is the fastest.
New covariance estimator for financial portfolios.
problem Estimating large financial covariances in non-stationary environments.
method Exponentially weighted averages and cross-validation for nonlinearly shrinking sample eigenvalues.
result Our estimator performs well in large dimensions compared to existing estimators.
The paper extends and applies a new shrinkage prior in Bayesian factor analysis.
problem Estimating the number of factors in sparse Bayesian factor analysis.
method Introduces and extends a generalized cumulative shrinkage process (CUSP) prior.
result Exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index increases.
Improved estimation of interaction effects in regression models.
problem Estimation of interaction effects in high-dimensional regression models.
method Developed a local shrinkage model linking main effects and interactions.
result Strong improvement in estimation of regression coefficients.
WeSpeR speeds up non-linear shrinkage for high-dimensional weighted covariance.
problem Computing non-linear shrinkage formulas for high-dimensional weighted sample covariance.
method Derive extit{WeSpeR} algorithm using asymptotic sample spectrum properties.
result Significantly speeds up non-linear shrinkage in dimensions higher than 1000.
Model selection based on classical information criteria, such as BIC, is generally computationally demanding, but its properties are well studied. On the other hand, model selection based on parameter shrinkage by ℓ1-type penalties is computationally efficient. In this paper we make an attempt to combine their st…
Improved portfolio optimization method reduces risk and improves performance.
problem Minimizing risk in large portfolios with limited data.
method Combines Tikhonov regularization and direct shrinkage of portfolio weights.
result Significantly reduces out-of-sample variance and Sharpe ratio compared to existing methods.
Extends covariance estimation with multiple targets for better performance.
problem Improving covariance estimation for multiple targets.
method Combines multiple constant matrices with sample covariance matrix, derives estimators and proves convergence.
result The multi-target linear shrinkage estimator outperforms other estimators in various situations.
This paper considers improved forecasting in possibly nonlinear dynamic settings, with high-dimension predictors ("big data" environments). To overcome the curse of dimensionality and manage data and model complexity, we examine shrinkage estimation of a back-propagation algorithm of a deep neural net with skip-layer c…
Develops a sparsity-inducing Bayesian Causal Forest for estimating heterogeneous treatment effects.
problem Estimating heterogeneous treatment effects using observational data with varying degrees of sparsity.
method Introduces a sparsity-inducing version of Bayesian Causal Forests with additional priors to adjust covariate weights.
result Improves adaptability to sparse data generating processes and uncovering moderating factors driving heterogeneity.
Robust Bayes-Assisted Conformal Prediction improves prediction set sizes.
problem Misspecification of Bayesian working model and prior misalignment.
method RoBAS (Robust Bayes-Assisted Shrinkage) framework with two instantiations.
result Improves prediction set sizes in shifted settings.
Robust Bayes-Assisted Conformal Prediction improves prediction set sizes.
problem Misspecification of Bayesian working model and prior misalignment.
method RoBAS (Robust Bayes-Assisted Shrinkage) framework with two instantiations.
result Proposed scores adapt to prior quality, reducing interval widths in shifted settings.
Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on factor models and shrinkage. Recent work of Ledoit and Wolf has extended the sh…
Estimates dependent parameters using Markovian dependence with shrinkage.
problem Estimating dependent parameters from a hidden Markov model.
method Developed a novel non-parametric shrinkage algorithm combining Tweedie-based ideas and efficient state estimation.
result Superior performance compared to non-shrinkage methods in hidden Markov models.
Flexible empirical Bayes for large-scale multiple linear regression.
problem Large-scale multiple linear regression with flexible priors and efficient computation.
method Adaptive shrinkage priors combined with variational approximations for hyperparameter estimation.
result The posterior mean from the empirical Bayes method solves a penalized regression problem.
We consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likel…
Improved estimation of higher order integrals using shrinkage techniques.
problem Estimating higher order Bochner integrals in non-parametric settings.
method Shrinkage of U-statistic towards a target element, considering kernel degeneracy.
result Consistent shrinkage estimators with fast rates of convergence, even for non-degenerate kernels.
New method improves covariance estimation for weighted samples.
problem Improving covariance estimation for weighted sample data.
method Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
result Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.
problem Large dimensional covariance matrix estimation with unknown mean under Kolmogorov asymptotics.
method Extending Ledoit-Wolf linear shrinkage to translation-invariant estimators, proving their convergence properties.
result A new estimator outperforms other standard estimators empirically.
RegimeFolio optimizes portfolios by adapting to changing market regimes.
problem Non-stationary markets with shifting volatility regimes.
method Explicitly models volatility regimes with sector-specific ensemble forecasting and adaptive mean-variance allocation.
result Significant improvement in return and robustness compared to conventional methods.
Self-distillation optimally improves model performance in spiked covariance models.
problem Improving model performance in spiked covariance models.
method Developed spectral shrinkage estimators and analyzed self-distillation.
result Self-distillation achieves optimal performance among spectral shrinkage estimators for spiked covariance matrices.
Stein shrinkage improves BN robustness against adversarial attacks.
problem Improving BN robustness against adversarial attacks.
method Applying Stein shrinkage to BN mean and variance estimates.
result Stein shrinkage outperforms vanilla BN in adversarial settings.