Stein shrinkage improves BN robustness against adversarial attacks.
problem Improving BN robustness against adversarial attacks.
method Applying Stein shrinkage to BN mean and variance estimates.
result Stein shrinkage outperforms vanilla BN in adversarial settings.
We propose an improved LASSO estimation technique based on Stein-rule. We shrink classical LASSO estimator using preliminary test, shrinkage, and positive-rule shrinkage principle. Simulation results have been carried out for various configurations of correlation coefficients (r), size of the parameter vector (β), …
Improved stochastic gradient estimation for deep learning in high dimensions.
problem Inadmissibility of mini-batch gradients in high-dimensional settings.
method Stein-rule shrinkage applied to gradient computation.
result The proposed SR-Adam outperforms Adam in large-batch settings.
Using integration by parts on Gaussian space we construct a Stein Unbiased Risk Estimator (SURE) for the drift of Gaussian processes using their local and occupation times. By almost-sure minimization of the SURE risk of shrinkage estimators we derive an estimation and de-noising procedure for an input signal perturbed…
Integrates ESG data into Black-Litterman for portfolio optimization.
problem Optimizing portfolios with ESG considerations.
method Black-Litterman framework with Stein shrinkage for ESG bias, multivariate affine normal-inverse Gaussian model, CVaR risk measure, daily reallocation.
result Successful portfolio optimization with returns of 40-45% annually.
We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically …
C-SURE improves complex-valued deep learning models by shrinking estimates, outperforming MLE and SurReal.
problem Improving accuracy and robustness of complex-valued deep learning models.
method Proposes a Stein's unbiased risk estimate (SURE) for complex-valued data and integrates it into a prototype CNN classifier.
result C-SURE outperforms SurReal and MLE in accuracy and robustness on complex-valued datasets.
Estimates true Sharpe ratio of selected assets with various methods.
problem Estimating the true Sharpe ratio of a selected asset with high in-sample ratio.
method Polyhedral lemma, James Stein shrinkage, debiasing, thresholding, empirical Bayes.
result James Stein estimator performs best across various parameter values.
Stein showed that the multivariate sample mean is outperformed by "shrinking" to a constant target vector. Ledoit and Wolf extended this approach to the sample covariance matrix and proposed a multiple of the identity as shrinkage target. In a general framework, independent of a specific estimator, we extend the shrink…
This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimato…
SCOPE estimator improves covariance and precision matrix estimation.
problem Estimating covariance and precision matrices accurately.
method Distributionally robust optimization with convex spectral divergence.
result SCOPE estimator reduces spectral bias and improves condition number.
New research shows shrinkage methods re-scale portfolio efficient frontiers under distributional misspecification.
problem Poor performance of mean-variance portfolio decisions under distributional assumptions.
method Investigation of shrinkage methods under different distributional assumptions (auto-correlation, skewness, excess kurtosis).
result Shrinkage methods re-scale the sample efficient frontier, implying standard comparison methods are flawed.
A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…
Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.
problem Estimation of normal mean in multivariate settings with correlated observations.
method Approximate risk minimization over a functional class of shrinkage-thresholding rules.
result Unified estimator NOMAD for shrinkage, thresholding, and regularization.
We revisit the problem of feature selection in linear discriminant analysis (LDA), that is, when features are correlated. First, we introduce a pooled centroids formulation of the multiclass LDA predictor function, in which the relative weights of Mahalanobis-transformed predictors are given by correlation-adjusted t…
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in…
JojoSCL improves scRNA-seq clustering by reducing intra-cluster dispersion.
problem High dimensionality and sparsity of scRNA-seq data challenge clustering models.
method Integrates shrinkage estimator and contrastive learning for improved clustering.
result JojoSCL outperforms existing methods on ten scRNA-seq datasets.
Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang …
We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a p-dimensional Gaussian random vector from n independent samples. The proposed model minimizes the worst case (maximum) of Stein's loss across all normal reference d…
Spatial statisticians and quantitative investors use the same mathematical object: a Schur complement, damped by one parameter.
problem The Schur complement is used in both spatial modeling and portfolio allocation, but the parameters are different.
method The Schur complement is interpreted as reliability shrinkage of a conditional Gaussian.
result The Schur complement is the same in both applications.
This paper deals with the problem of nonparametric independence testing, a fundamental decision-theoretic problem that asks if two arbitrary (possibly multivariate) random variables X,Y are independent or not, a question that comes up in many fields like causality and neuroscience. While quantities like correlation o…
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
Networks are a natural representation of complex systems across the sciences, and higher-order dependencies are central to the understanding and modeling of these systems. However, in many practical applications such as online social networks, networks are massive, dynamic, and naturally streaming, where pairwise inter…
Proposes PGPS for efficient Bayesian inference.
problem Efficient sampling from complex posterior distributions.
method Path-guided particle-based sampling with Log-weighted Shrinkage.
result PGPS generates samples closer to target distribution.
Improved estimator for least squares using random projections achieves smaller error.
problem Improving the accuracy of least squares solutions for large-scale problems.
method James-Stein estimator applied to Gaussian sketching of least squares problems.
result Upper and lower bounds match when SNR is small and data matrix is well-conditioned.
The paper extends and applies a new shrinkage prior in Bayesian factor analysis.
problem Estimating the number of factors in sparse Bayesian factor analysis.
method Introduces and extends a generalized cumulative shrinkage process (CUSP) prior.
result Exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index increases.
A sparse modeling is a major topic in machine learning and statistics. LASSO (Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling method while it has been known to yield unexpected large bias especially at a sparse representation. There have been several studies for improving this problem such…
Stein's method improves probabilistic inference and learning.
problem Improving probabilistic inference and learning methods.
method Constructing Stein discrepancies from Stein operators and Stein sets, discussing their properties.
result Connection between Stein operators and Stein variational gradient descent.
Paper proposes a new method for covariance estimation using M-estimators with eigenvalue shrinkage.
problem Estimating covariance matrices in heavy-tailed distributions.
method Replaces shrinkage sample covariance matrix with M-estimator of scatter matrix and optimizes shrinkage parameter.
result Shrinkage M-estimators outperform shrinkage SCM in heavy-tailed distributions.
WeSpeR speeds up non-linear shrinkage for high-dimensional weighted covariance.
problem Computing non-linear shrinkage formulas for high-dimensional weighted sample covariance.
method Derive extit{WeSpeR} algorithm using asymptotic sample spectrum properties.
result Significantly speeds up non-linear shrinkage in dimensions higher than 1000.
Improved portfolio optimization method reduces risk and improves performance.
problem Minimizing risk in large portfolios with limited data.
method Combines Tikhonov regularization and direct shrinkage of portfolio weights.
result Significantly reduces out-of-sample variance and Sharpe ratio compared to existing methods.
Extends covariance estimation with multiple targets for better performance.
problem Improving covariance estimation for multiple targets.
method Combines multiple constant matrices with sample covariance matrix, derives estimators and proves convergence.
result The multi-target linear shrinkage estimator outperforms other estimators in various situations.
PAS improves estimation of multiple means using ML predictions and shrinkage.
problem Improving statistical estimates with limited gold-standard data and noisy ML predictions.
method Prediction-Powered Adaptive Shrinkage (PAS) that combines PPI with empirical Bayes shrinkage.
result PAS adapts to the reliability of ML predictions and outperforms traditional methods in large-scale applications.
Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on factor models and shrinkage. Recent work of Ledoit and Wolf has extended the sh…
Estimates dependent parameters using Markovian dependence with shrinkage.
problem Estimating dependent parameters from a hidden Markov model.
method Developed a novel non-parametric shrinkage algorithm combining Tweedie-based ideas and efficient state estimation.
result Superior performance compared to non-shrinkage methods in hidden Markov models.
GC Stein manifolds characterized with embeddings and functions.
problem Characterize GC Stein manifolds using embeddings and functions.
method Extended Cartan's Theorem A and B, defined L-plurisubharmonic functions, established GH embeddings. result Characterized GC Stein manifolds via L-plurisubharmonic exhaustion functions and GH embeddings. This paper considers the problem of estimating a high-dimensional vector of parameters θ∈Rn from a noisy observation. The noise vector is i.i.d. Gaussian with known variance. For a squared-error loss function, the James-Stein (JS) estimator is known to dominate the simple maximum-likelihood (…
Stochastic Stein Discrepancies improve inference efficiency.
problem Intractable computation of Stein discrepancies.
method Subsampled approximations of Stein operators.
result Stochastic Stein Discrepancies inherit convergence properties of standard SDs.
Improved estimation of higher order integrals using shrinkage techniques.
problem Estimating higher order Bochner integrals in non-parametric settings.
method Shrinkage of U-statistic towards a target element, considering kernel degeneracy.
result Consistent shrinkage estimators with fast rates of convergence, even for non-degenerate kernels.
Extends Stein's lemma to exponential-family mixtures for gradient computation.
problem Computing gradients for complex distributions with weak assumptions.
method Generalizes Stein's lemma to exponential-family mixtures and applies it to reparameterization trick.
result Derives new gradient identities for various distributions.
We give a bordism-theoretic characterisation of those closed almost contact (2q+1)-manifolds (with q > 2) which admit a Stein fillable contact structure. Our method is to apply Eliashberg's h-principle for Stein manifolds in the setting of Kreck's modified surgery. As an application, we show that any simply connected a…
New method improves covariance estimation for weighted samples.
problem Improving covariance estimation for weighted sample data.
method Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
result Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
Improved sampling method using regularized Stein Variational Gradient Flow.
problem Improving the accuracy of sampling methods in machine learning.
method Proposed Regularized Stein Variational Gradient Flow to interpolate between SVGD and Wasserstein Gradient Flow.
result Established theoretical properties and provided preliminary numerical evidence of improved performance.
Develops Stein's method for Riemannian manifolds using diffusion.
problem Bounding integral metrics on probability measures on Riemannian manifolds.
method Exploits the relationship between diffusion generators and Stein operators to derive Stein factors.
result Derives curvature-dependent Stein factors that generalize existing results for Euclidean spaces.
This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.
problem Large dimensional covariance matrix estimation with unknown mean under Kolmogorov asymptotics.
method Extending Ledoit-Wolf linear shrinkage to translation-invariant estimators, proving their convergence properties.
result A new estimator outperforms other standard estimators empirically.
It is shown that every subcritical Stein manifold is deformation equivalent to the product of a Stein manifold with $\C$.
Regularized Stein thinning improves MCMC output approximations.
problem Pathologies in Stein thinning leading to poor approximations.
method Theoretical analysis and regularization to improve KSD.
result Regularized Stein thinning alleviates pathologies and improves efficiency.
Study Stein and Milnor fillings of links from surface singularities.
problem Comparing Stein and Milnor fillings of links from surface singularities.
method Analyzing the topology and obstructions of Stein fillings and Milnor fillings.
result Milnor fillings have bounded topology, while Stein fillings can be more varied.