arXiv research

Stein shrinkage improves BN robustness against adversarial attacks.

We propose an improved LASSO estimation technique based on Stein-rule. We shrink classical LASSO estimator using preliminary test, shrinkage, and positive-rule shrinkage principle. Simulation results have been carried out for various configurations of correlation coefficients ($r$), size of the parameter vector ($β$), …

Improved stochastic gradient estimation for deep learning in high dimensions.

Using integration by parts on Gaussian space we construct a Stein Unbiased Risk Estimator (SURE) for the drift of Gaussian processes using their local and occupation times. By almost-sure minimization of the SURE risk of shrinkage estimators we derive an estimation and de-noising procedure for an input signal perturbed…

Integrates ESG data into Black-Litterman for portfolio optimization.

We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically …

C-SURE improves complex-valued deep learning models by shrinking estimates, outperforming MLE and SurReal.

Estimates true Sharpe ratio of selected assets with various methods.

Stein showed that the multivariate sample mean is outperformed by "shrinking" to a constant target vector. Ledoit and Wolf extended this approach to the sample covariance matrix and proposed a multiple of the identity as shrinkage target. In a general framework, independent of a specific estimator, we extend the shrink…

This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimato…

SCOPE estimator improves covariance and precision matrix estimation.

New research shows shrinkage methods re-scale portfolio efficient frontiers under distributional misspecification.

A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…

Unified framework for shrinkage, thresholding, and regularization in normal mean estimation and linear regression.

We revisit the problem of feature selection in linear discriminant analysis (LDA), that is, when features are correlated. First, we introduce a pooled centroids formulation of the multiclass LDA predictor function, in which the relative weights of Mahalanobis-transformed predictors are given by correlation-adjusted $t$…

A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in…

JojoSCL improves scRNA-seq clustering by reducing intra-cluster dispersion.

Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular [Rahimi and Recht, 2007], where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration [Yang …

We introduce a distributionally robust maximum likelihood estimation model with a Wasserstein ambiguity set to infer the inverse covariance matrix of a $p$-dimensional Gaussian random vector from $n$ independent samples. The proposed model minimizes the worst case (maximum) of Stein's loss across all normal reference d…

Spatial statisticians and quantitative investors use the same mathematical object: a Schur complement, damped by one parameter.

This paper deals with the problem of nonparametric independence testing, a fundamental decision-theoretic problem that asks if two arbitrary (possibly multivariate) random variables $X,Y$ are independent or not, a question that comes up in many fields like causality and neuroscience. While quantities like correlation o…

The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.

Networks are a natural representation of complex systems across the sciences, and higher-order dependencies are central to the understanding and modeling of these systems. However, in many practical applications such as online social networks, networks are massive, dynamic, and naturally streaming, where pairwise inter…

Proposes PGPS for efficient Bayesian inference.

Improved estimator for least squares using random projections achieves smaller error.

The paper extends and applies a new shrinkage prior in Bayesian factor analysis.

A sparse modeling is a major topic in machine learning and statistics. LASSO (Least Absolute Shrinkage and Selection Operator) is a popular sparse modeling method while it has been known to yield unexpected large bias especially at a sparse representation. There have been several studies for improving this problem such…

Stein's method improves probabilistic inference and learning.

Paper proposes a new method for covariance estimation using M-estimators with eigenvalue shrinkage.

WeSpeR speeds up non-linear shrinkage for high-dimensional weighted covariance.

Improved portfolio optimization method reduces risk and improves performance.

Extends covariance estimation with multiple targets for better performance.

PAS improves estimation of multiple means using ML predictions and shrinkage.

Many machine learning algorithms require precise estimates of covariance matrices. The sample covariance matrix performs poorly in high-dimensional settings, which has stimulated the development of alternative methods, the majority based on factor models and shrinkage. Recent work of Ledoit and Wolf has extended the sh…

Estimates dependent parameters using Markovian dependence with shrinkage.

GC Stein manifolds characterized with embeddings and functions.

Stochastic Stein Discrepancies improve inference efficiency.

This paper considers the problem of estimating a high-dimensional vector of parameters $\boldsymbolθ \in \mathbb{R}^n$ from a noisy observation. The noise vector is i.i.d. Gaussian with known variance. For a squared-error loss function, the James-Stein (JS) estimator is known to dominate the simple maximum-likelihood (…

Improved estimation of higher order integrals using shrinkage techniques.

We give a bordism-theoretic characterisation of those closed almost contact (2q+1)-manifolds (with q > 2) which admit a Stein fillable contact structure. Our method is to apply Eliashberg's h-principle for Stein manifolds in the setting of Kreck's modified surgery. As an application, we show that any simply connected a…

Improved sampling method using regularized Stein Variational Gradient Flow.

Develops Stein's method for Riemannian manifolds using diffusion.

New method improves covariance estimation for weighted samples.

This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.

It is shown that every subcritical Stein manifold is deformation equivalent to the product of a Stein manifold with $\C$.

Regularized Stein thinning improves MCMC output approximations.

Study Stein and Milnor fillings of links from surface singularities.

Stein's method (Stein, 1973; 1981) is a powerful tool for statistical applications and has significantly impacted machine learning. Stein's lemma plays an essential role in Stein's method. Previous applications of Stein's lemma either required strong technical assumptions or were limited to Gaussian distributions with …