New measure of robustness for estimators, with tight bounds for Gaussian mean estimation.
problem Developing robust statistical estimators for datasets with noise or outliers.
method Introducing empirical sensitivity as a new robustness measure and proving lower bounds for Gaussian mean estimation.
result Empirical sensitivity bounds for optimal estimators are tight, showing obstructions on mean and variance.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…
This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.
problem Large dimensional covariance matrix estimation with unknown mean under Kolmogorov asymptotics.
method Extending Ledoit-Wolf linear shrinkage to translation-invariant estimators, proving their convergence properties.
result A new estimator outperforms other standard estimators empirically.
Simple private estimators for mean and covariance outperform existing methods.
problem Private estimation of mean and covariance at small sample sizes.
method Differentially private estimators for multivariate sub-Gaussian data.
result Asymptotic error rates match theoretical bounds and outperform previous methods.
New method for high-dimensional linear regression using empirical Bayes.
problem Estimating prior in high-dimensional linear regression.
method Variational empirical Bayes approach with NPMLE and mean field approximation.
result Established asymptotic consistency and computational efficiency of the method.
The MEM method uses data-driven priors for linear inverse problems, proving convergence and estimating differences.
problem Linear inverse problems with approximate priors.
method Maximum Entropy on the Mean (MEM) method with data-driven priors.
result Empirical mean convergence and estimates for prior differences based on epigraphical distance.
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in…
Estimates multiple means in high dimensions using convex combinations.
problem Estimating multiple multi-dimensional means from samples.
method Convex combinations of empirical means with data-dependent weights.
result Our methods asymptotically approach oracle (minimax) improvement.
Transformers solve Poisson means estimation via empirical Bayes.
problem Estimating Poisson means under empirical Bayes setting.
method Pre-trained transformer learns to adapt to unknown prior and do in-context learning.
result Transformers achieve vanishing regret with large models and outperform classical algorithms.
Paper shows robust estimators converge to true risk minimizers at optimal rates.
problem Understanding asymptotic properties of robust risk minimizers.
method Investigates robust analogues of empirical risk minimization, focusing on median of means estimator.
result Robust minimizers converge to true minimizers at optimal rates and have similar asymptotic variance.
A new sequential method estimates Poisson means in streaming data, achieving optimality and efficiency.
problem Estimating Poisson means in a streaming, or online, framework.
method A quasi-Bayesian approach based on Newton's algorithm for a sequential estimate.
result Established frequentist guarantees including consistency and asymptotic optimality.
Sharp inequalities for matrix means with unknown variance.
problem Estimating matrix means with unknown variance.
method Empirical Bernstein inequalities for symmetric random matrices.
result Adapts to unknown variance with tight deviation bounds.
We propose an empirical Bayes estimator based on Dirichlet process mixture model for estimating the sparse normalized mean difference, which could be directly applied to the high dimensional linear classification. In theory, we build a bridge to connect the estimation error of the mean difference and the misclassificat…
We introduce performance-based regularization (PBR), a new approach to addressing estimation risk in data-driven optimization, to mean-CVaR portfolio optimization. We assume the available log-return data is iid, and detail the approach for two cases: nonparametric and parametric (the log-return distribution belongs in …
A new method of moments estimator goes beyond data reweighting.
problem Estimation of moment restrictions and conditional moment restrictions.
method Kernel Method of Moments (KMM) based on maximum mean discrepancy.
result KMM achieves competitive performance on conditional moment restriction tasks.
New DP methods for estimating means and frequencies with varying privacy demands.
problem Estimating statistics with users having different privacy requirements.
method Proposes algorithms for empirical mean and frequency estimation under heterogeneous privacy constraints, considering both correlated and permuted datasets.
result Establishes theoretical performance guarantees for algorithms, achieving minimax optimality.
Enhances normal mean estimation with side info using NIT approach.
problem Compound estimation of normal means with side information.
method Empirical Bayes, nonparametric integrative Tweedie (NIT) approach.
result NIT approach improves estimation risk and convergence rate with increasing auxiliary data.
Study examines mean estimation in high dimensions with small data.
problem Efficiently estimating mean in high-dimensional data with limited data size.
method Extensive experimentation of various mean estimation techniques.
result Developed robust methods for mean estimation with low data size.
The problem of estimating the kernel mean in a reproducing kernel Hilbert space (RKHS) is central to kernel methods in that it is used by classical approaches (e.g., when centering a kernel PCA matrix), and it also forms the core inference step of modern kernel methods (e.g., kernel-based non-parametric tests) that rel…
Estimates mean of random vector with near-optimal error in all directions.
problem Estimating the mean of a random vector with direction-dependent accuracy.
method Proves existence of an estimator with near-optimal error in all directions under certain conditions.
result The estimator satisfies the error bound for all directions, with probability 1-δ.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
This study evaluates shrinkage estimators for improving mean and covariance in portfolio optimization.
problem Estimation errors in expected returns and covariance matrix in mean-variance model.
method Examined five shrinkage estimators for expected returns and eleven for covariance matrix across six datasets.
result GMV model with Ledoit Wolf COV2 outperforms traditional methods in most scenarios.
New insights into empirical Bayes and compound decision problems with improved regret bounds.
problem Estimating means of normally or Poisson distributed vectors under squared loss.
method Combines Bayesian and frequentist approaches using data-driven estimators.
result Optimal regret bounds for Poisson and normal mean models, resolving conjectures.
Paper proposes robust gossip algorithms for mean and trimmed mean estimation.
problem Vulnerability of mean-based gossip algorithms to malicious nodes.
method Developed extsc{GoRank} for rank estimation and extsc{GoTrim} for trimmed mean estimation.
result Established convergence rates for rank and trimmed mean estimation.
Estimates mean and covariance for large, unbalanced stock returns panels.
problem Estimating mean and covariance in large, unbalanced panel data.
method Nonparametric, kernel-based joint estimator for conditional mean and covariance matrices.
result The idiosyncratic risk explains more than 75% of cross-sectional variance.
Extends covariance estimation with multiple targets for better performance.
problem Improving covariance estimation for multiple targets.
method Combines multiple constant matrices with sample covariance matrix, derives estimators and proves convergence.
result The multi-target linear shrinkage estimator outperforms other estimators in various situations.
Improved multi-task averaging reduces mean squared error in high-dimensional data.
problem Joint estimation of multiple distributions using independent data sets.
method Exploits similarities between tasks by shrinking naive estimators towards local averages.
result The method provides a significant reduction in mean squared error, especially in high-dimensional spaces.
Mean embeddings provide an extremely flexible and powerful tool in machine learning and statistics to represent probability distributions and define a semi-metric (MMD, maximum mean discrepancy; also called N-distance or energy distance), with numerous successful applications. The representation is constructed as the e…
Private statistics estimation faces a bias, accuracy, and privacy trilemma.
problem Balancing privacy, accuracy, and bias in statistical estimation.
method Use differential privacy (DP) for private statistics, but clip samples to control sensitivity and add noise for privacy, introducing bias.
result No algorithm can simultaneously have low bias, low error, and low privacy loss for arbitrary distributions.
Unified framework combines views and optimization for better portfolio management.
problem Optimizing portfolio weights with dynamic adjustment based on volatility.
method Dynamic sliding window adjusting horizon, factor estimates, BL posterior returns, and weights over time.
result Outperforms dynamic mean-variance optimization without BL views, providing stronger downside risk control.
PAS improves estimation of multiple means using ML predictions and shrinkage.
problem Improving statistical estimates with limited gold-standard data and noisy ML predictions.
method Prediction-Powered Adaptive Shrinkage (PAS) that combines PPI with empirical Bayes shrinkage.
result PAS adapts to the reliability of ML predictions and outperforms traditional methods in large-scale applications.
Faster convergence of kernel mean embeddings using variance information.
problem Speeding up the convergence rate of kernel mean embeddings.
method Leveraging variance information in reproducing kernel Hilbert space and estimating variance from data.
result Efficiently estimate variance information from data to achieve distribution-agnostic convergence bounds.
Study robust estimation under varying corruption probabilities in data.
problem Robust estimation in scenarios with heterogeneous corruption rates.
method Developed estimators for mean and regression under various corruption patterns.
result Optimal estimators can discard corrupted samples beyond a specific threshold.
Mean shift clustering finds the modes of the data probability density by identifying the zero points of the density gradient. Since it does not require to fix the number of clusters in advance, the mean shift has been a popular clustering algorithm in various application fields. A typical implementation of the mean shi…
This paper introduces a class of k-nearest neighbor (k-NN) estimators called bipartite plug-in (BPI) estimators for estimating integrals of non-linear functions of a probability density, such as Shannon entropy and Rényi entropy. The density is assumed to be smooth, have bounded support, and be uniformly bounded from…
Paper improves CI and CS for bounded means using betting and mixtures.
problem Estimating means of bounded random variables.
method Composite nonnegative martingales, testing by betting, method of mixtures.
result Empirically outperforms existing CI and CS methods.
The stochastic multi-armed bandit problem is well understood when the reward distributions are sub-Gaussian. In this paper we examine the bandit problem under the weaker assumption that the distributions have moments of order 1+ε, for some ε∈(0,1]. Surprisingly, moments of order 2 (i.e., finite variance) are suffi…
A novel k-NN method estimates conditional mean and variance efficiently.
problem Joint estimation of conditional mean and variance.
method Integrates k-NN with automated variance selection.
result Achieves fast convergence rates and improved precision.
The paper analyzes high-dimensional linear regression using parametric empirical Bayes methods.
problem Estimation of i.i.d. priors in high-dimensional Bayesian linear regression with random design.
method Parametric empirical Bayes estimation, variational lower bound maximization, phase transition analysis.
result The vEB estimator is information theoretically optimal up to p=o(n2/3) but sub-optimal in higher dimensions. New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
We study an optimization-based approach to con- struct a mean-reverting portfolio of assets. Our objectives are threefold: (1) design a portfolio that is well-represented by an Ornstein-Uhlenbeck process with parameters estimated by maximum likelihood, (2) select portfolios with desirable characteristics of high mean r…
Markowitz (1952, 1959) laid down the ground-breaking work on the mean-variance analysis. Under his framework, the theoretical optimal allocation vector can be very different from the estimated one for large portfolios due to the intrinsic difficulty of estimating a vast covariance matrix and return vector. This can res…
HAVER improves error bounds for estimating the largest mean in machine learning tasks.
problem Estimating the largest mean among multiple distributions.
method Proposes HAVER, a novel algorithm for maximum mean estimation.
result HAVER achieves better error bounds than the oracle in many cases.
Efficient and robust algorithms for decentralized estimation in networks are essential to many distributed systems. Whereas distributed estimation of sample mean statistics has been the subject of a good deal of attention, computation of U-statistics, relying on more expensive averaging over pairs of observations, is…
Deep neural networks improve mean function estimation for functional data.
problem Estimating mean functions of functional data.
method Deep neural networks with ReLU activation, sparsely connected.
result Achieves optimal nonparametric convergence rate in empirical norm.
We propose a new algorithm, Mean Actor-Critic (MAC), for discrete-action continuous-state reinforcement learning. MAC is a policy gradient algorithm that uses the agent's explicit representation of all action values to estimate the gradient of the policy, rather than using only the actions that were actually executed. …
Proposes a new framework for deep learning conditional mean estimation with confidence regions.
problem Lack of asymptotic properties in deep nonparametric regression models.
method Transforms deep estimation into conditional diffusion model for conditional mean estimation.
result Developed end-to-end convergence rate and asymptotic normality for conditional diffusion model.