Quantum algorithm estimates mean with sub-Gaussian error.
problem Estimating mean of quantum-computed random variables.
method Quantum mean estimation algorithm with sub-Gaussian error rate.
result Achieves nearly-optimal quadratic speedup over classical methods.
We tackle the problem of estimating a location parameter with differential privacy guarantees and sub-Gaussian deviations. Recent work in statistics has focused on the study of estimators that achieve sub-Gaussian type deviations even for heavy tailed data. We revisit some of these estimators through the lens of differ…
New study shows mean estimation algorithms can't beat sub-Gaussian rate in general.
problem Improving mean estimation beyond worst-case scenarios.
method Constructing counterexamples and introducing neighborhood optimality.
result No reasonable estimator can achieve better than sub-Gaussian error rate for any distribution.
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly …
New characterization limits sampling with inexact scores.
problem Limiting sampling with inexact scores for unbiased results.
method Characterized types of inexact score oracle access.
result Weaker error assumptions rule out tractability of unbiased sampling.
Paper presents robust clustering methods for general mixture models.
problem Clustering with sub-Gaussian error assumptions often invalid in practice.
method Hybrid clustering with robust centroid estimate and data-driven initialization.
result Provably near-optimal mislabeling guarantees for general error distributions.
Score-based diffusion models achieve optimal error bounds under non-parametric assumptions.
problem Improving the minimax optimality of score-based diffusion models.
method Kernel-based score estimation and early stopping strategy.
result Achieves minimax optimal error bounds under sub-Gaussian and Sobolev space assumptions.
New bounds for kernel regression under non-Gaussian noise.
problem Uncertainty quantification for function estimates from noisy observations.
method Novel non-asymptotic probabilistic uniform error bounds for kernel-based regression.
result Proposed bounds apply to a broad class of non-Gaussian noise distributions.
Study shows how over-parameterized classifiers can still perform well on noisy data.
problem Understanding how maximum margin classifiers perform in over-parameterized settings with noisy data.
method Analyzes maximum margin classifiers on sub-Gaussian mixtures, providing risk bounds.
result Characterizes conditions for 'benign overfitting' in linear classification problems.
New winsorized mean improves robustness to up to 50% contamination.
problem Improving robustness of mean estimation in the presence of outliers.
method Outlyingness-induced winsorized mean approach.
result Achieves up to 50% contamination robustness with sub-Gaussian performance.
New algorithm guarantees performance on noisy data.
problem Learning with noisy data and heavy-tailed distributions.
method Anytime online-to-batch conversion for smooth objectives.
result Stochastic gradient-based algorithm with sub-Gaussian error bounds.
We propose an estimator for the mean of a random vector in R d \mathbb{R}^d R d that can be computed in time O ( n 4 + n 2 d ) O(n^4+n^2d) O ( n 4 + n 2 d ) for n n n i.i.d.~samples and that has error bounds matching the sub-Gaussian case. The only assumptions we make about the data distribution are that it has finite mean and covariance; in particular, we mak…
DP-PCA improves privacy in PCA computations with optimal statistical error.
problem Differentially private principal component analysis with sub-linear sample complexity.
method Private minibatch gradient ascent with private mean estimation.
result Achieves optimal statistical error rates for sub-Gaussian data with n = i l d e O ( d ) n= ilde O(d) n = i l d e O ( d ) samples. New method improves privacy in linear regression with optimal error bounds.
problem Differentially private linear regression with suboptimal error bounds.
method One-pass mini-batch stochastic gradient descent (DP-AMBSSGD) with adaptive clipping.
result Nearly optimal error bounds in terms of key parameters like dimensionality, number of points, and noise standard deviation.
New method tightens sub-Gaussian concentration inequalities.
problem Estimating variance-type parameters of sub-Gaussian distributions.
method Using sub-Gaussian intrinsic moment norm to maximize normalized moments.
result Provides tighter sub-Gaussian concentration inequalities.
Robust clustering algorithm for datasets with outliers.
problem Clustering with arbitrary outliers.
method Spectral clustering with a rounding scheme on a Gaussian kernel matrix.
result Misclassification error decays exponentially with signal-to-noise ratio.
The paper strengthens the classical result of MLE convergence to a Gaussian distribution.
problem The classical result of MLE convergence to a Gaussian distribution.
method Sub-Gaussian concentration and entropic normality of the normalized MLE.
result Entropic central limit theorem for a smoothed version of the estimator.
Paper analyzes neural network models for sub-Gaussian distributions, proving approximation and generalization abilities.
problem Estimating unknown distributions from i.i.d. observations using neural network models.
method Score-based neural network generative models (SGMs) with specific network architectures and stopping strategies.
result SGMs can approximate scores with high accuracy and achieve nearly optimal convergence rates under mild assumptions.
Improved generalization bounds for SGD in non-convex learning.
problem Understanding generalization properties of SGD in non-convex settings.
method Introducing Type II perturbed SGD (T2pm-SGD) to analyze generalization error bounds.
result Tighter generalization error bounds for SGD in non-convex learning, especially for sub-Gaussian and bounded loss functions.
A new algorithm estimates mean adaptively to covariance, faster and more flexible than existing methods.
problem Estimating mean of a distribution with unknown covariance efficiently and privately.
method Adaptive differentially private algorithm with optimal convergence rates and near-linear sample complexity.
result Achieves optimal rates of convergence with respect to the Mahalanobis norm ∣ ∣ ⋅ ∣ ∣ Σ ||\cdot||_Σ ∣∣ ⋅ ∣ ∣ Σ . Efficiently estimates covariance for sub-Weibull vectors with sub-Gaussian rate.
problem Outliers in high-dimensional covariance estimation.
method Cross-Fitted Norm-Truncated Estimator for Sub-Weibull distributions.
result Achieves optimal sub-Gaussian rate with O ( N d 2 ) O(Nd^2) O ( N d 2 ) operations. Paper tackles robust matrix completion with heavy-tailed noise.
problem Estimating a low-rank matrix from noisy incomplete data.
method Adaptive Huber loss for robustness, nonconvex algorithm with spectral initialization.
result Achieves minimax-optimal statistical estimation error under bounded second moment condition.
Optimizes sub-Gaussian matrices for preserving data distances.
problem Improving the performance of sub-Gaussian matrices in preserving data distances.
method Analyzes sub-Gaussian matrices and their dependence on the sub-Gaussian norm, presenting optimal bounds.
result Optimal dependence on the sub-Gaussian norm for sub-Gaussian matrices as near isometries on sets.
Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian α α α -stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that e…
Proposes a new model for mixed membership in Gaussian mixture.
problem Limited to single component membership in Gaussian mixture models.
method Mixed membership sub-Gaussian model, spectral algorithm.
result Estimation error can be made arbitrarily small with high probability.
Sharp sub-Gaussian bounds for subsolutions of Trudinger's equation on Riemannian manifolds.
problem Bounding weak subsolutions of Trudinger's equation on Riemannian manifolds.
method Proving sub-Gaussian upper bounds for weak subsolutions.
result The upper bounds are sharp for specific classes of manifolds, including \(\mathbb{R}^{n}\).
Paper uses Stochastic Mirror Descent for large-scale sparse recovery problems.
problem Statistical estimation of high-dimensional sparse parameters.
method Non-Euclidean Composite Stochastic Mirror Descent (CSMD) algorithm for solving penalized stochastic optimization problems.
result The proposed algorithm achieves optimal convergence in sparse Generalized Linear Regression problems.
UCB algorithm adapted for large-scale, non-sub-Gaussian problems.
problem Selecting the best alternative from a large set of options with non-sub-Gaussian performance distributions.
method Adapted UCB algorithm for non-sub-Gaussian settings, focusing on sample size and meta-UCB selection.
result UCB algorithms can achieve sample optimality in large-scale, non-sub-Gaussian problems.
The paper tackles best arm identification in contaminated bandits with optimal error guarantees and sample complexity.
problem Best arm identification in stochastic bandits with adversarial reward contamination.
method Proposes two algorithms: a gap-based algorithm and a successive elimination-based algorithm for sub-Gaussian bandits.
result Asymptotically optimal sample complexity for both algorithms.
Proves new concentration inequalities for sub-gaussian and sub-exponential variables.
problem Understanding functions of independent random variables better.
method Sub-gaussian and sub-exponential conditions, Rademacher complexities, Lipschitz function classes.
result Extension of Rademacher complexities to unbounded sub-exponential distributions.
Study improves self-normalized bounds for vector-valued processes beyond sub-Gaussianity.
problem Limited understanding of self-normalized concentration for vector-valued processes outside sub-Gaussian frameworks.
method Developed concentration inequalities for self-normalized processes with light tails (e.g., Bennett, Bernstein bounds) for vector-valued data.
result Provided new insights and bounds for self-normalized processes with non-sub-Gaussian distributions.
Concentration inequalities form an essential toolkit in the study of high dimensional (HD) statistical methods. Most of the relevant statistics literature in this regard is based on sub-Gaussian or sub-exponential tail assumptions. In this paper, we first bring together various probabilistic inequalities for sums of in…
We consider the problem of estimating the discrete clustering structures under the Sub-Gaussian Mixture Model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation …
New algorithm converts data into sub-gaussian designs efficiently.
problem Efficiently converting large datasets into sub-gaussian random designs for robust performance.
method Algorithmic Gaussianization through sketching and averaging, using LESS embeddings.
result Efficient data sketches nearly indistinguishable from sub-gaussian designs.
Sharp comparison for sub-Gaussian random variables in convex order.
problem Comparing sub-Gaussian random variables in convex order.
method Proving dominance using moment generating functions and convex functions.
result Sharp comparison established between specific sub-Gaussian random variables.
Efficiently estimates common mean in high-dimensional data with noisy subset.
problem Estimating common mean in high-dimensional data with noisy subset.
method Iterative refinement strategy with rejection sampling and dimension-reduction.
result Achieves near-optimal error of f ( α , N ) + D / ( α N ) f(α,N) + \sqrt{D/(αN)} f ( α , N ) + D / ( α N ) . Thompson Sampling bounds for contextual bandits with sub-Gaussian rewards.
problem Improving the performance of Thompson Sampling in contextual bandits with sub-Gaussian rewards.
method Proved comprehensive bounds on Thompson Sampling expected cumulative regret based on mutual information and lifted information ratio for sub-Gaussian rewards.
result Explicit regret bounds for various contextual bandit scenarios.
We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence. Our core technique itself makes use of PAC-Bayesian inequalities in order to derive a robust risk estimator, which by design is easy to …
In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…
Proposes a robust method for high-dimensional linear models.
problem Inference in high-dimensional settings with heavy-tailed errors and clustered data.
method Residual randomization procedure for Lasso-based inference.
result Outperforms state-of-the-art methods in challenging settings.
New method reduces summary points for datasets while maintaining quality.
problem Thinning datasets to reduce summary points while maintaining quality.
method Low-rank analysis of sub-Gaussian thinning.
result Guarantees high-quality compression for any distribution and kernel.
In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a bound on the generalization error of empirical risk minimization based on the mutual information $I(S;…
Bayesian approach learns linear networks from high-dimensional data.
problem Learning high-dimensional linear Bayesian networks.
method Iterative estimation of topological ordering and parents using inverse partial covariance matrix with Bayesian regularization.
result The method successfully recovers network structure under certain conditions.
Estimates sub-Gaussian parameter with consistent and optimal rates.
problem Estimating sub-Gaussian parameter from random variables.
method Constrained maximization of empirical weighted cumulant generating function.
result Root-n rate estimator is consistent and optimal under certain conditions.
We propose robust sparse reduced rank regression for analyzing large and complex high-dimensional data with heavy-tailed random noise. The proposed method is based on a convex relaxation of a rank- and sparsity-constrained non-convex optimization problem, which is then solved using the alternating direction method of m…
Max-affine regression method converges linearly using GD and SGD.
problem Regression of max-affine models in signal processing and statistics.
method Gradient descent and mini-batch stochastic gradient descent analysis.
result GD and SGD converge linearly to a neighborhood of the ground truth under sub-Gaussian assumptions.
Proposes a new model for clustering with heavier tails.
problem Clustering with heavy-tailed data.
method Finite mixture of skewed sub-Gaussian stable distributions, maximum likelihood estimation, EM algorithm.
result The proposed model can robustly handle heavy-tailed data.
New GLS estimator handles high-dimensional data with autocorrelated errors.
problem High-dimensional regressions with autocorrelated errors.
method LASSO regression, autoregressive model fitting, and whitening.
result The method outperforms unadjusted LASSO in estimating errors driven by autoregressive processes.