arXiv research

Quantum algorithm estimates mean with sub-Gaussian error.

We tackle the problem of estimating a location parameter with differential privacy guarantees and sub-Gaussian deviations. Recent work in statistics has focused on the study of estimators that achieve sub-Gaussian type deviations even for heavy tailed data. We revisit some of these estimators through the lens of differ…

New study shows mean estimation algorithms can't beat sub-Gaussian rate in general.

We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find upper bounds on the rates of convergence of the LSE when the errors have uniformly …

New characterization limits sampling with inexact scores.

Paper presents robust clustering methods for general mixture models.

Score-based diffusion models achieve optimal error bounds under non-parametric assumptions.

New bounds for kernel regression under non-Gaussian noise.

Study shows how over-parameterized classifiers can still perform well on noisy data.

New winsorized mean improves robustness to up to 50% contamination.

New algorithm guarantees performance on noisy data.

We propose an estimator for the mean of a random vector in $\mathbb{R}^d$ that can be computed in time $O(n^4+n^2d)$ for $n$ i.i.d.~samples and that has error bounds matching the sub-Gaussian case. The only assumptions we make about the data distribution are that it has finite mean and covariance; in particular, we mak…

DP-PCA improves privacy in PCA computations with optimal statistical error.

New method improves privacy in linear regression with optimal error bounds.

New method tightens sub-Gaussian concentration inequalities.

The paper strengthens the classical result of MLE convergence to a Gaussian distribution.

Paper analyzes neural network models for sub-Gaussian distributions, proving approximation and generalization abilities.

Improved generalization bounds for SGD in non-convex learning.

A new algorithm estimates mean adaptively to covariance, faster and more flexible than existing methods.

Efficiently estimates covariance for sub-Weibull vectors with sub-Gaussian rate.

Paper tackles robust matrix completion with heavy-tailed noise.

Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian $α$-stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that e…

Proposes a new model for mixed membership in Gaussian mixture.

Sharp sub-Gaussian bounds for subsolutions of Trudinger's equation on Riemannian manifolds.

Paper uses Stochastic Mirror Descent for large-scale sparse recovery problems.

Random linear mappings are widely used in modern signal processing, compressed sensing and machine learning. These mappings may be used to embed the data into a significantly lower dimension while at the same time preserving useful information. This is done by approximately preserving the distances between data points,…

UCB algorithm adapted for large-scale, non-sub-Gaussian problems.

Proves new concentration inequalities for sub-gaussian and sub-exponential variables.

The paper tackles best arm identification in contaminated bandits with optimal error guarantees and sample complexity.

Study improves self-normalized bounds for vector-valued processes beyond sub-Gaussianity.

We consider the problem of clustering datasets in the presence of arbitrary outliers. Traditional clustering algorithms such as k-means and spectral clustering are known to perform poorly for datasets contaminated with even a small number of outliers. In this paper, we develop a provably robust spectral clustering algo…

Concentration inequalities form an essential toolkit in the study of high dimensional (HD) statistical methods. Most of the relevant statistics literature in this regard is based on sub-Gaussian or sub-exponential tail assumptions. In this paper, we first bring together various probabilistic inequalities for sums of in…

We consider the problem of estimating the discrete clustering structures under the Sub-Gaussian Mixture Model. Our main results establish a hidden integrality property of a semidefinite programming (SDP) relaxation for this problem: while the optimal solution to the SDP is not integer-valued in general, its estimation …

New algorithm converts data into sub-gaussian designs efficiently.

Sharp comparison for sub-Gaussian random variables in convex order.

Efficiently estimates common mean in high-dimensional data with noisy subset.

Thompson Sampling bounds for contextual bandits with sub-Gaussian rewards.

We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence. Our core technique itself makes use of PAC-Bayesian inequalities in order to derive a robust risk estimator, which by design is easy to …

In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…

Proposes a robust method for high-dimensional linear models.

New method reduces summary points for datasets while maintaining quality.

In statistical learning theory, generalization error is used to quantify the degree to which a supervised machine learning algorithm may overfit to training data. Recent work [Xu and Raginsky (2017)] has established a bound on the generalization error of empirical risk minimization based on the mutual information $I(S;…

Estimates sub-Gaussian parameter with consistent and optimal rates.

Bayesian approach learns linear networks from high-dimensional data.

We propose robust sparse reduced rank regression for analyzing large and complex high-dimensional data with heavy-tailed random noise. The proposed method is based on a convex relaxation of a rank- and sparsity-constrained non-convex optimization problem, which is then solved using the alternating direction method of m…

Max-affine regression method converges linearly using GD and SGD.

Proposes a new model for clustering with heavier tails.

New GLS estimator handles high-dimensional data with autocorrelated errors.