New algorithm converts data into sub-gaussian designs efficiently.
problem Efficiently converting large datasets into sub-gaussian random designs for robust performance.
method Algorithmic Gaussianization through sketching and averaging, using LESS embeddings.
result Efficient data sketches nearly indistinguishable from sub-gaussian designs.
We tackle the problem of estimating a location parameter with differential privacy guarantees and sub-Gaussian deviations. Recent work in statistics has focused on the study of estimators that achieve sub-Gaussian type deviations even for heavy tailed data. We revisit some of these estimators through the lens of differ…
New method reduces summary points for datasets while maintaining quality.
problem Thinning datasets to reduce summary points while maintaining quality.
method Low-rank analysis of sub-Gaussian thinning.
result Guarantees high-quality compression for any distribution and kernel.
Flexible model captures varying scales in data clusters.
problem Real-world data often exhibits varying scales or intensities, violating the homogeneity assumption of classical Gaussian mixture models.
method Individual-heterogeneous sub-Gaussian mixture model with an efficient spectral method for exact recovery.
result The method provably achieves exact recovery of true cluster labels under mild separation conditions.
This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded. The bound applies to many loss functions besides the squared loss, and scales only with the sub-Gaussian or …
Paper improves SLCB regret bound for bounded noise.
problem Stochastic linear contextual bandits with bounded noise.
method Set-membership estimation (SME) and optimism in the face of uncertainty (OFU).
result Improved regret bound of O ( log T ) O(\log T) O ( log T ) . New method tightens sub-Gaussian concentration inequalities.
problem Estimating variance-type parameters of sub-Gaussian distributions.
method Using sub-Gaussian intrinsic moment norm to maximize normalized moments.
result Provides tighter sub-Gaussian concentration inequalities.
The paper provides a method to minimize regret in estimate-then-optimize decision-making.
problem Errors in estimation lead to sub-optimal decisions in data-driven decision-making.
method A novel bound on regret for smooth and unconstrained optimization problems, followed by experimental design to minimize this regret.
result A general procedure for experimental design to minimize regret resulting from estimate-then-optimize.
Global convergence for robust regression problems via IRLS with enhancements.
problem Global convergence for robust regression problems.
method Augmentations to IRLS to ensure global recovery and improved robustness.
result Global recovery guarantees for robust regression problems, outperforming state-of-the-art algorithms.
A new algorithm estimates mean adaptively to covariance, faster and more flexible than existing methods.
problem Estimating mean of a distribution with unknown covariance efficiently and privately.
method Adaptive differentially private algorithm with optimal convergence rates and near-linear sample complexity.
result Achieves optimal rates of convergence with respect to the Mahalanobis norm ∣ ∣ ⋅ ∣ ∣ Σ ||\cdot||_Σ ∣∣ ⋅ ∣ ∣ Σ . Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian α α α -stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that e…
The paper tackles resource allocation for arms with unknown and random rewards, achieving optimal regret bounds.
problem Allocating resources on arms with unknown and random rewards.
method Developed two algorithms with optimal regret bounds for b ∈ [ 0 , 1 ] b \in [0,1] b ∈ [ 0 , 1 ] , demonstrating a phase transition at b = 1 / 2 b=1/2 b = 1/2 . result Achieved optimal gap-dependent and gap-independent regret bounds for b ∈ [ 0 , 1 ] b \in [0,1] b ∈ [ 0 , 1 ] . Sharp sub-Gaussian bounds for subsolutions of Trudinger's equation on Riemannian manifolds.
problem Bounding weak subsolutions of Trudinger's equation on Riemannian manifolds.
method Proving sub-Gaussian upper bounds for weak subsolutions.
result The upper bounds are sharp for specific classes of manifolds, including \(\mathbb{R}^{n}\).
Random linear mappings are widely used in modern signal processing, compressed sensing and machine learning. These mappings may be used to embed the data into a significantly lower dimension while at the same time preserving useful information. This is done by approximately preserving the distances between data points,…
UCB algorithm adapted for large-scale, non-sub-Gaussian problems.
problem Selecting the best alternative from a large set of options with non-sub-Gaussian performance distributions.
method Adapted UCB algorithm for non-sub-Gaussian settings, focusing on sample size and meta-UCB selection.
result UCB algorithms can achieve sample optimality in large-scale, non-sub-Gaussian problems.
Proves new concentration inequalities for sub-gaussian and sub-exponential variables.
problem Understanding functions of independent random variables better.
method Sub-gaussian and sub-exponential conditions, Rademacher complexities, Lipschitz function classes.
result Extension of Rademacher complexities to unbounded sub-exponential distributions.
Study improves self-normalized bounds for vector-valued processes beyond sub-Gaussianity.
problem Limited understanding of self-normalized concentration for vector-valued processes outside sub-Gaussian frameworks.
method Developed concentration inequalities for self-normalized processes with light tails (e.g., Bennett, Bernstein bounds) for vector-valued data.
result Provided new insights and bounds for self-normalized processes with non-sub-Gaussian distributions.
Concentration inequalities form an essential toolkit in the study of high dimensional (HD) statistical methods. Most of the relevant statistics literature in this regard is based on sub-Gaussian or sub-exponential tail assumptions. In this paper, we first bring together various probabilistic inequalities for sums of in…
Quantum algorithm estimates mean with sub-Gaussian error.
problem Estimating mean of quantum-computed random variables.
method Quantum mean estimation algorithm with sub-Gaussian error rate.
result Achieves nearly-optimal quadratic speedup over classical methods.
Sharp comparison for sub-Gaussian random variables in convex order.
problem Comparing sub-Gaussian random variables in convex order.
method Proving dominance using moment generating functions and convex functions.
result Sharp comparison established between specific sub-Gaussian random variables.
LinMED is a new linear bandit algorithm with near-optimal regret bound.
problem Optimizing decision-making in linear bandit problems with sub-Gaussian distributions.
method LinMED is a randomized linear bandit algorithm with closed-form arm sampling probabilities.
result LinMED achieves a near-optimal regret bound of d n d\sqrt{n} d n up to logarithmic factors. Thompson Sampling bounds for contextual bandits with sub-Gaussian rewards.
problem Improving the performance of Thompson Sampling in contextual bandits with sub-Gaussian rewards.
method Proved comprehensive bounds on Thompson Sampling expected cumulative regret based on mutual information and lifted information ratio for sub-Gaussian rewards.
result Explicit regret bounds for various contextual bandit scenarios.
We derive PAC-Bayesian learning guarantees for heavy-tailed losses, and obtain a novel optimal Gibbs posterior which enjoys finite-sample excess risk bounds at logarithmic confidence. Our core technique itself makes use of PAC-Bayesian inequalities in order to derive a robust risk estimator, which by design is easy to …
Estimates sub-Gaussian parameter with consistent and optimal rates.
problem Estimating sub-Gaussian parameter from random variables.
method Constrained maximization of empirical weighted cumulant generating function.
result Root-n rate estimator is consistent and optimal under certain conditions.
Proposes a new model for clustering with heavier tails.
problem Clustering with heavy-tailed data.
method Finite mixture of skewed sub-Gaussian stable distributions, maximum likelihood estimation, EM algorithm.
result The proposed model can robustly handle heavy-tailed data.
New study shows mean estimation algorithms can't beat sub-Gaussian rate in general.
problem Improving mean estimation beyond worst-case scenarios.
method Constructing counterexamples and introducing neighborhood optimality.
result No reasonable estimator can achieve better than sub-Gaussian error rate for any distribution.
New winsorized mean improves robustness to up to 50% contamination.
problem Improving robustness of mean estimation in the presence of outliers.
method Outlyingness-induced winsorized mean approach.
result Achieves up to 50% contamination robustness with sub-Gaussian performance.
Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.
problem Optimizing risk-averse bandit problems with sub-Gaussian rewards.
method Anchor-free nonparametric Thompson Sampling algorithm ρ e x t − N P T S S G ρ ext{-}NPTS_{\mathrm{SG}} ρ e x t − N P T S SG . result Achieves regret matching the instance-dependent lower bound to leading order in log n \log n log n . New algorithm reduces worst-case regret for heavy-tailed bandits.
problem Stochastic Multi-Armed Bandit problem with heavy-tailed rewards.
method Modified minimax policy MOSS with saturated empirical mean.
result Worst-case regret matching lower bound for heavy-tailed distributions.
The paper proves a regret bound for a sub-Gaussian mixture on unbounded data.
problem Tackles the challenge of achieving regret bounds for sub-Gaussian mixtures on unbounded data.
method Uses path-wise (deterministic) regret bounds and a cumulative variance process to derive the bound.
result Shows that on a specific event, the regret is eventually bounded by ln(ln V_T).
We study the problem of estimating the mean of a random vector X X X given a sample of N N N independent, identically distributed points. We introduce a new estimator that achieves a purely sub-Gaussian performance under the only condition that the second moment of X X X exists. The estimator is based on a novel concept of a…
Two new algorithms improve robust PCA and Schatten packing.
problem Robustly estimating the top eigenvector of corrupted sub-Gaussian data.
method Two iterative filtering and nearly-linear time algorithms.
result First polynomial-time algorithms for non-trivial covariance estimation.
Paper analyzes SGMs for learning sub-Gaussian distributions without dimensionality constraints.
problem Learning sub-Gaussian distributions in high dimensions with SGMs.
method Introduced complexity notion and proved approximation and generalization rates.
result SGMs can approximate target sub-Gaussian distributions in total variation with dimension-independent rate.
SVGD algorithm converges at rate 1/sqrt(log log n) for sub-Gaussian distributions.
problem Approximating a probability distribution with particles.
method Stein variational gradient descent (SVGD) with finite particles and sub-Gaussian target distribution.
result SVGD achieves a convergence rate of 1/sqrt(log log n) for sub-Gaussian distributions.
Generative Adversarial Networks improve robust statistics for various distributions.
problem Estimating unknown parameters in adversarially corrupted samples.
method Designing GANs with specific loss functions for robust estimation.
result Extends robust estimation to broader families of distributions.
Andreas Maurer in the paper "A vector-contraction inequality for Rademacher complexities" extended the contraction inequality for Rademacher averages to Lipschitz functions with vector-valued domains; He did it replacing the Rademacher variables in the bounding expression by arbitrary idd symmetric and sub-gaussian var…
New algorithms detect changes in non-stationary MABs for better performance.
problem Non-stationary MAB environments where arm reward distributions change over time.
method Modular Detection Augmented Bandit (DAB) procedures with improved performance lower bounds.
result Modular DAB procedures achieve order-optimal regret bounds for various change detectors and bandit algorithms.
Improved median of means estimator with tighter bounds.
problem Improving the efficiency and reliability of median of means estimator.
method Modification of the median of means estimator with sub-Gaussian deviation bounds.
result Achieves nearly optimal constants under minimal assumptions.
Study shows how over-parameterized classifiers can still perform well on noisy data.
problem Understanding how maximum margin classifiers perform in over-parameterized settings with noisy data.
method Analyzes maximum margin classifiers on sub-Gaussian mixtures, providing risk bounds.
result Characterizes conditions for 'benign overfitting' in linear classification problems.
Improved Lasso estimator speeds up variable selection.
problem Efficient variable selection in high-dimensional data.
method Stability principle-based generalized debiased Lasso.
result Significantly reduces computational cost of resampling-based methods.
New bounds for KRR condition number reveal overfitting phenomena.
problem Characterizing overfitting in KRR with varying kernel spectral decay.
method Derived new bounds for kernel matrices, enhanced test error bounds, and identified feature independence role.
result Identified tempered and catastrophic overfitting phenomena.
One-bit clustering method for two-component sub-Gaussian mixture models
problem Clustering in sub-Gaussian mixture models
method One-bit clustering using dithered quantization
result Decaying misclassification rate with exponential signal-to-noise ratio
Proposes a robust method for high-dimensional linear models.
problem Inference in high-dimensional settings with heavy-tailed errors and clustered data.
method Residual randomization procedure for Lasso-based inference.
result Outperforms state-of-the-art methods in challenging settings.
New characterization limits sampling with inexact scores.
problem Limiting sampling with inexact scores for unbiased results.
method Characterized types of inexact score oracle access.
result Weaker error assumptions rule out tractability of unbiased sampling.
Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE -- a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential f…
This paper extends the MAB problem to consider risk-reward tradeoffs.
problem Maximizing reward while accounting for risk in multi-armed bandit problems.
method Introduced the Risk Aware Lower Confidence Bound (RALCB) algorithm to solve the mean-variance MAB problem.
result The RALCB algorithm performs better than the algorithm in Sani et al. (2012) in both independent and dependent scenarios.
Paper analyzes neural network models for sub-Gaussian distributions, proving approximation and generalization abilities.
problem Estimating unknown distributions from i.i.d. observations using neural network models.
method Score-based neural network generative models (SGMs) with specific network architectures and stopping strategies.
result SGMs can approximate scores with high accuracy and achieve nearly optimal convergence rates under mild assumptions.
SGD converges to an invariant distribution with sub-Gaussian or sub-exponential properties.
problem Optimizing smooth and strongly convex objectives using SGD.
method Analysis through Markov chains, focusing on convergence and concentration properties.
result SGD iterates and their invariant limit distribution inherit sub-Gaussian or sub-exponential concentration properties.