Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
problem Estimating intrinsic dimension from data.
method Finite-sample concentration and anti-concentration bounds for Gaussian kernel sums.
result Explicit dependence on sample size, bandwidth, and geometric parameters.
New method certifies anti-concentration for various non-Gaussian distributions.
problem Efficiently certifying anti-concentration for non-Gaussian distributions.
method Sum-of-Squares relaxation of integer program for anti-concentration.
result Quasi-polynomial time certificates for non-Gaussian distributions.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O ( poly log T ) O(\operatorname{poly} \log T) O ( poly log T ) cumulative expected regret. We give concentration bounds for martingales that are uniform over finite times and extend classical Hoeffding and Bernstein inequalities. We also demonstrate our concentration bounds to be optimal with a matching anti-concentration inequality, proved using the same method. Together these constitute a finite-time versi…
Sharp concentration bounds for i.i.d. variables.
problem Controlling the tail probabilities of independent variables.
method Extension of Sanov's theorem using large deviations and information theory.
result Matching concentration and anti-concentration bounds for i.i.d. samples of any size.
Low-degree method fails to predict robust subspace recovery problem.
problem Predicting computational tractability of robust subspace recovery problem.
method Low-degree polynomial framework, anti-concentration properties.
result Low-degree method fails to predict computational tractability of robust subspace recovery problem even up to high degree.
OPSRL algorithm reduces regret with few samples in reinforcement learning.
problem High regret in reinforcement learning with limited data.
method Optimistic Posterior Sampling (OPSRL) with logarithmic sample complexity.
result Guaranteed high-probability regret bound of O ~ ( H 3 S A T ) \widetilde{\mathcal{O}}(\sqrt{H^3SAT}) O ( H 3 S A T ) . Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.
problem Optimizing exploration in reinforcement learning without bonuses.
method Bayes-UCBVI uses a quantile of a Q-value function posterior as an upper confidence bound.
result Proves a regret bound of order O ~ ( H 3 S A T ) \widetilde{O}(\sqrt{H^3SAT}) O ( H 3 S A T ) for tabular reinforcement learning. Robustly learns Ising models with corrupted data.
problem Learning Ising models corrupted by a constant fraction of adversarial samples.
method Develops a computationally efficient algorithm for robust learning.
result First near-optimal error guarantees for robust learning of Ising models.
Robustly clusters mixtures of Gaussians even with outliers.
problem Clustering mixtures of statistically separated Gaussians robustly to outliers.
method Uses certifiable hypercontractivity, bounded variance, and anti-concentration of linear projections.
result First efficient algorithm for robust clustering of statistically separated Gaussians mixtures.
Improved subspace recovery algorithm with dimension-independent error and polynomial time.
problem Efficiently recover a covariance matrix from a mix of inliers and adversarial outliers.
method List-decodable subspace recovery algorithm with faster fixed-polynomial time and less restrictive distributional assumptions.
result Achieved dimension-independent error guarantee of O(1/α) with poly(1/α d^O(1)) time complexity.
We give the first polynomial-time algorithm for robust regression in the list-decodable setting where an adversary can corrupt a greater than 1 / 2 1/2 1/2 fraction of examples. For any α < 1 α< 1 α < 1 , our algorithm takes as input a sample { ( x i , y i ) } i ≤ n \{(x_i,y_i)\}_{i \leq n} {( x i , y i ) } i ≤ n of n n n linear equations where α n αn α n of the equations satisfy $y_i = \l…
We propose a new online algorithm for cumulative regret minimization in a stochastic linear bandit. The algorithm pulls the arm with the highest estimated reward in a linear model trained on its perturbed history. Therefore, we call it perturbed-history exploration in a linear bandit (LinPHE). The perturbed history is …
AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.
problem Binary classification in robust one-bit compressed sensing with adversarial errors.
method AdaBoost and max- ℓ 1 \ell_1 ℓ 1 -margin-classifier approach, with convergence rates improved under certain feature conditions. result Improved convergence rates and explanation for harmless interpolating adversarial noise.
Algorithm learns Gaussian mixtures robust to outliers.
problem Efficiently learn high-dimensional Gaussian mixtures with outliers.
method Sum-of-Squares based proofs to algorithms approach.
result Polynomial time algorithm for k k k -mixture with pairwise separated components. Paper addresses concentration of distances for fractional quasi p-norms, identifying conditions for concentration and anti-concentration.
problem Understanding concentration of distances for fractional quasi p-norms in high dimensions.
method Analyzes conditions for concentration and anti-concentration of distances for fractional quasi p-norms.
result Identifies conditions for concentration and anti-concentration of fractional quasi p-norms, ruling out some approaches and specifying conditions for control.
The paper explores how linear neural networks can overfit without bias when data is well-behaved.
problem Understanding why linear neural networks can generalize well despite fitting noisy data.
method Analyzing two-layer linear neural networks trained with gradient flow, deriving bounds on excess risk.
result The excess risk depends on initialization quality and data covariance matrix properties.
Adversarial training improves robustness of halfspaces in noisy data.
problem Learning robust halfspaces in the presence of label noise.
method Adversarial training with binary cross-entropy or nonconvex sigmoidal loss.
result Adversarial training yields robust halfspaces with improved classification error.
The paper analyzes tensor recovery from symmetric rank-one measurements using information theory.
problem Recovering tensors with low symmetric rank from symmetric rank-one measurements.
method Covering numbers argument, Carbery-Wright inequality, orthogonal polynomials, Fano's inequality.
result Near-optimal sample complexity bounds for log-concave distributions.
Efficient algorithm for near-optimal online learning with generalized linear functions.
problem Exponential gap between statistically optimal regret and efficient regret for some function classes.
method Computational efficient algorithm for realizable K-wise linear classification and over-parameterized polynomial featurization.
result First algorithm with log(T/σ) regret for realizable K-wise linear classification.
Max-affine regression method converges linearly using GD and SGD.
problem Regression of max-affine models in signal processing and statistics.
method Gradient descent and mini-batch stochastic gradient descent analysis.
result GD and SGD converge linearly to a neighborhood of the ground truth under sub-Gaussian assumptions.
Fictitious play is a simple and widely studied adaptive heuristic for playing repeated games. It is well known that fictitious play fails to be Hannan consistent. Several variants of fictitious play including regret matching, generalized regret matching and smooth fictitious play, are known to be Hannan consistent. In …
Polynomial-time algorithm for estimating covariance in corrupted Gaussian data.
problem Estimating covariance in data with up to 1-α fraction of adversarial corruptions.
method Uses low-degree sum-of-squares certificates for anti-concentration and hypercontractivity.
result Outputs a list of candidate parameters with high probability containing a nearly correct covariance.
Several fundamental problems that arise in optimization and computer science can be cast as follows: Given vectors v 1 , … , v m ∈ R d v_1,\ldots,v_m \in \mathbb{R}^d v 1 , … , v m ∈ R d and a constraint family B ⊆ 2 [ m ] {\cal B}\subseteq 2^{[m]} B ⊆ 2 [ m ] , find a set S ∈ B S \in \cal{B} S ∈ B that maximizes the squared volume of the simplex spanned by the vectors in S S S . A motivatin…
New algorithm learns halfspaces with noise using Forster decomposition.
problem Learning halfspaces in noisy data.
method Forster decomposition and efficient mixture of distributions.
result First polynomial-time algorithm with strongly polynomial sample complexity.
Paper proposes a new RLHF framework for human preference learning.
problem Handling dependent online human preference outcomes with dynamic contexts.
method Two-stage algorithm with ε ε ε -greedy followed by exploitation; anti-concentration inequalities and matrix martingale concentration techniques. result Our method achieves optimal regret bound and asymptotic normality of estimators.
Algorithm learns halfspaces in noisy data efficiently.
problem Learning halfspaces with Tsybakov noise.
method Novel semi-definite programming and online convex optimization.
result First non-trivial PAC learning algorithm for Tsybakov noise.
Improved Lasso estimator speeds up variable selection.
problem Efficient variable selection in high-dimensional data.
method Stability principle-based generalized debiased Lasso.
result Significantly reduces computational cost of resampling-based methods.
Study on ReLU regression with Massart noise, achieving exact parameter recovery.
problem Efficiently fitting ReLUs to data in the presence of Massart noise.
method Developed an efficient algorithm for exact parameter recovery under mild assumptions.
result Achieved exact parameter recovery in ReLU regression with Massart noise.
In this article, we investigate large sample properties of model selection procedures in a general Bayesian framework when a closed form expression of the marginal likelihood function is not available or a local asymptotic quadratic approximation of the log-likelihood function does not exist. Under appropriate identifi…
We solve ReLU regression with efficient approximations for various distributions.
problem Finding the best fitting ReLU function with square loss from unknown distributions.
method Introduced efficient constant-factor approximation algorithm and polynomial-time approximation scheme.
result First constant-factor approximation algorithm for ReLU regression with weak concentration conditions.
Paper proposes Sp-GD for sparse max-affine regression with theoretical guarantees.
problem Sparse max-affine regression model selection and estimation.
method Sparse Gradient Descent (Sp-GD) initialization using sparse PCA and covering search.
result Sp-GD provides ε-accurate estimates with optimal number of observations.
Self-training improves weak classifiers in mixture models.
problem Improving weak classifiers in mixture models.
method Iterative self-training algorithm using pseudolabels and unlabeled data.
result Self-training converts weak learners to strong learners in mixture models.
Two preprocessing techniques reduce neural network training cost.
problem Training over-parameterized neural networks efficiently.
method Two novel preprocessing techniques to reduce training cost.
result Training cost reduced to sublinear per iteration.
A new method reduces the bias in estimating inverse covariance matrices from sketches.
problem Reducing the bias in estimating inverse covariance matrices from sketches.
method Developed a framework for analyzing inversion bias and proposed a new sketching technique called LEverage Score Sparsified (LESS) embeddings.
result The new sketching technique reduces the inversion bias to O ( 1 / d ) O(1/\sqrt d) O ( 1/ d ) for m = O ( d ) m=O(d) m = O ( d ) , significantly smaller than the Θ ( 1 ) Θ(1) Θ ( 1 ) approximation error. This work proposes efficient classical training protocols for IQP circuits to train quantum generative models.
problem Training quantum generative models on industrially relevant probability distributions is challenging due to high computational cost.
method Developed protocols for classical training of IQP circuits, which are hard to sample but have efficient gradient computation.
result Classically trained IQP circuits can efficiently sample from target probability distributions, demonstrating practical quantum advantage.
The study optimizes bounds for comparing training and population loss.
problem Optimizing bounds for comparing training and population loss.
method Derives generic information-theoretic and PAC-Bayesian generalization bounds using convex comparator functions.
result The tightest possible bound is obtained with the comparator being the convex conjugate of the CGF of the bounding distribution.
Introduces bounded scale measure and generalizes property A.
problem Defining property A for large scale spaces with bounded geometry.
method Introduces bounded scale measure, shows its coarse invariance, and generalizes property A.
result Definition of property A for large scale spaces with bounded scale measure is a coarse invariant.
Paper improves PAC-Bayes bounds for various loss types.
problem Improving PAC-Bayes bounds for different types of losses.
method Introducing new high-probability PAC-Bayes bounds for bounded and general tail behaviors losses, and extending to anytime-valid bounds.
result New fast-rate and mixed-rate bounds for losses with bounded ranges, and parameter-free bounds for losses with general tail behaviors.
Improved bounds for Monte Carlo Rademacher Averages using self-bounding functions.
problem Proving sharper concentration bounds for MCERA.
method Deriving new bounds through self-bounding functions and concentration of measure.
result Novel bounds depend on data-dependent quantities, improving over standard methods.
Study bounds on self-shrinkers with bounded HA for applications.
problem Understanding bounds on self-shrinkers with bounded HA.
method Integral and pointwise bounds on the second fundamental form of self-shrinkers.
result Gap and compactness results for self-shrinkers.
Investigates tight PAC-Bayes bounds for small datasets.
problem Tightening PAC-Bayes bounds for small data.
method Generic PAC-Bayes theorem, meta-learning, synthetic tasks.
result PAC-Bayes bounds are competitive with Chernoff bounds but not as tight.
Extends Fatou theorem to bounded harmonic maps.
problem Classical Fatou theorem for bounded harmonic functions.
method Extending theorem to bounded harmonic maps.
result Identifies bounded harmonic maps on unit disk with bounded measurable functions on boundary.
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
problem Generalization bounds with strict assumptions like uniformly bounded loss.
method Relax uniform bounds assumptions to on-average bounded loss and gradient norm.
result Proposes a new generalization bound with a surrogate of model complexity.
Jiang et al. (2020) found no uniformly tight generalization bounds for neural networks in the overparameterized setting.
problem Finding uniformly tight generalization bounds for neural networks in the overparameterized setting.
method Examined more than a dozen generalization bounds, proving that no bounds can be uniformly tight in the overparameterized setting.
result No generalization bounds can be uniformly tight in the overparameterized setting.
Willmore-type inequalities for bounded domains in manifolds with curvature bounds.
problem Establishing inequalities for bounded domains in manifolds with curvature bounds.
method Using asymptotic or integral Ricci curvature bounds to establish inequalities.
result Recovering a recent inequality of Jin-Yin.
Lower bounds on curvature integral for manifolds with curvature constraints.
problem Bounding curvature integrals under curvature constraints.
method Proving a lower bound for the curvature integral using dimension, upper curvature bounds, and injectivity radius.
result Uniformly bounded below integral of scalar curvature.
Paper improves SLCB regret bound for bounded noise.
problem Stochastic linear contextual bandits with bounded noise.
method Set-membership estimation (SME) and optimism in the face of uncertainty (OFU).
result Improved regret bound of O ( log T ) O(\log T) O ( log T ) .