New method certifies anti-concentration for various non-Gaussian distributions.
problem Efficiently certifying anti-concentration for non-Gaussian distributions.
method Sum-of-Squares relaxation of integer program for anti-concentration.
result Quasi-polynomial time certificates for non-Gaussian distributions.
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
problem Estimating intrinsic dimension from data.
method Finite-sample concentration and anti-concentration bounds for Gaussian kernel sums.
result Explicit dependence on sample size, bandwidth, and geometric parameters.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O ( poly log T ) O(\operatorname{poly} \log T) O ( poly log T ) cumulative expected regret. Robustly learns Ising models with corrupted data.
problem Learning Ising models corrupted by a constant fraction of adversarial samples.
method Develops a computationally efficient algorithm for robust learning.
result First near-optimal error guarantees for robust learning of Ising models.
Improved subspace recovery algorithm with dimension-independent error and polynomial time.
problem Efficiently recover a covariance matrix from a mix of inliers and adversarial outliers.
method List-decodable subspace recovery algorithm with faster fixed-polynomial time and less restrictive distributional assumptions.
result Achieved dimension-independent error guarantee of O(1/α) with poly(1/α d^O(1)) time complexity.
Low-degree method fails to predict robust subspace recovery problem.
problem Predicting computational tractability of robust subspace recovery problem.
method Low-degree polynomial framework, anti-concentration properties.
result Low-degree method fails to predict computational tractability of robust subspace recovery problem even up to high degree.
We give the first polynomial-time algorithm for robust regression in the list-decodable setting where an adversary can corrupt a greater than 1 / 2 1/2 1/2 fraction of examples. For any α < 1 α< 1 α < 1 , our algorithm takes as input a sample { ( x i , y i ) } i ≤ n \{(x_i,y_i)\}_{i \leq n} {( x i , y i ) } i ≤ n of n n n linear equations where α n αn α n of the equations satisfy $y_i = \l…
Robustly clusters mixtures of Gaussians even with outliers.
problem Clustering mixtures of statistically separated Gaussians robustly to outliers.
method Uses certifiable hypercontractivity, bounded variance, and anti-concentration of linear projections.
result First efficient algorithm for robust clustering of statistically separated Gaussians mixtures.
We give concentration bounds for martingales that are uniform over finite times and extend classical Hoeffding and Bernstein inequalities. We also demonstrate our concentration bounds to be optimal with a matching anti-concentration inequality, proved using the same method. Together these constitute a finite-time versi…
Sharp concentration bounds for i.i.d. variables.
problem Controlling the tail probabilities of independent variables.
method Extension of Sanov's theorem using large deviations and information theory.
result Matching concentration and anti-concentration bounds for i.i.d. samples of any size.
OPSRL algorithm reduces regret with few samples in reinforcement learning.
problem High regret in reinforcement learning with limited data.
method Optimistic Posterior Sampling (OPSRL) with logarithmic sample complexity.
result Guaranteed high-probability regret bound of O ~ ( H 3 S A T ) \widetilde{\mathcal{O}}(\sqrt{H^3SAT}) O ( H 3 S A T ) . Adversarial training improves robustness of halfspaces in noisy data.
problem Learning robust halfspaces in the presence of label noise.
method Adversarial training with binary cross-entropy or nonconvex sigmoidal loss.
result Adversarial training yields robust halfspaces with improved classification error.
Bayes-UCBVI tackles reinforcement learning with a new upper confidence bound method.
problem Optimizing exploration in reinforcement learning without bonuses.
method Bayes-UCBVI uses a quantile of a Q-value function posterior as an upper confidence bound.
result Proves a regret bound of order O ~ ( H 3 S A T ) \widetilde{O}(\sqrt{H^3SAT}) O ( H 3 S A T ) for tabular reinforcement learning. Algorithm learns Gaussian mixtures robust to outliers.
problem Efficiently learn high-dimensional Gaussian mixtures with outliers.
method Sum-of-Squares based proofs to algorithms approach.
result Polynomial time algorithm for k k k -mixture with pairwise separated components. Paper addresses concentration of distances for fractional quasi p-norms, identifying conditions for concentration and anti-concentration.
problem Understanding concentration of distances for fractional quasi p-norms in high dimensions.
method Analyzes conditions for concentration and anti-concentration of distances for fractional quasi p-norms.
result Identifies conditions for concentration and anti-concentration of fractional quasi p-norms, ruling out some approaches and specifying conditions for control.
Fictitious play is a simple and widely studied adaptive heuristic for playing repeated games. It is well known that fictitious play fails to be Hannan consistent. Several variants of fictitious play including regret matching, generalized regret matching and smooth fictitious play, are known to be Hannan consistent. In …
Polynomial-time algorithm for estimating covariance in corrupted Gaussian data.
problem Estimating covariance in data with up to 1-α fraction of adversarial corruptions.
method Uses low-degree sum-of-squares certificates for anti-concentration and hypercontractivity.
result Outputs a list of candidate parameters with high probability containing a nearly correct covariance.
Several fundamental problems that arise in optimization and computer science can be cast as follows: Given vectors v 1 , … , v m ∈ R d v_1,\ldots,v_m \in \mathbb{R}^d v 1 , … , v m ∈ R d and a constraint family B ⊆ 2 [ m ] {\cal B}\subseteq 2^{[m]} B ⊆ 2 [ m ] , find a set S ∈ B S \in \cal{B} S ∈ B that maximizes the squared volume of the simplex spanned by the vectors in S S S . A motivatin…
We propose a new online algorithm for cumulative regret minimization in a stochastic linear bandit. The algorithm pulls the arm with the highest estimated reward in a linear model trained on its perturbed history. Therefore, we call it perturbed-history exploration in a linear bandit (LinPHE). The perturbed history is …
New algorithm learns halfspaces with noise using Forster decomposition.
problem Learning halfspaces in noisy data.
method Forster decomposition and efficient mixture of distributions.
result First polynomial-time algorithm with strongly polynomial sample complexity.
AdaBoost improves binary classification in robust one-bit compressed sensing with adversarial errors.
problem Binary classification in robust one-bit compressed sensing with adversarial errors.
method AdaBoost and max- ℓ 1 \ell_1 ℓ 1 -margin-classifier approach, with convergence rates improved under certain feature conditions. result Improved convergence rates and explanation for harmless interpolating adversarial noise.
Improved Lasso estimator speeds up variable selection.
problem Efficient variable selection in high-dimensional data.
method Stability principle-based generalized debiased Lasso.
result Significantly reduces computational cost of resampling-based methods.
The paper explores how linear neural networks can overfit without bias when data is well-behaved.
problem Understanding why linear neural networks can generalize well despite fitting noisy data.
method Analyzing two-layer linear neural networks trained with gradient flow, deriving bounds on excess risk.
result The excess risk depends on initialization quality and data covariance matrix properties.
Study on ReLU regression with Massart noise, achieving exact parameter recovery.
problem Efficiently fitting ReLUs to data in the presence of Massart noise.
method Developed an efficient algorithm for exact parameter recovery under mild assumptions.
result Achieved exact parameter recovery in ReLU regression with Massart noise.
The paper analyzes tensor recovery from symmetric rank-one measurements using information theory.
problem Recovering tensors with low symmetric rank from symmetric rank-one measurements.
method Covering numbers argument, Carbery-Wright inequality, orthogonal polynomials, Fano's inequality.
result Near-optimal sample complexity bounds for log-concave distributions.
Max-affine regression method converges linearly using GD and SGD.
problem Regression of max-affine models in signal processing and statistics.
method Gradient descent and mini-batch stochastic gradient descent analysis.
result GD and SGD converge linearly to a neighborhood of the ground truth under sub-Gaussian assumptions.
We solve ReLU regression with efficient approximations for various distributions.
problem Finding the best fitting ReLU function with square loss from unknown distributions.
method Introduced efficient constant-factor approximation algorithm and polynomial-time approximation scheme.
result First constant-factor approximation algorithm for ReLU regression with weak concentration conditions.
Efficient algorithm for near-optimal online learning with generalized linear functions.
problem Exponential gap between statistically optimal regret and efficient regret for some function classes.
method Computational efficient algorithm for realizable K-wise linear classification and over-parameterized polynomial featurization.
result First algorithm with log(T/σ) regret for realizable K-wise linear classification.
Paper proposes a new RLHF framework for human preference learning.
problem Handling dependent online human preference outcomes with dynamic contexts.
method Two-stage algorithm with ε ε ε -greedy followed by exploitation; anti-concentration inequalities and matrix martingale concentration techniques. result Our method achieves optimal regret bound and asymptotic normality of estimators.
Algorithm learns halfspaces in noisy data efficiently.
problem Learning halfspaces with Tsybakov noise.
method Novel semi-definite programming and online convex optimization.
result First non-trivial PAC learning algorithm for Tsybakov noise.
Self-training improves weak classifiers in mixture models.
problem Improving weak classifiers in mixture models.
method Iterative self-training algorithm using pseudolabels and unlabeled data.
result Self-training converts weak learners to strong learners in mixture models.
Two preprocessing techniques reduce neural network training cost.
problem Training over-parameterized neural networks efficiently.
method Two novel preprocessing techniques to reduce training cost.
result Training cost reduced to sublinear per iteration.
This work proposes efficient classical training protocols for IQP circuits to train quantum generative models.
problem Training quantum generative models on industrially relevant probability distributions is challenging due to high computational cost.
method Developed protocols for classical training of IQP circuits, which are hard to sample but have efficient gradient computation.
result Classically trained IQP circuits can efficiently sample from target probability distributions, demonstrating practical quantum advantage.
In this article, we investigate large sample properties of model selection procedures in a general Bayesian framework when a closed form expression of the marginal likelihood function is not available or a local asymptotic quadratic approximation of the log-likelihood function does not exist. Under appropriate identifi…
Paper proposes Sp-GD for sparse max-affine regression with theoretical guarantees.
problem Sparse max-affine regression model selection and estimation.
method Sparse Gradient Descent (Sp-GD) initialization using sparse PCA and covering search.
result Sp-GD provides ε-accurate estimates with optimal number of observations.
A new method reduces the bias in estimating inverse covariance matrices from sketches.
problem Reducing the bias in estimating inverse covariance matrices from sketches.
method Developed a framework for analyzing inversion bias and proposed a new sketching technique called LEverage Score Sparsified (LESS) embeddings.
result The new sketching technique reduces the inversion bias to O ( 1 / d ) O(1/\sqrt d) O ( 1/ d ) for m = O ( d ) m=O(d) m = O ( d ) , significantly smaller than the Θ ( 1 ) Θ(1) Θ ( 1 ) approximation error.