Paper analyzes sample complexity of polynomial neural networks.
problem Understanding the sample complexity of polynomial neural networks.
method Extends previous literature to polynomial neural networks and analyzes sample complexity.
result Obtains novel results on sample complexity of polynomial neural networks.
Polynomial-time DP algorithm for learning Gaussians with matching sample complexity.
problem Learning Gaussian distributions while maintaining privacy.
method General framework for reducing DP estimation to non-private, polynomial-time algorithm for Gaussian learning.
result Matching sample complexity to information-theoretic upper bound for Gaussian learning.
New bounds for learning polynomial surrogates with L ∞ L_\infty L ∞ guarantees.
problem Learning polynomial surrogates for bounded binary functions with L ∞ L_\infty L ∞ error guarantees. method Characterized minimax sample complexity for two classes of polynomials under subgaussian noise.
result Sample complexity rates differ from noiseless case, scaling as n d + 1 n^{d+1} n d + 1 for degree d d d polynomials and n s 2 ns^2 n s 2 for sparse polynomials. Efficiently samples arbitrary compact bodies with polynomial complexity.
problem Uniform sampling from arbitrary compact bodies efficiently.
method Warm start algorithm under isoperimetry and volume growth condition.
result Substantial generalization of known results for convex and star-shaped bodies.
We present a simple, general technique for reducing the sample complexity of matrix and tensor decomposition algorithms applied to distributions. We use the technique to give a polynomial-time algorithm for standard ICA with sample complexity nearly linear in the dimension, thereby improving substantially on previous b…
New framework allows reinforcement learning with polynomial sample complexity.
problem Generalization in reinforcement learning with function approximation.
method Introduces Bilinear Classes, a structural framework for RL.
result Polynomial sample complexity for Bilinear Classes, matching best known bounds.
New bounds for score matching in polynomial exponential families.
problem Understanding the sample complexity of score matching for polynomial exponential families.
method Non-asymptotic sample complexity analysis for score matching.
result First finite sample bounds for score matching in polynomial exponential families.
The study reveals the efficiency of sampling from tilted distributions.
problem Sampling from a tilted distribution of an unknown underlying distribution.
method Self-normalized importance sampling to characterize accuracy.
result Polynomial vs super-polynomial sample complexity for bounded vs unbounded distributions.
We study computational and sample complexity of parameter and structure learning in graphical models. Our main result shows that the class of factor graphs with bounded factor size and bounded connectivity can be learned in polynomial time and polynomial number of samples, assuming that the data is generated by a netwo…
Paper reduces sample complexity for bilinear systems identification to nearly constant.
problem Identifying discrete-time bilinear systems under bounded disturbances.
method Uses trajectory-dependent regressors and polynomial mean-square state growth analysis.
result Proves sample complexity of O ~ ( 1 / ε ) \widetilde{\mathcal O}(1/ε) O ( 1/ ε ) for estimation error ε ε ε . Robustly estimates multivariate polynomials in noisy data.
problem Estimating multivariate polynomials in noisy data with outliers.
method Generalizes robust multivariate polynomial regression to n-variate setting.
result Achieves optimal sample complexity and approximation error.
Sample- and computationally-efficient distribution estimation is a fundamental tenet in statistics and machine learning. We present SURF, an algorithm for approximating distributions by piecewise polynomials. SURF is: simple, replacing prior complex optimization techniques by straight-forward {empirical probability} ap…
New algorithm learns halfspaces with noise using Forster decomposition.
problem Learning halfspaces in noisy data.
method Forster decomposition and efficient mixture of distributions.
result First polynomial-time algorithm with strongly polynomial sample complexity.
The paper analyzes tensor recovery from symmetric rank-one measurements using information theory.
problem Recovering tensors with low symmetric rank from symmetric rank-one measurements.
method Covering numbers argument, Carbery-Wright inequality, orthogonal polynomials, Fano's inequality.
result Near-optimal sample complexity bounds for log-concave distributions.
Abstract reviews algorithms for multi-index models, focusing on polynomial-time methods and their limitations.
problem Estimating the index space in multi-index models efficiently and accurately.
method Polynomial-time algorithms in Gaussian space, nonparametric gradient estimation, and neural network fitting.
result A gap exists between computationally efficient methods and information-theoretical minimum.
New findings on depth vs. width in neural networks, showing depth can improve learnability.
problem Understanding the role of depth in neural networks, especially when width is unbounded.
method Analyzing sample complexity for learnability in norm-controlled depth-2 and depth-3 ReLU networks.
result Depth can improve learnability of functions that are otherwise unlearnable with depth-2 networks.
Efficient algorithms improve learning of large-margin halfspaces.
problem Learning large-margin halfspaces efficiently and reproducibly.
method Design of efficient, dimension-independent, polynomial-time algorithms; SGD-based approach; DP-to-Replicability reduction.
result Improved sample complexity compared to previous algorithms, with optimal sample complexity for one algorithm.
Three-layer neural networks learn hierarchical polynomial functions efficiently.
problem Learning hierarchical polynomial functions with three-layer neural networks.
method Layerwise gradient descent on square loss, focusing on feature learning.
result Achieves optimal sample complexity for learning hierarchical polynomials.
Optimizes algorithms for non-concave bandit problems.
problem Optimizing algorithms for non-concave bandit problems.
method Unified zeroth-order optimization paradigm.
result Minimax-optimal algorithms in the dimension for low-rank generalized linear bandit problems.
Tensor PCA problem analyzed with statistical query lower bounds.
problem Estimating the expected value of a rank-1 tensor from Gaussian samples.
method Sharp analysis of optimal sample complexity in the Statistical Query model.
result SQ algorithms with polynomial query complexity fail in the conjectured hard phase and have sub-optimal sample complexity.
Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.
problem PAC learning γ-margin halfspaces with Random Classification Noise.
method Established an information-computation tradeoff and provided a simple efficient algorithm with sample complexity O(1/(γ^2 ε^2)). Also, proved lower bounds for SQ algorithms and low-degree polynomial tests.
result Inherent gap between sample complexity and computational efficiency for learning halfspaces with random noise.
Efficient algorithm for sampling from arbitrary compact bodies.
problem Sampling from arbitrary compact bodies efficiently.
method Warm start algorithm with polynomial complexity.
result Substantial generalization of known results for convex and star-shaped bodies.
Improves RLHF sample efficiency by scaling reward complexity polynomially.
problem Exponential sample complexity in RLHF algorithms for skewed preferences.
method SE-POPO, an online RLHF algorithm that achieves polynomial sample complexity.
result SE-POPO outperforms existing algorithms in sample efficiency.
FNOs learn solution operators of dissipative equations efficiently via spectral methods.
problem Learning and approximation of solution operators for dissipative equations.
method Introducing spectral methods and deriving FNO approximation bounds and sample complexity guarantees.
result Polynomial sample complexity guarantees for FNOs learning solution operators of dissipative equations.
Efficiently learns polytrees with known skeleton in polynomial time and sample complexity.
problem Learning polytrees with known skeleton structure.
method Proposes an efficient algorithm for learning d d d -polytrees in polynomial time and sample complexity when the skeleton is known. result Establishes finite-sample guarantees for efficient learning of d d d -polytrees. Algorithm learns polynomials in Gaussian inputs with reduced sample complexity.
problem Learning polynomials of few relevant dimensions in high-dimensional data.
method Filtered PCA for warm start, geodesic SGD for accuracy.
result Sample complexity roughly N = O r , d ( n log 2 ( 1 / ε ) ( log n ) d ) N = O_{r,d}(n \log^2(1/ε) (\log n)^d) N = O r , d ( n log 2 ( 1/ ε ) ( log n ) d ) , runtime O r , d ( N n 2 ) O_{r,d}(N n^2) O r , d ( N n 2 ) . Lyapunov-based analysis shows polynomial sample complexity for WCMDPs and RBs.
problem Learning in WCMDPs and RBs under a generative model.
method Lyapunov-based analysis framework.
result Near-optimal policies can be learned with polynomial complexity.
Study analyzes learnability of RKHS under L∞ norm for kernel methods.
problem Understand performance of kernel methods and random feature models.
method Relate L∞ learnability to kernel spectrum decay and establish sample complexity bounds.
result Conditions for efficient L∞ learning of RKHS identified.
New RL algorithms achieve optimal policies with polynomial sample complexity for mean-field problems.
problem Statistical efficiency of Mean-Field Reinforcement Learning with general function approximation.
method Introduce MF-MBED to characterize problem complexity, propose algorithms based on maximal likelihood estimation.
result Rich mean-field RL problems have low MF-MBED, leading to polynomial sample complexity.
New method uses higher-order Langevin dynamics for efficient parallel sampling.
problem Efficient parallel sampling from high-dimensional log-concave distributions.
method Combines higher-order Langevin dynamics with blockwise Lagrange polynomial interpolation.
result Reduces the number of parallel points required for a target accuracy.
Piecewise polynomial interpolation-based gradient descent reduces oracle complexity for smooth loss functions.
problem Optimizing empirical risk minimization loss functions
method Piecewise polynomial interpolation-based gradient descent
result Oracle complexity is reduced for smooth loss functions
Paper addresses ERM in LDP, reducing sample complexity for smooth and convex losses.
problem Achieving error α in ERM with non-interactive LDP, especially for high-dimensional data.
method Developed algorithms using Bernstein polynomial and polynomial approximation techniques.
result For smooth and convex losses, sample complexity is linear in dimensionality.
Two-layer NN with channel attention learns low-degree spherical polynomials efficiently.
problem Learning low-degree spherical polynomials with over-parameterized neural networks.
method Two-layer neural network with channel attention, vanilla gradient descent, learnable channel selection.
result Minimally improved sample complexity of $n \asymp Θ(d^{\ell_0}/\eps)$ for learning low-degree polynomials.
Neural network learns low-dimensional polynomials with SGD near information-theoretic limit.
problem Learning a single-index target function with gradient descent.
method Two-layer neural network optimized by SGD on squared loss.
result Sample and runtime complexity of n ≃ T = Θ ( d ⋅ p o l y l o g d ) n \simeq T = Θ(d\!\cdot\! \mathrm{polylog} d) n ≃ T = Θ ( d ⋅ polylog d ) for polynomial single-index models, matching information theoretic limit up to polylogarithmic factors. Gradient Descent with Projection learns low-degree polynomials efficiently.
problem Learning low-degree spherical polynomials with neural networks.
method Over-parameterized two-layer neural network with Gradient Descent with Projection.
result Achieves nearly minimax optimal sample complexity and risk bound.
PER-ETD improves ETD by reducing variance to polynomial complexity.
problem Large variance in ETD leading to exponential sample complexity.
method Periodically restart and update the follow-on trace for a finite period.
result PER-ETD converges to the same fixed point as ETD but with improved sample complexity.
We give a simple, fast algorithm for hyperparameter optimization inspired by techniques from the analysis of Boolean functions. We focus on the high-dimensional regime where the canonical example is training a neural network with a large number of hyperparameters. The algorithm --- an iterative application of compresse…
Study sample complexity of robust binary hypothesis testing under different contamination models.
problem Analyzing the sample complexity of robust binary hypothesis testing under various contamination models.
method Examined three standard contamination models: ε-additive (Huber), ε-subtractive, and ε-total variation (TV). Provided explicit formulas for least favourable distributions and compared sample complexities across models.
result Sample complexities are highly unstable in the contamination parameter ε and comparable up to constant-factor rescaling of ε across models.
New algorithm reduces sample complexity for sparse linear regression.
problem Sparse linear regression with correlated covariates and approximate dependencies.
method Polynomial-time algorithm that adapts the Lasso to tolerate approximate dependencies.
result Achieves near-optimal sample complexity for constant sparsity and ill-conditioned covariates.
Three-layer networks learn complex hierarchical polynomials of multiple nonlinear features.
problem Understanding how neural networks learn hierarchical features of multiple nonlinear inputs.
method Examine a broad class of functions using three-layer neural networks, showing complete recovery and efficient learning.
result Three-layer neural networks trained via gradient descent can learn hierarchical polynomials of multiple nonlinear features efficiently.
In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared loss of such CNNs is l o c a l l y s t r o n g l y c o n v e x \mathit{~locally~strongly~convex} locally strongly convex in a basin of attraction…
Federated learning supports exact support recovery with minimal communication.
problem Learning the exact support of sparse linear regression in federated learning.
method One-shot communication algorithm for exact support recovery without optimization.
result Polynomial sample complexity and logarithmic number of clients required.
New framework uses unsupervised learning for efficient exploration in RL.
problem Efficient exploration in reinforcement learning with rich observations.
method Combines unsupervised learning and no-regret RL algorithms.
result Proves sample complexity for finding near-optimal policies.
New sampling method for heavy-tailed distributions using Langevin Algorithm.
problem Sampling from heavy-tailed distributions efficiently.
method Transformed Unadjusted Langevin Algorithm on specific transformations.
result Polynomial-order oracle complexities for certain heavy-tailed densities.
Improved sample complexity for training diffusion models.
problem How many samples are needed to train an accurate diffusion model?
method Analyzing the sample complexity of training diffusion models using neural networks.
result Exponential improvement in the dependence on Wasserstein error and depth, along with improved dependencies on other parameters.
Polynomial convergence proved for SGM, improving over previous methods.
problem Learning probability distributions from data and generating samples efficiently.
method Proved polynomial convergence for SGM using accurate score estimates.
result First polynomial convergence guarantees for SGM, independent of dimensionality.
Paper settles sample complexity for learning from multiple distributions.
problem Learning from multiple data distributions with a hypothesis class of bounded VC dimension.
method Introduced an algorithm with sample complexity of O((d+k)ε^-2)·(k/ε)^o(1).
result Algorithm matches lower bound up to sub-polynomial factor.
Efficiently estimates prediction error in regression with Gaussian covariates under privacy constraints.
problem Private regression with Gaussian covariates under differential privacy constraints.
method Sum-of-Squares framework combined with robust estimators.
result Sample-optimal private regression algorithm with optimal error rates.