New framework connects online learning to statistical learning for better generalization bounds.
problem Deriving generalization bounds for statistical learning algorithms.
method Constructing an online learning game and showing a connection to statistical learning.
result Established a connection between online and statistical learning, leading to new generalization bounds.
Improves statistical learning bounds with self-concordant losses.
problem Statistical prediction with nuisance components.
method Orthogonal statistical learning with self-concordant loss.
result Non-asymptotic bounds on excess risk improved by a dimension factor.
Establishes statistical and computational bounds for influence diagnostics.
problem Identifying influential datapoints or subsets in machine learning models.
method Finite-sample statistical bounds and computational complexity for influence functions and approximate maximum influence perturbations.
result Established statistical and computational guarantees for influence diagnostics.
Novel mutual information bound improves statistical inference rates.
problem Improving statistical inference rates in Bayesian nonparametrics.
method Introduces a novel mutual information bound.
result Improved contraction rates for fractional posteriors.
The paper introduces gapped scale-sensitive dimensions to improve learning rate bounds.
problem Improving lower bounds on rates of convergence in statistical and online learning.
method Introducing and analyzing gapped scale-sensitive dimensions for function classes.
result Gapped dimensions lead to stronger lower bounds on offset Rademacher averages.
New bounds show complex neural networks need many queries to learn.
problem Learning non-polynomial activation functions with Gaussian marginals.
method Gradient boosting procedure to amplify lower bounds on SQ dimension of neural networks.
result Statistical-query lower bounds for ReLU regression with 2ncε queries. Study spectral learning for odeco tensors, addressing initialization bottlenecks.
problem Recovering orthogonally decomposable tensors under noise.
method Investigates perturbation bounds, non-convex optimization, and initialization strategies.
result Initialization is the main bottleneck for efficient algorithms.
New complexity measure for interactive learning reduces regret to near-optimal levels.
problem Challenges in sample-efficient, adaptive learning algorithms for interactive decision making.
method Introduces the Decision-Estimation Coefficient and the Estimation-to-Decisions (E2D) principle.
result Unified algorithm design principle E2D achieves optimal sample-efficient learning.
Within the framework of statistical learning theory it is possible to bound the minimum number of samples required by a learner to reach a target accuracy. We show that if the bound on the accuracy is taken into account, quantum machine learning algorithms for supervised learning---for which statistical guarantees are …
Statistical model checking for PCTL on MDPs using reinforcement learning.
problem Model checking PCTL specifications on MDPs with statistical methods.
method Reinforcement learning for policy search, statistical model checking with UCB-based Q-learning.
result Provably guaranteed statistical model checking method for PCTL specifications on MDPs.
Prove non-asymptotic bounds for minimal risk in statistical learning
problem Estimating minimal risk in statistical learning
method Using concentration inequalities
result Non-asymptotic bounds for minimal risk
Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach, mainly studied in the statistical physics literature, is the study of generalization in simple synthetic-data models. Here we discuss the c…
Learning to control linear systems is statistically hard, especially for underactuated systems.
problem Statistical difficulty of learning to control linear systems, especially underactuated ones.
method Utilized minimax lower bounds and structural assumptions to prove learning complexity can be exponential.
result Learning complexity can be at most exponential with the controllability index of the system.
Study excess risk in statistical inference with transformations.
problem Excess risk in estimating random variables from feature vectors and transformations.
method Characterize lossless transformations, develop test statistics, and information-theoretic bounds.
result Strongly consistent partitioning test statistic for lossless transformations.
New bounds on learning shared representations improve model performance and efficiency.
problem Improving model performance and efficiency through shared representations across clients.
method Established new upper and lower bounds on statistical error, designed a spectral estimator for non-convex least-squares solutions.
result Optimal statistical rate achieved when shared representation is well covered across clients.
Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RK…
Study shows multi-distribution learning has slower rates than single-task learning.
problem Understanding the statistical complexity of learning from heterogeneous sources.
method Structured hypothesis-testing framework to capture the statistical cost of certifying near-optimality under bounded noise.
result Learning across multiple distributions incurs slow rates scaling with k/ε2, even under constant noise levels. Score attack method provides a lower bound on privacy-constrained minimax risk.
problem Characterizing the optimality of privacy-constrained statistical models.
method Score attack based on tracing attack concept.
result Optimally lower bounds the minimax risk of estimating unknown model parameters.
This paper examines error bounds for deep learning classifiers with noisy labels.
problem Understanding the performance of classifiers trained on noisy data.
method Derives error bounds for excess risk, decomposing it into statistical and approximation errors. Uses independent block construction for statistical dependencies and vector-valued setting for approximation error.
result Established theoretical results for error bounds in deep learning with noisy labels, mitigating the impact of high-dimensional input spaces.
The study examines how quantum resources enhance the complexity of quantum circuits.
problem Quantum resource enhancement on circuit complexity.
method Utilizing quantum resource theories, the study analyzes statistical complexities of quantum circuits with limited quantum resources.
result Bounds for statistical complexities of quantum circuits are derived and applied to specific cases.
Paper develops PAC verification for hypothesis classes and statistical algorithms.
problem Verifying machine learning models interactively.
method Develops interactive proof for PAC verification, proves lower bounds, and introduces a generalization.
result Improved protocol for verifying unions of intervals and statistical query algorithms.
New bounds for score matching in polynomial exponential families.
problem Understanding the sample complexity of score matching for polynomial exponential families.
method Non-asymptotic sample complexity analysis for score matching.
result First finite sample bounds for score matching in polynomial exponential families.
A note on extending Chernoff bound for unit interval random variables.
problem Extending the Chernoff bound to random variables in the unit interval.
method Proof of extension of the Chernoff bound.
result A proof provided for the extension of the Chernoff bound.
Paper improves Bayesian inference in federated learning with new algorithm VR-FALD*.
problem Bayesian inference in federated learning with communication bottlenecks and statistical heterogeneity.
method Federated Averaging Langevin Dynamics (FALD) and VR-FALD*.
result VR-FALD* corrects client drift due to statistical heterogeneity, improving convergence.
New error bounds for GANs with nonlinear objective functions derived.
problem Statistical consistency of GANs with nonlinear objective functions.
method Derivation of statistical error bounds for (f,Γ)-GANs using Rademacher complexity. result Proves the statistical consistency of (f,Γ)-GANs. New SQ lower bounds show learning mixtures of bounded covariance Gaussians is hard.
problem Learning mixtures of Gaussians with bounded covariance matrices is hard.
method Statistical Query (SQ) lower bounds.
result Any SQ algorithm requires complexity at least dΩ(1/ε) for learning mixtures of bounded covariance Gaussians. New bounds on learning from multiple distributions for VC classes.
problem Understanding the sample complexity of learning from multiple data distributions.
method Analyzing the gap between known upper and lower bounds for PAC-learnable classes.
result Recent progress on sample complexity for VC dimension d classes on k distributions.
Estimates matrix trace optimization with statistical learning theory.
problem Optimizing trace of parameter-dependent matrices.
method Monte Carlo estimator with bounds derived from epsilon nets and generic chaining.
result Predicts small sampling amount for matrices with small off-diagonal mass.
Loose bounds found for least-norm interpolant in over-parameterized settings.
problem Failures of model-dependent generalization bounds for least-norm interpolation.
method Analysis of generalization performance of least-norm linear regressor in over-parameterized regime.
result Generalization bounds for least-norm interpolant can be very loose, even when true excess risk goes to zero.
Study shows depth improves generalization in deep learning models.
problem Understanding why and when depth improves generalization in deep learning.
method Implementation-agnostic state-transition model to analyze depth and generalization.
result Identifies geometric and semigroup mechanisms that keep entropy contribution saturated or polynomial, clarifying depth's statistical advantage.
The paper offers error bounds for quantized dynamical models.
problem Accuracy of dynamical models from dependent data sequences.
method Developed uniform error bounds for quantized models and imperfect optimization algorithms.
result Unified bounds for slow and fast rates, scaling with model encoding bits.
A new stopping criterion for active learning based on deterministic generalization bounds.
problem Determining the optimal stopping point for active learning when data acquisition is costly.
method The proposed stopping criterion is based on the difference in expected generalization errors and hypothesis testing, derived from PAC-Bayesian theory.
result The proposed stopping criterion effectively stops active learning by combining an upper bound with a statistical test.
Optimal SQ bounds for learning binary product distributions and Ising models.
problem Learning binary product distributions and Ising models robustly.
method Statistical Query (SQ) lower bounds for robust learning.
result Optimal SQ lower bounds match known algorithm error guarantees.
Study near-optimal bounds for learning Gaussian halfspaces with random noise.
problem Learning general halfspaces with Gaussian distribution and random classification noise.
method Established nearly-matching algorithmic and SQ lower bounds, developed a computationally efficient learning algorithm.
result Sample complexity of learning algorithm is O(d/ε+d/(max{p,ε})2), SQ lower bound is Ω(d1/2/(max{p,ε})2). New SQ lower bound shows complexity nearly matches known upper bound for smoothed agnostic learning.
problem Smoothed agnostic learning of halfspaces under subgaussian distributions.
method Statistical Query (SQ) lower bound using moment-matching hard distribution and linear programming duality.
result First non-trivial lower bound on complexity nearly matches known upper bound.
We uncover a fairly general principle in online learning: If regret can be (approximately) expressed as a function of certain "sufficient statistics" for the data sequence, then there exists a special Burkholder function that 1) can be used algorithmically to achieve the regret bound and 2) only depends on these suffic…
Proves SQ lower bounds for learning two-hidden-layer neural networks.
problem Learning two-hidden-layer ReLU networks with Gaussian inputs.
method Refined lifting procedure to reduce Boolean PAC learning to Gaussian.
result Superpolynomial SQ lower bounds for Gaussian inputs.
The paper provides bounds for the empirical angular measure and applies them to improve statistical learning in extreme regions.
problem Estimating the angular measure in high-dimensional data with different distributions.
method Established bounds for the maximal deviations of the empirical angular measure from the true measure, using rank transformation and analyzing the most extreme observations.
result The bounds provide performance guarantees for statistical learning procedures in extreme regions, such as binary classification and anomaly detection.
This work provides statistical guarantees for VAEs using PAC-Bayesian theory.
problem Theoretical properties of VAEs remain open questions.
method PAC-Bayesian theory to derive statistical guarantees.
result Upper bounds on Wasserstein distance between input and generative model.
Study problem-dependent rates in statistical learning theory, achieving optimal generalization error bounds.
problem Generalization error in statistical learning theory.
method Uniform localized convergence framework.
result Optimal generalization error bounds for various learning problems.
We study statistical risk minimization problems under a privacy model in which the data is kept confidential even from the learner. In this local privacy framework, we establish sharp upper and lower bounds on the convergence rates of statistical estimation procedures. As a consequence, we exhibit a precise tradeoff be…
New algorithms learn multi-index models via harmonic analysis, achieving statistical and computational trade-offs.
problem Learning multi-index models with unknown projections of input data.
method Exploiting the equivariance of the problem under the orthogonal group, we derive lower bounds and construct spectral algorithms based on harmonic tensor unfolding.
result Achieve statistical and computational trade-offs between sample and runtime complexity.
New approach uses interpolation models and error bounds for verifiable scientific machine learning.
problem Challenges in verifying and validating modern scientific machine learning workflows.
method Combines multiple standard interpolation techniques with error bounds for efficient computation and comparative performance analysis.
result Error bounds for interpolation techniques can be computed or estimated efficiently, aiding in validation goals.
Near-optimal rates for multi-task learning with shared representations.
problem Approximation and statistical complexity of learning multiple operators.
method Multiple Neural Operators (MNO) architecture and comparison with DeepONet.
result Near-optimal upper and lower bounds for approximation and generalization.
Survey on domain adaptation theory, focusing on learning bounds and guarantees.
problem Learning under changing distributions without new labeled data.
method Theoretical analysis of existing transfer learning frameworks.
result Overview of state-of-the-art theoretical results in domain adaptation.
Study on U-statistics with heavy-tailed samples, providing tail bounds and LDP.
problem Deviation of U-statistics with heavy-tailed samples.
method Exponential tail bounds and Large Deviation Principle (LDP) for U-statistics.
result Obtained an exponential upper bound for U-statistics tail decay, showing two regions of decay.
Paper analyzes statistical efficiency of TD learning in Hilbert spaces with Freedman's inequality.
problem Statistical efficiency of distributional TD learning in Hilbert spaces.
method Non-parametric distributional TD (NTD) and variance-reduced variants of NTD and CTD.
result Sharp statistical rates achieved through novel Freedman's inequality in Hilbert spaces.
Estimates shared parameters across related learning problems using robust statistics and LASSO.
problem Simultaneously learning related but heterogeneous problems like store demand or patient risk.
method Two-stage multitask learning estimator combining robust statistics and LASSO regression.
result Improved sample complexity bounds for multitask learning, especially beneficial for 'data-poor' instances.