Paper extends learning theory to dependent data with uniform risk bounds.
problem Learning with dependent data sequences.
method Derives uniform risk bounds for dependent data using VC-dimension and Rademacher complexity.
result Standard classification risk bounds hold for dependent data, same as for independent data.
Develops uniform convergence guarantees for a broad class of risk functionals in supervised learning.
problem Bounding generalization gaps for various risk functionals beyond the expectation.
method Establishes uniform convergence for Hölder risk functionals, providing guarantees for empirical risk minimization.
result First uniform convergence results for estimating the CDF of loss distributions, applicable to various risk functionals.
Logistic regression gets a new, simpler uniform bound.
problem Finding a uniform bound for logistic regression's empirical risk.
method PAC-Bayes approach with second-order expansion and Rademacher-complexity bounds.
result Provides a dimension-free uniform concentration bound.
Establishes a link between risk measures and uniform integrability in finance.
problem Understanding uniform integrability in the context of financial risk measures.
method Introduces the folding score of distortion risk measures to study uniform integrability directly with gains and losses.
result Obtains three sets of equivalent conditions for uniform integrability involving coherent risk measures.
One fundamental goal in any learning algorithm is to mitigate its risk for overfitting. Mathematically, this requires that the learning algorithm enjoys a small generalization risk, which is defined either in expectation or in probability. Both types of generalization are commonly used in the literature. For instance, …
New approach to adversarial robustness with non-uniform perturbations.
problem Real-world adversaries craft adversarial examples with non-uniform perturbations.
method Proposes non-uniform perturbations based on feature dependencies and data distribution.
result Shows improved robustness to real-world attacks compared to uniform perturbations.
This work establishes uniform convergence of subdifferentials in stochastic optimization.
problem Understanding how empirical stationary points approximate population ones in nonsmooth, nonconvex stochastic optimization.
method Reduction principle for weakly convex stochastic objectives, focusing on subgradient convergence.
result Sharp uniform convergence rates for subdifferential mappings in stochastic convex-composite optimization.
We investigate optimal consumption and investment problems for a Black-Scholes market under uniform restrictions on Value-at-Risk and Expected Shortfall. We formulate various utility maximization problems, which can be solved explicitly. We compare the optimal solutions in form of optimal value, optimal control and opt…
Study uniform learnability of binary classification networks with communication.
problem Learning a network with communication between vertices from uniform ergodic Random Graph Process.
method Introduced structural Rademacher complexity and used martingale method and Marton's coupling.
result Uniform learnability as worst-case theoretical limits for binary classification problems.
New algorithms achieve uniform stability for empirical risk minimization.
problem Designing uniformly stable optimization algorithms for empirical risk minimization.
method Black-box conversion of smooth optimization algorithms and development of Mirror Descent for smooth optimization.
result Optimal algorithms with uniform stability and convergence rates for smooth optimization.
Improved DP SO with large Lipschitz parameters, handling outliers and heavy-tailed data.
problem Differential privacy in stochastic optimization with large Lipschitz parameters.
method Assumes bounded k-th order moments, provides linear-time algorithms for smooth convex and non-smooth convex losses.
result Improved risk bounds scaling with k-th moment, not uniform Lipschitz parameter.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
New method turns optimization algorithms into uniformly stable learning algorithms for non-Euclidean norms.
problem Non-Euclidean norms in binary classification problems.
method Black-box reduction method using uniformly convex regularizers.
result Achieves optimal statistical risk bounds on excess risk for non-Euclidean norms.
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
problem Generalization bounds with strict assumptions like uniformly bounded loss.
method Relax uniform bounds assumptions to on-average bounded loss and gradient norm.
result Proposes a new generalization bound with a surrogate of model complexity.
There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in …
The paper derives uniform stability-based coverage bounds for conformal prediction methods.
problem Establishing theoretical guarantees for conformal prediction methods.
method Uniform stability perspective applied to full-conformal, jackknife+, and CV+ prediction regions.
result Coverage bounds for finite-dimensional models derived using a concentration argument.
The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.
problem Limited theoretical understanding of learning algorithms achieving high accuracy and good calibration.
method Focuses on smooth calibration error, providing a uniform convergence bound and proving the relationship with functional gradient.
result Derives conditions for simultaneous classification and calibration guarantees in gradient boosting trees, kernel boosting, and neural networks.
Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss…
Develops bounds for deep learning risk via Hilbert coresets.
problem Risk estimation for complex deep learning models.
method Hilbert coreset approach for transductive risk bounds.
result Effective and meaningful bounds for deep neural networks.
The Statistical Learning Theory (SLT) provides the theoretical guarantees for supervised machine learning based on the Empirical Risk Minimization Principle (ERMP). Such principle defines an upper bound to ensure the uniform convergence of the empirical risk Remp(f), i.e., the error measured on a given data sample, to …
The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.
problem Understanding the risk of CV-tuned regularized estimators.
method Derives asymptotic risk function of CV-tuned estimators and connects it to SURE.
result The risk function provides a more detailed picture of predictive performance than uniform bounds.
Paper introduces risk assessment for contextual bandits without experiments.
problem Evaluate policies using logged data in context bandits.
method Lipschitz risk functionals and Off-Policy Risk Assessment (OPRA) framework.
result OPRA provides finite sample guarantees for various risk estimates.
This guide simplifies high-probability regret bounds in empirical risk minimization.
problem High-probability regret bounds in empirical risk minimization.
method Modular presentation, three-step recipe, localized Rademacher complexity, local maximal inequalities, metric-entropy integrals.
result Recover familiar rates for various function classes and derive regret bounds for nuisance components.
A new method for matrix completion with model-free weights.
problem Matrix completion under non-uniform missing structures.
method Constructs weights via convex optimization to adjust for non-uniformity without modeling observation probabilities.
result Recover matrix with stronger theoretical guarantees, especially in heterogeneous missing settings.
The paper analyzes SMOTE for imbalanced classification, providing theoretical bounds and guidelines.
problem The challenge of imbalanced classification problems, especially with minority classes.
method Theoretical analysis of SMOTE and related oversampling techniques for minority classes.
result Derives concentration and excess risk bounds for SMOTE and kernel-based classifiers.
We investigate optimal consumption problems for a Black-Scholes market under uniform restrictions on Value-at-Risk and Expected Shortfall for logarithmic utility functions. We find the solutions in terms of a dynamic strategy in explicit form, which can be compared and interpreted. This paper continues our previous wor…
Paper improves risk bounds for nonconvex-strongly-concave minimax problems.
problem Achieving sharper risk bounds for nonconvex-strongly-concave minimax problems.
method Using uniform localized convergence to derive high probability generalization error bounds.
result Derives n times faster excess primal risk bounds for popular algorithms.
We study adversarial perturbations when the instances are uniformly distributed over { 0 , 1 } n \{0,1\}^n { 0 , 1 } n . We study both "inherent" bounds that apply to any problem and any classifier for such a problem as well as bounds that apply to specific problems and specific hypothesis classes. As the current literature contains multiple…
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
New stability bounds for SGD on nonsmooth convex losses.
problem Understanding stability of SGD on nonsmooth convex losses.
method Sharp upper and lower bounds for SGD and full-batch GD on nonsmooth convex losses.
result SGD can be less stable but still useful for generalization bounds.
New method reduces generalization error for interpolating predictors.
problem Understanding and reducing generalization error for predictors that interpolate training data.
method Derandomization and conditional distribution to control generalization error.
result Surrogates constructed by conditioning and denoising have uniformly small generalization error.
New algorithm reduces ERM problem size while maintaining accuracy.
problem Empirical risk minimization problem size reduction.
method Adaptive Deterministic Uniform-Weight Trimming (ADUWT) algorithm.
result Uniform ( 1 ± ε ) (1\pm\varepsilon) ( 1 ± ε ) relative-error approximation for ERM objective. Paper analyzes risk bounds for in-context learning in multiclass classification.
problem Risk bounds for in-context learning in multiclass classification.
method Formalizes tasks as sequences of labeled examples and queries, estimates conditional class probabilities, establishes oracle inequality for KL divergence.
result ICL achieves minimax optimal rate for conditional probability estimation.
Unified framework for risk-aware policy learning in contextual bandits.
problem Optimizing decision rules in high-stakes domains with adverse outcomes.
method Distributional framework for Lipschitz-continuous risk functionals, with novel empirical concentration inequalities.
result Data-dependent suboptimality bounds with an i l d e O ( 1 / n ) ilde{\mathcal{O}}(1/\sqrt{n}) i l d e O ( 1/ n ) rate, matching risk-neutral offline policy optimization. Study problem-dependent rates in statistical learning theory, achieving optimal generalization error bounds.
problem Generalization error in statistical learning theory.
method Uniform localized convergence framework.
result Optimal generalization error bounds for various learning problems.
PAC-Bayes bounds for Gibbs posteriors derived via singular learning theory.
problem Generalization bounds for overparameterized models with data-dependent priors.
method Explicit non-asymptotic PAC-Bayes bounds using singular learning theory.
result Explicit posterior-averaged risk bounds for overparameterized models.
The paper examines the tilted empirical risk's generalization and robustness under negative tilt.
problem The generalization error of machine learning algorithms under negative tilt.
method Uniform and information-theoretic bounds on the tilted generalization error under negative tilt.
result The tilted empirical risk's generalization error has a convergence rate of \(O(n^{-ε/(1+ε)})\).
Optimal posterior distributions improve SVM classifiers and parameter selection.
problem Improving SVM classifiers and selecting optimal regularization parameters.
method PAC-Bayesian approach with optimal posterior identification for stochastic classifiers.
result Optimal posteriors yield tight risk bounds and improved SVM performance.
Study tests uniformity of categorical data against missing-ball alternatives, finding chi-squared test outperforms.
problem Testing uniformity of categorical data against missing-ball alternatives.
method Characterizes minimax risk, uses collisions and chi-squared test, reduces to structured subset of alternatives.
result Minimax test outperforms chi-squared test under least favorable alternative.
Expanding on techniques of concentration of measure, we develop a quantitative framework for modeling liquidity risk using convex risk measures. The fundamental objects of study are curves of the form ( ρ ( λ X ) ) λ ≥ 0 (ρ(λX))_{λ\ge 0} ( ρ ( λ X ) ) λ ≥ 0 , where ρ ρ ρ is a convex risk measure and X X X a random variable, and we call such a curve a \emph{liqu…
Stochastic Gradient Descent underperforms on some problems, contrary to expectations.
problem Understanding the generalization performance of SGD on specific problem instances.
method Analysis of stochastic convex optimization framework, proving empirical and generalization gaps for SGD.
result SGD exhibits both empirical risk and generalization gap of Ω ( 1 ) Ω(1) Ω ( 1 ) on some problem instances, contradicting its conventional understanding. Paper proposes a new framework to improve stability-based bounds in deep learning.
problem Explaining generalization in overparameterized neural networks.
method Decomposes excess risk dynamics into signal and noise components, applying stability-based bounds only to the noise.
result The decomposition framework improves stability-based bounds and explains generalization in neural networks.
New method improves solving combinatorial optimization problems with smoothed policies.
problem Solving combinatorial optimization problems repeatedly with varying instances.
method Smoothed policies with controlled random perturbations to linear oracle, leading to differentiable surrogate risk.
result Generalization bound decomposes excess risk into bias, estimation, and optimization components.
Improved uniform convergence bound with fat-shattering dimension reduces sample complexity gap.
problem Gap between upper and lower bounds on sample complexity for fat-shattering dimension.
method Provided an improved uniform convergence bound.
result Closed the gap between existing upper and lower bounds on sample complexity.
Paper estimates the order of vertices in random recursive trees.
problem Estimating the order of arrival of vertices in random recursive trees.
method Proposes an order estimator based on the Jordan centrality measure and defines risk measures.
result Establishes a nearly optimal estimator for the problem.
Paper presents a low-cost algorithm for bipartite ranking with improved sample size requirements.
problem Bipartite ranking's quadratic dependence on sample size makes it computationally expensive.
method Uses a novel uniform risk bound based on matrix and vector concentration inequalities to achieve low cost and competitive performance.
result Shows that the sample size required for competitive performance is not quadratic, improving efficiency.
We revisit Spakula's uniform K-homology, construct the external product for it and use this to deduce homotopy invariance of uniform K-homology. We define uniform K-theory and on manifolds of bounded geometry we give an interpretation of it via vector bundles of bounded geometry. We further construct a cap product with…
Uniform entropy bound for Ricci shrinkers with bounded curvature.
problem Bounding entropy for Ricci shrinkers with specific curvature constraints.
method Establishing uniform entropy bounds for simply connected Ricci shrinkers with a finite second homotopy group and uniform curvature bounds.
result Uniform entropy bound for simply connected Ricci shrinkers with a finite second homotopy group and uniform curvature bounds.