Logistic regression gets a new, simpler uniform bound.
problem Finding a uniform bound for logistic regression's empirical risk.
method PAC-Bayes approach with second-order expansion and Rademacher-complexity bounds.
result Provides a dimension-free uniform concentration bound.
Sharp bounds on uniform generalization errors in binary linear classification.
problem Understanding the uniform generalization errors in binary linear classification.
method Isoperimetric arguments, Poincaré and log-Sobolev inequalities for joint distributions.
result Sharp concentration bounds on uniform generalization errors, almost sure convergence in broad settings.
We give concentration bounds for martingales that are uniform over finite times and extend classical Hoeffding and Bernstein inequalities. We also demonstrate our concentration bounds to be optimal with a matching anti-concentration inequality, proved using the same method. Together these constitute a finite-time versi…
Uniform TD(0) bound derived for function approximation with Markov noise.
problem Uniform concentration bound for TD(0) with function approximation.
method Contractive stochastic approximation, martingale and Markov noises, Poisson equation, relaxed concentration inequalities.
result Uniform all-time concentration bound for TD(0) with linear function approximation.
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and error functionals of l2-regularized algorithms.
One fundamental goal in any learning algorithm is to mitigate its risk for overfitting. Mathematically, this requires that the learning algorithm enjoys a small generalization risk, which is defined either in expectation or in probability. Both types of generalization are commonly used in the literature. For instance, …
Proves convergence of gradient Ricci shrinkers with uniform bounds.
problem Compactness and energy concentration in gradient Ricci shrinkers.
method Bubble-tree convergence and local energy analysis.
result No energy concentrates in neck regions, leading to a local diffeomorphism finiteness theorem.
New sampling bounds improve uniform coverage verification in machine learning.
problem Conservative bounds in classical coverage analyses at small failure probabilities.
method Variance-based analysis of uniform random sampling on a d-dimensional unit hypercube. result Sample complexity bound with logarithmic dependence on failure probability.
We give tight concentration bounds for mixtures of martingales that are simultaneously uniform over (a) mixture distributions, in a PAC-Bayes sense; and (b) all finite times. These bounds are proved in terms of the martingale variance, extending classical Bernstein inequalities, and sharpening and simplifying prior wor…
Paper addresses concentration of distances for fractional quasi p-norms, identifying conditions for concentration and anti-concentration.
problem Understanding concentration of distances for fractional quasi p-norms in high dimensions.
method Analyzes conditions for concentration and anti-concentration of distances for fractional quasi p-norms.
result Identifies conditions for concentration and anti-concentration of fractional quasi p-norms, ruling out some approaches and specifying conditions for control.
Unified framework for robust clustering under various dissimilarity measures.
problem Improving center-based clustering methods to handle outliers and non-Euclidean data.
method Median-of-Means (MoM) estimation framework with uniform concentration bounds.
result Strong consistency and error rate of O(n−1/2) under mild conditions. The paper derives uniform stability-based coverage bounds for conformal prediction methods.
problem Establishing theoretical guarantees for conformal prediction methods.
method Uniform stability perspective applied to full-conformal, jackknife+, and CV+ prediction regions.
result Coverage bounds for finite-dimensional models derived using a concentration argument.
The paper analyzes SMOTE for imbalanced classification, providing theoretical bounds and guidelines.
problem The challenge of imbalanced classification problems, especially with minority classes.
method Theoretical analysis of SMOTE and related oversampling techniques for minority classes.
result Derives concentration and excess risk bounds for SMOTE and kernel-based classifiers.
Valid p-value for bounded random variables without distributional assumptions.
problem Calibration of predictive algorithms in a distribution-free setting.
method Built a super-uniform p-value based on a concentration inequality.
result Super-uniform p-value is tighter than existing alternatives.
New concentration inequality for U-statistics of Markov chains.
problem Proving a concentration inequality for U-statistics of order two in uniformly ergodic Markov chains.
method Inductive analysis using martingale techniques, uniform ergodicity, Nummelin splitting, and Bernstein's inequality.
result Recovery of convergence rate for U-statistics of independent random variables and canonical kernels, with improved results for dependent kernels.
We compute the expected value of the Kullback-Leibler divergence to various fundamental statistical models with respect to canonical priors on the probability simplex. We obtain closed formulas for the expected model approximation errors, depending on the dimension of the models and the cardinalities of their sample sp…
NUTS mixing time scales as d^(1/4) for Gaussian distributions.
problem Improving the efficiency of the No-U-Turn Sampler (NUTS) for Gaussian distributions.
method Coupling argument leveraging geometric structure of Gaussian concentration, uniformity analysis of NUTS transitions.
result The mixing time of NUTS scales as d^(1/4) for Gaussian distributions, up to logarithmic factors.
There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in …
Expanding on techniques of concentration of measure, we develop a quantitative framework for modeling liquidity risk using convex risk measures. The fundamental objects of study are curves of the form (ρ(λX))λ≥0, where ρ is a convex risk measure and X a random variable, and we call such a curve a \emph{liqu…
Paper shows how online betting algorithms' regret can be used to create tight confidence sequences.
problem Estimating the expectation of random variables from samples and creating time-uniform confidence sequences.
method Converts the regret guarantee of universal portfolio algorithms into time-uniform concentration inequalities and confidence sequences.
result Numerically obtained confidence sequences are never vacuous and satisfy the law of iterated logarithm.
New bounds on self-normalized martingales improve online linear regression performance.
problem Improving regret bounds in online linear regression.
method Characterizing scale-invariant bounds on self-normalized martingales.
result For d=1, O(logT) doubly-uniform regret is possible; for d>1, sublinear doubly-uniform regret is impossible. New stability framework relaxes boundedness assumptions for generalization bounds.
problem Overly restrictive assumptions for modern learning settings with heavy-tailed or unbounded losses.
method Develops a stability-based framework requiring only finite Lp moment conditions. result Sharp generalization bounds derived for various learning paradigms.
The paper develops bounds for predictive values in binary classification.
problem Lack of confidence intervals for positive and negative predictive values.
method Bi-criterion framework and distribution-free large deviation and uniform convergence bounds.
result New bounds for predictive values without relying on concentration inequalities.
New inequalities for unbounded functions improve denoising score matching.
problem Statistical error bounds for denoising score matching with unbounded objective functions.
method Derive new concentration inequalities using McDiarmid's inequality and Rademacher complexity bounds.
result Improved statistical error bounds for denoising score matching.
Greedy algorithm achieves sublinear regret for various distributions.
problem Efficient performance of greedy algorithms in linear contextual bandit problems.
method Introduced Local Anti-Concentration (LAC) condition to ensure sublinear regret.
result Greedy algorithm achieves O(polylogT) cumulative expected regret. We give a unified statement and proof of a class of wellknown mean value inequalities for nonnegative functions with a nonlinear bound on the Laplacian. We generalize these to domains with boundary, requiring a (possibly nonlinear) bound on the normal derivative at the boundary. These inequalities give rise to an energ…
UCRL3 improves UCRL2's efficiency in reinforcement learning by reducing exploration.
problem Long burn-in phases in numerical experiments of UCRL2.
method UCRL3 uses state-of-the-art time-uniform concentration inequalities and adaptive support computation to tighten exploration.
result UCRL3 achieves a better numerical improvement over UCRL2 in standard environments.
New method removes scalar curvature assumption in Ricci flow smoothing.
problem Uniform bounds on scalar curvature and other factors for Ricci flow.
method Quantitative short-time existence of Ricci flow without scalar curvature assumption.
result Ricci flow smoothing for measure space limits, Gromov-Hausdorff compactness, and topological rigidity results.
Max-sliced Wasserstein metric reduces high-dimensional data to 1D for better estimation.
problem Curse of dimensionality in optimal transport.
method Introduces max-sliced Wasserstein metric to reduce high-dimensional problems to 1D.
result Uniform ratio bounds of empirical measures on RKHS concentrate uniformly fast at parametric rates.
Paper improves CI and CS for bounded means using betting and mixtures.
problem Estimating means of bounded random variables.
method Composite nonnegative martingales, testing by betting, method of mixtures.
result Empirically outperforms existing CI and CS methods.
Uniform scaling limits in AdamW-trained transformers converge to ODEs.
problem Understanding the dynamics of large-depth transformers trained with AdamW.
method Modeling transformer dynamics as an interacting particle system coupled through attention, proving convergence to ODEs.
result The joint dynamics of hidden states and backpropagated variables converge uniformly to an ODE system.
Paper presents a low-cost algorithm for bipartite ranking with improved sample size requirements.
problem Bipartite ranking's quadratic dependence on sample size makes it computationally expensive.
method Uses a novel uniform risk bound based on matrix and vector concentration inequalities to achieve low cost and competitive performance.
result Shows that the sample size required for competitive performance is not quadratic, improving efficiency.
Paper introduces risk assessment for contextual bandits without experiments.
problem Evaluate policies using logged data in context bandits.
method Lipschitz risk functionals and Off-Policy Risk Assessment (OPRA) framework.
result OPRA provides finite sample guarantees for various risk estimates.
This guide simplifies high-probability regret bounds in empirical risk minimization.
problem High-probability regret bounds in empirical risk minimization.
method Modular presentation, three-step recipe, localized Rademacher complexity, local maximal inequalities, metric-entropy integrals.
result Recover familiar rates for various function classes and derive regret bounds for nuisance components.
Paper tightens PAC-Bayes bounds using coin-betting for better estimates.
problem Estimating mean of random elements with possibly S-dependent parameters.
method Refined PAC-Bayes proof strategy based on coin-betting framework.
result Derives tighter concentration inequalities for all sample sizes.
This paper refines learning algorithm bounds, proving sharper generalization and lower bounds.
problem Improving generalization bounds for uniformly stable learning algorithms.
method Developed a new concentration inequality for weakly correlated random variables and proved sharper generalization and lower bounds.
result The new concentration inequality implies a stronger generalization bound than previous results.
New property ensures neural networks generalize well with limited data.
problem Limited training data limits model generalization in neural networks.
method Introduces NeuRIP, a uniform concentration event for ReLU networks.
result All shallow ReLU networks generalize uniformly if they achieve NeuRIP.
Paper quantizes heavy-tailed data for near optimal estimation rates.
problem Estimating parameters from heavy-tailed data with quantization.
method Truncate and dither data, then uniformly quantize; achieves near minimax rates.
result Near optimal estimation rates achievable with quantized data.
New method certifies anti-concentration for various non-Gaussian distributions.
problem Efficiently certifying anti-concentration for non-Gaussian distributions.
method Sum-of-Squares relaxation of integer program for anti-concentration.
result Quasi-polynomial time certificates for non-Gaussian distributions.
Tests for overfitting in machine learning models.
problem Overfitting in high complexity models.
method Hypothesis test using concentration bounds.
result Valid test for identifying overfitting.
Hypergraph partitioning lies at the heart of a number of problems in machine learning and network sciences. Many algorithms for hypergraph partitioning have been proposed that extend standard approaches for graph partitioning to the case of hypergraphs. However, theoretical aspects of such methods have seldom received …
Proposes a new latent variable model for hyperspherical latent spaces.
problem Efficiently modeling heavy-tailed distributions in hyperspherical latent spaces.
method Introduces spherical Cauchy (spCauchy) latent variables and applies Möbius transformations.
result Shows spCauchy recovers vMF geometry in high-concentration limits and avoids complex evaluations.
Study uniform convergence of random walk Laplacians to diffusion Laplacian on smooth manifolds.
problem Uniform convergence of random walk Laplacians to diffusion Laplacian on smooth manifolds.
method Analysis of random walks on geometric and directed kNN graphs, using concentration tools and differential geometry.
result Uniform convergence of kNN Laplacians to diffusion Laplacian, without continuity of transition kernel. Paper improves learning efficiency by focusing on effective dimensionality.
problem Dimensionality bottleneck in modern learning tasks.
method Developed tools to reduce dimensional costs using effective dimensionality.
result Uniform concentration bounds involving effective dimensionality, improving over existing results.
Study provides bounds for estimating intrinsic dimension using Gaussian kernels.
problem Estimating intrinsic dimension from data.
method Finite-sample concentration and anti-concentration bounds for Gaussian kernel sums.
result Explicit dependence on sample size, bandwidth, and geometric parameters.
Sharp threshold for exact recovery in non-uniform hypergraph stochastic block model.
problem Community detection in random hypergraphs with non-uniform hyperedge probabilities.
method Sharp threshold established; two efficient algorithms for exact recovery.
result Sharp threshold for exact recovery; information-theoretic lower bound on misclassification.
Q-learning for average cost MDPs gets a concentration bound.
problem Finding bounds for Q-learning in average cost MDPs.
method Derives a concentration bound using shortest path problem equivalence.
result Numerical comparison with relative value iteration shows the bound's effectiveness.
In this paper, we observe a set of functionals of metrics which are all decrease under the Calabi flow and have uniform lower bound along the flow, which give rise to a set of integral estimates on the curvature flow. Using these estimates, together with weak compactness we obtained in previous papers [8] and [10], we …