The paper provides a uniform convergence bound for smooth calibration error and its relationship with functional gradient.
problem Limited theoretical understanding of learning algorithms achieving high accuracy and good calibration.
method Focuses on smooth calibration error, providing a uniform convergence bound and proving the relationship with functional gradient.
result Derives conditions for simultaneous classification and calibration guarantees in gradient boosting trees, kernel boosting, and neural networks.
Data-driven models are subject to model errors due to limited and noisy training data. Key to the application of such models in safety-critical domains is the quantification of their model error. Gaussian processes provide such a measure and uniform error bounds have been derived, which allow safe control based on thes…
New bounds for kernel regression under non-Gaussian noise.
problem Uncertainty quantification for function estimates from noisy observations.
method Novel non-asymptotic probabilistic uniform error bounds for kernel-based regression.
result Proposed bounds apply to a broad class of non-Gaussian noise distributions.
This paper addresses error bounds and posterior variance for Gaussian process regression.
problem Deriving performance guarantees for Gaussian process regression without prior knowledge.
method Lipschitz continuity and analysis of posterior variance function.
result Uniform error bounds for Gaussian process regression are derived.
ET-GP-UCB optimizes time-varying functions without knowing change rates.
problem Sequentially optimizing a time-varying objective function with unknown change rates.
method Event-triggered Bayesian optimization with adaptive resets based on probabilistic uniform error bounds.
result ET-GP-UCB outperforms other GP-UCB algorithms in synthetic and real-world data.
Study shows gap between uniform convergence and test error in random feature models.
problem Understanding the gap between uniform convergence and test error in random feature models.
method Analytical expressions for uniform convergence over norm balls, interpolators, and minimum norm interpolator risk derived and proved.
result Uniform convergence over interpolators still gives a non-trivial bound of test error even when classical uniform convergence is vacuous.
Sharp bounds on uniform generalization errors in binary linear classification.
problem Understanding the uniform generalization errors in binary linear classification.
method Isoperimetric arguments, Poincaré and log-Sobolev inequalities for joint distributions.
result Sharp concentration bounds on uniform generalization errors, almost sure convergence in broad settings.
New bounds show multicalibration error is close to prediction error.
problem Addressing fairness in machine learning systems.
method Sample complexity bounds for uniform convergence of multicalibration error.
result Uniform convergence guarantees for multicalibration error, independent of prediction error.
We study the spherical cap packing problem with a probabilistic approach. Such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector. When the set of unit vectors …
The paper offers error bounds for quantized dynamical models.
problem Accuracy of dynamical models from dependent data sequences.
method Developed uniform error bounds for quantized models and imperfect optimization algorithms.
result Unified bounds for slow and fast rates, scaling with model encoding bits.
This paper introduces minimum-risk recalibration for probabilistic classifiers, improving their reliability and accuracy.
problem Improving the reliability and accuracy of probabilistic classifiers.
method Minimum-risk recalibration within the MSE decomposition framework, analyzing UMB method and label shift adaptation.
result The optimal number of bins for UMB scales with n1/3, resulting in a risk bound of approximately O(n−2/3). The paper analyzes the probabilistic structure of DDPMs and bounds their sampling error.
problem Understanding and controlling errors in discrete-time DDPMs.
method Structural analysis of score functions, Schrödinger's problem, and FBSDEs.
result Explicit upper bound for total variation distance between sampling and target distributions.
Improved error estimate for SGLD sampling algorithm.
problem Establishing a precise error bound for SGLD.
method Sharp uniform-in-time error estimate for SGLD under mild assumptions.
result Uniform-in-time O(η2) bound for KL-divergence between SGLD and Langevin diffusion. The paper explores why a specific type of predictor works well in noisy data.
problem Understanding why a specific type of predictor (minimum-norm interpolator) works well in noisy data.
method The paper uses uniform convergence and zero-error predictors in a norm ball to explain the success of the minimum-norm interpolator.
result The minimum-norm interpolator is consistent, and this can be explained by uniform convergence of zero-error predictors in a norm ball.
A new error bound improves safety in Bayesian optimization.
problem Ensuring safety in Bayesian optimization with probabilistic models.
method Introducing a novel error bound using Wiener kernel regression for Gaussian processes and noise.
result The new error bound provides larger safety regions than previous methods.
Continuous-time PCD for MLE with explicit error bounds.
problem Maximum likelihood estimation of unnormalised densities.
method Continuous-time formulation as coupled SDEs, deriving UiT bounds.
result Explicit error bounds between PCD iterates and MLE solution.
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and error functionals of l2-regularized algorithms.
The paper provides guarantees for feedback control with sensor errors.
problem Certifying performance and safety in feedback control systems with sensor errors.
method Solving a supervised learning problem to characterize sensor errors and providing uniform error bounds.
result Finite-time convergence rate on sub-optimality of using a regressor in closed-loop for waypoint tracking.
Study problem-dependent rates in statistical learning theory, achieving optimal generalization error bounds.
problem Generalization error in statistical learning theory.
method Uniform localized convergence framework.
result Optimal generalization error bounds for various learning problems.
New bounds for SGLD show error decreases with more data.
problem Establishing generalization error bounds for SGLD in non-convex settings.
method Using dissipativity, smoothness, and uniform stability, time-independent bounds are derived.
result Error bounds decay to zero as sample size increases.
Improved bounds for proximal gradient algorithms with computational errors.
problem Analyzing convergence of proximal gradient algorithms with inaccuracies.
method Deriving new tighter deterministic and probabilistic bounds for convex composite problems.
result Probabilistic bounds are more robust and accurate for algorithm verification and performance guarantees.
New method calibrates probabilistic regression models without restrictive assumptions.
problem Ensuring predictive distributions accurately reflect true uncertainty.
method Nonparametric re-calibration algorithm based on conditional kernel mean embeddings.
result Consistently outperforms prior re-calibration approaches across various benchmarks.
The success of deep learning has led to a rising interest in the generalization property of the stochastic gradient descent (SGD) method, and stability is one popular approach to study it. Existing works based on stability have studied nonconvex loss functions, but only considered the generalization error of the SGD in…
Uniform bounds for neural networks' generalization error in overparameterized settings.
problem Generalization error in overparameterized neural networks.
method Neural Tangent kernel theory and Mercer decomposition of the NT kernel in spherical harmonics.
result Uniform generalization bounds for overparameterized neural networks in RKHS.
Deep ReLU networks generalize well with few parameters.
problem Generalization of overparametrized deep neural networks.
method Explicit bounds on test error independent of overparametrization and VC dimension.
result Generalization error is independent of network architecture and overparametrization.
Paper analyzes ECE bias and provides bounds for its estimation.
problem Understanding the estimation bias in ECE for machine learning models.
method Information-theoretic approach to analyze bias in uniform mass and uniform width binning strategies.
result Established upper bounds on ECE estimation bias and optimal number of bins.
We consider an original problem that arises from the issue of security analysis of a power system and that we name optimal discovery with probabilistic expert advice. We address it with an algorithm based on the optimistic paradigm and on the Good-Turing missing mass estimator. We prove two different regret bounds on t…
Improved multiclass classification with class-weighted nearest neighbors.
problem Multiclass classification with large or imbalanced classes.
method Class-weighted k-nearest neighbors algorithm, derived bounds on accuracy and risk.
result Optimized classification metrics like F1 score or Matthew's Correlation Coefficient.
New bounds for learning polynomial surrogates with L∞ guarantees.
problem Learning polynomial surrogates for bounded binary functions with L∞ error guarantees. method Characterized minimax sample complexity for two classes of polynomials under subgaussian noise.
result Sample complexity rates differ from noiseless case, scaling as nd+1 for degree d polynomials and ns2 for sparse polynomials. Paper analyzes GCNN sensitivity to probabilistic graph perturbations.
problem Investigating how GCNNs handle probabilistic graph errors.
method Establishes error bounds and linear relationships between GSO perturbations and GCNN outputs.
result GCNNs maintain stability under graph edge perturbations if GSO errors are bounded.
The paper proposes using function approximations to reduce the computational burden in measuring counterparty credit exposure.
problem The need for regular exposure calculations in finance, balancing between computational cost and risk simplification.
method Replacing derivative pricers with function approximations, proving error bounds, and using Chebyshev interpolation for convergence.
result Derives probabilistic and finite sample error bounds, showing significant run-time reductions and asymptotic efficiency gains.
No algorithm outperforms uniform sampling in A/B testing.
problem Identifying the best arm in A/B testing with fixed budget.
method Introducing consistent and stable algorithms, deriving lower bounds, and proving optimality of uniform sampling.
result No algorithm performs better than uniform sampling in A/B testing.
We study discrete curvatures computed from nets of curvature lines on a given smooth surface, and prove their uniform convergence to smooth principal curvatures. We provide explicit error bounds, with constants depending only on properties of the smooth limit surface and the shape regularity of the discrete net.
The paper tackles learning from non-irreducible Markov chains, proving learnability and generalization bounds.
problem Learning from temporal dependent data with non-irreducible Markov chains.
method Uniform convergence and generalization bounds for sample error under uniform ergodicity.
result Learnability and generalization bounds for approximate sample error minimization algorithm.
We compute the expected value of the Kullback-Leibler divergence to various fundamental statistical models with respect to canonical priors on the probability simplex. We obtain closed formulas for the expected model approximation errors, depending on the dimension of the models and the cardinalities of their sample sp…
Sharp 2-Wasserstein bounds for DDPMs derived from Föllmer process.
problem Sampling error bounds for DDPMs in 2-Wasserstein distance.
method Lipschitz-type conditions on score function, Föllmer process, and log-concave target distributions.
result Sharp upper bounds for DDPMs in 2-Wasserstein distance, optimal in dimension and steps.
Deep learning improves probabilistic PPDE solution accuracy.
problem Approximating solutions to path-dependent PDEs with limited basis selection.
method Deep learning for conditional expectation estimation with error bounds.
result Deep learning yields more accurate PPDE solutions, especially in high dimensions.
Optimizes pruning masks for neural networks using probabilistic fine-tuning and PAC-Bayes bounds.
problem Improving neural network performance through adaptive pruning of weights.
method Optimizes stochastic pruning masks by minimizing expected loss, considering data-adaptive regularization and feature alignment.
result Probabilistic fine-tuning leads to improved test error over baseline methods in neural networks.
The paper develops bounds for predictive values in binary classification.
problem Lack of confidence intervals for positive and negative predictive values.
method Bi-criterion framework and distribution-free large deviation and uniform convergence bounds.
result New bounds for predictive values without relying on concentration inequalities.
This study uses neural networks to approximate Bayesian filtering problems.
problem Estimating latent time-series signal statistics from observation sequences.
method Formulated a generic recurrent neural network framework to learn recursive mappings directly.
result Approximation error bounds for filtering in non-compact domains and strong time-uniform bounds.
A framework assesses the trustworthiness of probabilistic classifiers using local calibration error.
problem Assessing the trustworthiness of probabilistic classifiers beyond traditional metrics.
method I-trustworthy framework linking local calibration to trustworthiness; Kernel Local Calibration Error (KLCE) method for hypothesis testing.
result The effectiveness of the proposed test statistic demonstrated through simulated and real-world datasets.
Aimed at explaining the surprisingly good generalization behavior of overparameterized deep networks, recent works have developed a variety of generalization bounds for deep learning, all based on the fundamental learning-theoretic technique of uniform convergence. While it is well-known that many of these existing bou…
Uniform convergence of interpolators proven for Gaussian data.
problem Interpolation learning in high-dimensional linear regression with Gaussian data.
method Generic uniform convergence guarantee in terms of Gaussian width.
result Consistency of interpolators for minimum-norm and near-minimal-norm cases.
Suppose some classifiers are selected from a set of hypothesis classifiers to form an equally-weighted ensemble that selects a member classifier at random for each input example. Then the ensemble has an error bound consisting of the average error bound for the member classifiers, a term for selectivity that varies fro…
Improved matrix completion for non-uniformly sampled data.
problem Estimating unobserved entries in a matrix with varying sampling probabilities.
method Developed entry-specific bounds for low-rank matrix completion under structured non-uniform sampling.
result Error bounds for each entry match minimax lower bounds under certain conditions.
This paper improves DNN generalization by accurately estimating mutual information.
problem Intractability of estimating mutual information in DNNs.
method Introduces a probabilistic representation of DNNs to accurately estimate mutual information.
result Derives a tighter generalization bound than previous relaxations.
New bounds on ReLU networks for low-regular functions.
problem Bounding approximation error for ReLU networks on low-regular functions.
method Complexity analysis of Fourier features residual networks to ReLU networks.
result Approximation error bound proportional to target function norm and inversely proportional to network width and depth.
SGD-trained deep nets have bounds on their generalization error.
problem Bounding generalization error for deep neural networks trained by SGD.
method Combining dynamical control of parameter norms and Rademacher complexity estimates.
result Explicit bounds depend on loss trajectory, work for various architectures.