In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the Chi-square divergence and the expectation value of the estimator. By using the relation b…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “divergence bounds”
New bounds for Neyman-Pearson region using -divergences.
problem Bounding the Neyman-Pearson region for hypothesis testing.
method Establishing novel lower and upper bounds using -divergences.
result Best possible lower bound for the Neyman-Pearson boundary using hockey-stick -divergences.
Improved bounds for estimating discrete distributions in KL divergence.
problem Estimating discrete distributions in KL divergence with accuracy.
method Used Laplace estimator and established concentration bounds.
result Deviation from mean scales as for .
Technical report on f-divergences and f-GAN training properties.
problem Understanding and optimizing f-divergences for GAN training.
method Elementary derivation and detailed expressions of f-divergences and their variational lower bounds.
result Informative properties of f-divergences and f-GAN training, including gradient matching and stability improvements.
Innovative inequalities for divergences with applications in PAC-Bayesian bounds and Monte Carlo.
problem Developing new inequalities for divergences.
method Introducing novel change of measure inequalities for -divergences and -divergences.
result Applications in PAC-Bayesian bounds and Monte Carlo estimates.
New PAC-Bayesian bounds for multi-view learning using Rényi divergence.
problem Applying PAC-Bayesian theory to multi-view learning.
method Introducing novel PAC-Bayesian bounds based on Rényi divergence for multi-view learning.
result Efficient optimization algorithms that align with theoretical bounds.
Study compares chi-squared divergence and KL-divergence posteriors for PAC-Bayesian bounds.
problem Investigates optimal posteriors for PAC-Bayesian bounds using chi-squared divergence.
method Analyzes bounds for three distance functions, derives FP equations for computation.
result Chi-squared divergence based posteriors have weaker bounds and worse test errors.
We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…
Unified framework for deriving generalization bounds in supervised learning.
problem Generalization error bounds in supervised learning.
method Data Processing Inequality PAC-Bayesian framework.
result Unified bounds on binary Kullback-Leibler generalization gap for various divergences.
The paper improves generalization bounds using interpolation between various divergences.
problem Improving generalization bounds in machine learning.
method Derives new PAC-Bayes generalization bounds based on -divergence and interpolates between various divergences.
result Connects derived bounds to earlier statistical learning results and provides practical training objectives.
Tight Bounds on the Binomial CDF, and the Minimum of i.i.d Binomials, in terms of KL-Divergencemath.PR
The study tightens bounds on binomial probabilities and minimums using KL-divergence.
problem Tightening bounds on binomial probabilities and minimums of i.i.d. Binomials.
method Applied Sanov's theorem to derive upper and lower bounds on binomial tail probabilities and minimums, expressed in terms of KL-divergence.
result High probability upper and lower bounds on the minimum of i.i.d. Binomial random variables, finite sample, asymptotically tight.
Locally private mechanisms' output divergence bounds derived.
problem Bounding divergence between locally private mechanisms' outputs.
method Sharp upper bounds on divergence between input and output distributions.
result Established locally private versions of estimation risk bounds.
Upper bound found for divergence-free Killing 2-tensors on manifolds.
problem Bounding the space of divergence-free symmetric Killing 2-tensors.
method Witten deformation and Morse function analysis.
result Explicit calculation of dimension for .
We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …
Study improves density estimation for compact domains using -lifted KL divergence.
problem Estimating probability density functions on compact domains.
method Introduced -lifted Kullback--Leibler (KL) divergence for risk minimization.
result Proved bound on estimation error.
Study on divergence and thickness for Coxeter groups, generalizing previous work.
problem Characterizing and bounding divergence and thickness for Coxeter groups.
method Characterization of linear divergence, introduction of hypergraph index, new construction of Coxeter systems.
result Upper bounds on divergence and thickness for Coxeter groups, conjectured to be equalities.
Flow matching KL divergence bound derived for smooth distributions.
problem Estimating smooth distributions efficiently.
method Deterministic upper bound on KL divergence derived from flow-matching loss.
result Flow matching achieves nearly minimax-optimal efficiency under TV distance.
Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we view BBVI with generalized divergences as a form of estimating the marginal likelihood via biased import…
Information-theoretic measures such as the entropy, cross-entropy and the Kullback-Leibler divergence between two mixture models is a core primitive in many signal processing tasks. Since the Kullback-Leibler divergence of mixtures provably does not admit a closed-form formula, it is in practice either estimated using …
New framework using Jensen-Shannon divergence improves domain adaptation theory.
problem Incoherence between empirical domain adversarial training and theoretical -divergence.
method Established new theoretical framework based on Jensen-Shannon divergence, derived bi-directional upper bounds.
result Framework exhibits flexibilities for various transfer learning problems.
Tail-Sensitive KL and Rényi Convergence of Unadjusted Hamiltonian Monte Carlo via One-Shot Couplingsstat.ML
This work improves convergence guarantees for unadjusted HMC in KL and Rényi divergences.
problem Understanding convergence properties of unadjusted HMC in divergences like KL and Rényi.
method One-shot couplings to establish regularization and lift convergence bounds.
result Quantitative control of relative density mismatch and warm-start requirements.
Stochastic variational inference (SVI) plays a key role in Bayesian deep learning. Recently various divergences have been proposed to design the surrogate loss for variational inference. We present a simple upper bound of the evidence as the surrogate loss. This evidence upper bound (EUBO) equals to the log marginal li…
New aggregation strategy handles unbounded losses with regret bounds.
problem Online optimization with unbounded loss functions.
method Follow The Regularized Leader (FTRL) with φ-divergence.
result Worst regret bound for unbounded losses with alternative divergences.
Beta Diffusioncs.LG
Beta diffusion generates bounded data using multiplicative transitions.
problem Generating data within specific ranges.
method Integrates demasking and denoising with scaled and shifted beta distributions.
result KLUBs are more effective for optimizing beta diffusion compared to negative ELBOs.
Fundamental tone estimates for elliptic operators in divergence form and geometric applicationsmath.DG
We establish a method for giving lower bounds for the fundamental tone of elliptic operators in divergence form in terms of the divergence of vector fields. We then apply this method to the operator associated to immersed hypersurfaces with locally bounded -th mean curvature of the space forms …
Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main cont…
Neural networks estimate statistical divergences with performance guarantees.
problem Estimating statistical divergences with theoretical performance guarantees.
method Parametrizing empirical variational form by a neural network and optimizing over parameter space.
result Established non-asymptotic absolute error bounds for neural estimators of four -divergences.
PAC-Bayesian learning bounds are of the utmost interest to the learning community. Their role is to connect the generalization ability of an aggregation distribution to its empirical risk and to its Kullback-Leibler divergence with respect to some prior distribution . Unfortunately, most of the available bounds …
In this paper we explore relationships between divergence and thick groups, and with the same techniques we estimate lengths of shortest conjugators. We produce examples, for every positive integer n, of CAT(0) groups which are thick of order n and with polynomial divergence of order n+1, both these phenomena are new. …
Towards a Sharp Analysis of Offline Policy Learning for -Divergence-Regularized Contextual Banditscs.LG
Paper analyzes sample complexity for offline -divergence-regularized contextual bandits.
problem Lack of tight analyses for sample complexity in offline reinforcement learning.
method Novel pessimism-based analysis for reverse KL divergence, establishing sample complexity.
result Achieves sample complexity for reverse KL divergence, surpassing existing bounds.
Rank-statistic method approximates -divergences without density-ratio estimation.
problem Approximating -divergences without explicit density-ratio estimation.
method Mapping distribution rank histograms to discrete -divergence and averaging over random projections.
result The rank-statistic estimator is a lower bound of the true -divergence and converges under mild conditions.
DM framework improves robustness and efficiency in latent-mixture models.
problem Efficient and robust inference in latent-mixture models.
method Divergence-minimization framework with monotonic convergence and robustness guarantees.
result DM yields consistent and asymptotically normal estimators under correct specification.
Unified framework for high-dimensional online learning with non-divergent error bounds and adaptive gains.
problem Divergence of error bounds in high-dimensional online learning as data batches increase.
method Asynchronous decomposition framework with summary statistics and dynamic regularization.
result Non-divergent error bounds and adaptive gains in sparse online optimization.
Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the sa…
We speed up marginal inference by ignoring factors that do not significantly contribute to overall accuracy. In order to pick a suitable subset of factors to ignore, we propose three schemes: minimizing the number of model factors under a bound on the KL divergence between pruned and full models; minimizing the KL dive…
New bounds for estimating partition functions under bounded f-divergence.
problem Estimating partition functions with limited sample access.
method Information-theoretic characterization using integrated coverage profile and -divergences.
result Sharp phase transitions in sample complexity under -divergences.
New method proves dimension-free convergence for ULD in KL divergence.
problem Polynomial scaling of existing convergence guarantees in high dimensions.
method Refined KL local error framework, focusing on tr(H) instead of d.
result First dimension-free KL divergence bounds for discretized ULD.
TRM improves long-horizon LLM RL by masking divergent sequences.
problem Long-horizon reinforcement learning with LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
problem Long-horizon reinforcement learning for LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
The paper provides estimates for eigenvalues of elliptic differential problems.
problem Computing eigenvalue estimates for elliptic differential problems.
method Analytical computation of eigenvalues for specific types of elliptic differential equations.
result Universal estimates of eigenvalues and gaps between consecutive eigenvalues are derived.
We establish bounds on the KL divergence between two multivariate Gaussian distributions in terms of the Hamming distance between the edge sets of the corresponding graphical models. We show that the KL divergence is bounded below by a constant when the graphs differ by at least one edge; this is essentially the tighte…
Paper improves PAC-Bayes bounds using a better-than-KL divergence.
problem Estimating the generalization error of stochastic algorithms.
method Developed new PAC-Bayes bounds with a novel divergence.
result Achieved strictly tighter bounds than the KL divergence.
Sharp bounds for high-probability estimation of discrete distributions.
problem Estimating discrete distributions with high probability under -divergence.
method Sharp upper and lower bounds for the classical Laplace estimator, and characterization of minimax high-probability risk for any estimator.
result Sharp bounds for high-probability estimation of discrete distributions can be achieved through a simple smoothing strategy.
Paper studies regularized KKL divergence for distributions with disjoint supports.
problem Inability of original KKL divergence to handle distributions with disjoint supports.
method Proposes a regularized variant of KKL divergence, derives bounds, and provides closed-form expression.
result Regularized KKL divergence is well-defined for all distributions and has finite-sample bounds.
The study quantifies geodesic divergence on Riemannian planes with bounded geometry.
problem Understanding geodesic divergence on Riemannian planes with specific geometric constraints.
method Recalling quasi-redirection and using it to quantify geodesic divergence, compactifying Riemannian planes into or .
result Necessary and sufficient conditions for the quasi-redirecting compactification being are derived in terms of asymptotic cones.
Unified technique for sequential estimation of convex divergences.
problem Estimating convex divergences between distributions.
method Martingale methods and maximal inequalities for reverse submartingales.
result Valid time-uniform confidence sequences for arbitrary stopping times.
This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…
New PAC-Bayes bounds derived using Legendre transform and f-divergences.
problem Deriving PAC-Bayes bounds under various assumptions.
method Combining Legendre transform and Fenchel--Young inequality to derive change-of-measure inequalities.
result Extended PAC-Bayesian guarantees under tailored assumptions.