arXiv research

CO2 algorithm creates coresets for generic smooth divergences efficiently.

This work investigates the properties of Gaussian-smoothed sliced divergences for comparing distributions.

The divergence theorem in its usual form applies only to suitably smooth vector fields. For vector fields which are merely piecewise smooth, as is natural at a boundary between regions with different physical properties, one must patch together the divergence theorem applied separately in each region. We give an elegan…

New divergence measures improve KL approximation.

The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to generalize the concept of Lipschitz smoothness condition to the relative smoothn…

Flow matching KL divergence bound derived for smooth distributions.

New optimization method corrects data-driven optimizer's curse.

LMC algorithm converges to target in Chi-squared and Renyi divergence.

Proves Sard conjecture for specific distributions, controlling divergence of vector fields.

This manuscript develops the theory of agglomerative clustering with Bregman divergences. Geometric smoothing techniques are developed to deal with degenerate clusters. To allow for cluster models based on exponential families with overcomplete representations, Bregman divergences are developed for nondifferentiable co…

Sharp bounds for high-probability estimation of discrete distributions.

Graph Laplacians converge under symmetric divergence conditions.

We give a comprehensive theoretical characterization of a nonparametric estimator for the $L_2^2$ divergence between two continuous distributions. We first bound the rate of convergence of our estimator, showing that it is $\sqrt{n}$-consistent provided the densities are sufficiently smooth. In this smooth regime, we t…

Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential inequality convergence bound derived for any divergence estimators. The main cont…

Paper proposes an algorithm for sampling from complex mixture distributions without requiring smoothness.

New proof of Willmore inequality using geometric divergence inequality.

Reformulates divergence map for Turaev cobracket in non-commutative geometry.

This paper introduces a variational approximation framework using direct optimization of what is known as the {\it scale invariant Alpha-Beta divergence} (sAB divergence). This new objective encompasses most variational objectives that use the Kullback-Leibler, the R{é}nyi or the gamma divergences. It also gives access…

Model-free preference under ambiguity defined and applied.

Improved inference-time alignment using Best-of-N and smoothing.

We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise. This paper investigates the application of this stochastic approximation in training a family of Restricted Boltzmann Machines (RBM) that are closely linked to Bregman divergences.…

Confirming operator characterization on smooth manifolds.

New tools quantify deep generative models' performance.

In Riemannian geometry geodesics are integral curves of the Riemannian distance gradient. We extend this classical result to the framework of Information Geometry. In particular, we prove that the rays of level-sets defined by a pseudo-distance are generated by the sum of two tangent vectors. By relying on these vector…

This work improves convergence guarantees for unadjusted HMC in KL and Rényi divergences.

The paper is devoted to differential geometry of singular distributions (i.e., of varying dimension) on a Riemannian manifold. Such distributions are defined as images of the tangent bundle under smooth endomorphisms. We prove the novel divergence theorem with the divergence type operator and deduce the Codazzi equatio…

Introduces a new divergence measure for optimal transport.

New algorithm solves saddle point problems in Banach spaces.

Develops a contraction framework for MCMC mixing rates.

Optimizes sampling from target distributions with applications to online learning.

We consider rough metrics on smooth manifolds and corresponding Laplacians induced by such metrics. We demonstrate that globally continuous heat kernels exist and are Hölder continuous locally in space and time. This is done via local parabolic Harnack estimates for weak solutions of operators in divergence form with b…

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…

We consider nonparametric estimation of $L_2$, Renyi-$α$ and Tsallis-$α$ divergences between continuous distributions. Our approach is to construct estimators for particular integral functionals of two densities and translate them into divergence estimators. For the integral functionals, our estimators are based on cor…

New guarantees for VI in symmetric cases, extending previous results.

Improved analysis for diffusion models reduces KL divergence error dependence on data dimension and discretization step size.

New divergence identity for scalar curvature helps prove rigidity of tensors.

Efficient algorithms for large Maxent models improve wildfire probability predictions.

New measure captures differences across entire distributions of counterfactual outcomes.

Unified ML and adversarial learning via α-divergence.

Exponential families and mixture families are parametric probability models that can be geometrically studied as smooth statistical manifolds with respect to any statistical divergence like the Kullback-Leibler (KL) divergence or the Hellinger divergence. When equipping a statistical manifold with the KL divergence, th…

Extends normalizing flows to arbitrary smooth manifolds.

The paper analyzes mirror descent in measure spaces and its applications.

Parallel sampling for smooth distributions with fast convergence.

Consistency models accelerate generation with theoretical guarantees.

The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence measures. Surprisingly, an equally important problem, estimating an unknown Markov ch…

Diverging Flows detects extrapolations in flow models, ensuring reliable predictions.

Proposes NRS to find flat minima in deep neural networks.

The paper analyzes MACD using operator theory.