arXiv research

In this paper, we introduce new classes of divergences by extending the definitions of the Bregman divergence and the skew Jensen divergence. These new divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy some properties similar to the Bregman or skew Jensen divergence. We show these g-diverge…

Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be …

Study explores relationship between Hölder and FDPD divergences.

Unified representation of density-power-based divergences simplifies estimation to M-estimation.

This paper improves active learning by using robust divergences for committee disagreement.

New divergence measures improve KL approximation.

The paper improves semi-supervised learning using $f$-divergences and $α$-Rényi divergences.

$f$-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler divergence, chi-squared divergence, squared Hellinger distance, total variation distance e…

We introduce a new quasi-isometry invariant, called the divergence spectrum, to study finitely generated groups. We compare the concept of divergence spectrum with the other classical notions of divergence and we examine the divergence spectra of relatively hyperbolic groups. We show the existence of an infinite collec…

We study the logarithmic $L^{(α)}$-divergence which extrapolates the Bregman divergence and corresponds to solutions to novel optimal transport problems. We show that this logarithmic divergence is equivalent to a conformal transformation of the Bregman divergence, and, via an explicit affine immersion, is equivalent t…

The study defines divergence for multivector fields on infinite-dimensional manifolds.

Technical report on f-divergences and f-GAN training properties.

The paper evaluates biased methods for alpha-divergence minimization.

This work presents a parametrized family of divergences, namely Alpha-Beta Log- Determinant (Log-Det) divergences, between positive definite unitized trace class operators on a Hilbert space. This is a generalization of the Alpha-Beta Log-Determinant divergences between symmetric, positive definite matrices to the infi…

Develops a new divergence framework that combines $f$-divergences and IPMs.

Study compares statistical properties and power of divergence measures for credit risk monitoring.

New $\alpha$-divergence loss function improves neural density ratio estimation.

Study on geometric Jensen-Shannon divergence for Gaussian measures in Hilbert space.

The paper explores how information geometry impacts classical CR inequalities.

We extend CS divergence to conditional distributions and show its advantages in time series data and sequential decision making.

Proposes practical kernel tests for $f$-divergences with theoretical guarantees.

Rényi divergence is related to Rényi entropy much like Kullback-Leibler divergence is related to Shannon's entropy, and comes up in many settings. It was introduced by Rényi as a measure of information that satisfies almost the same axioms as Kullback-Leibler divergence, and depends on a parameter that is called its or…

New optimal transport divergences derived from scoring functions.

Classifies divergence and thickness in right-angled Coxeter groups.

The paper explores statistical and topological properties of sliced probability divergences.

New framework using Jensen-Shannon divergence improves domain adaptation theory.

This work extends alpha-beta divergences to complex data and finds closed-form solutions.

Study random walks on groups with superlinear divergent geodesics.

We describe the underlying probabilistic interpretation of alpha and beta divergences. We first show that beta divergences are inherently tied to Tweedie distributions, a particular type of exponential family, known as exponential dispersion models. Starting from the variance function of a Tweedie model, we outline how…

Deep energy-based models (EBMs) are very flexible in distribution parametrization but computationally challenging because of the intractable partition function. They are typically trained via maximum likelihood, using contrastive divergence to approximate the gradient of the KL divergence between data and model distrib…

Regularizes $f$-divergences with MMD to analyze Wasserstein flows.

The paper introduces a new risk assessment framework using φ-divergence.

In the field of statistics, many kind of divergence functions have been studied as an amount which measures the discrepancy between two probability distributions. In the differential geometrical approach in statistics (information geometry), dually flat spaces play a key role. In a dually flat space, there exist dual a…

New KL-divergence for Gaussian distributions based on Wasserstein geometry.

New divergences help audit DP in high dimensions.

Estimates KL divergence with fairness considerations for sub-populations.

For distributions $\mathbb{P}$ and $\mathbb{Q}$ with different supports or undefined densities, the divergence $\textrm{D}(\mathbb{P}||\mathbb{Q})$ may not exist. We define a Spread Divergence $\tilde{\textrm{D}}(\mathbb{P}||\mathbb{Q})$ on modified $\mathbb{P}$ and $\mathbb{Q}$ and describe sufficient conditions for t…

We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…

Disagreement between two classifiers regarding the class membership of an observation in pattern recognition can be indicative of an anomaly and its nuance. As in general classifiers base their decision on class aposteriori probabilities, the most natural approach to detecting classifier incongruence is to use divergen…

We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …

Replacing MSE with f-divergence in diffusion models improves robustness under data contamination.

This paper introduces $f$-DPO, a generalized approach to Direct Preference Optimization using diverse divergence constraints.

Paper introduces f-divergence variational inference for broader application.

Study on divergence and thickness for Coxeter groups, generalizing previous work.

New $f$-divergence measures improve robustness in noisy label learning.

Parametric adversarial divergences, which are a generalization of the losses used to train generative adversarial networks (GANs), have often been described as being approximations of their nonparametric counterparts, such as the Jensen-Shannon divergence, which can be derived under the so-called optimal discriminator …

Unified deep metric learning approach using neural networks.

Optimal transport with $f$-divergence regularization using generalized Sinkhorn algorithm.