New framework using Jensen-Shannon divergence improves domain adaptation theory.
problem Incoherence between empirical domain adversarial training and theoretical H-divergence. method Established new theoretical framework based on Jensen-Shannon divergence, derived bi-directional upper bounds.
result Framework exhibits flexibilities for various transfer learning problems.
The paper explores how information geometry impacts classical CR inequalities.
problem Deriving and generalizing CR inequalities using information geometry.
method Examining Eguchi's theory and applying Amari-Nagoaka's theory to KL-divergence, and then extending to other divergences.
result Generalized CR inequalities derived from various divergences.
Formulae for divergent integrals with singularities, useful in string theory.
problem Handling divergent integrals with singularities.
method Formulae for finite part of divergent integrals, using local residue map.
result Formulae for finite part of divergent integrals expressed in terms of local residue map.
This paper re-examines Bregman functions and their divergences, introducing new properties and functions.
problem Exploring properties and applications of Bregman functions and divergences.
method Re-examination of existing Bregman functions and introduction of new ones, providing sufficient conditions for construction.
result Several known Bregman functions are reclassified, and new Bregman functions are introduced.
New entropies and divergences defined using group theory.
problem Defining new entropies and divergences with group-theoretical properties.
method Formal group theory and information geometry to construct new entropies and divergences.
result A method for constructing new entropies and divergences from known ones.
Warped product affects divergences in information geometry.
problem Warped product's impact on divergences in information geometry.
method Study of warped product on information geometry.
result Warped product does not preserve canonical divergences.
It is well-known that there are a number of relations between theoretical finance theory and information theory. Some of these relations are exact and some are approximate. In this paper we will explore some of these relations and determine under which conditions the relations are exact. It turns out that portfolio the…
Study random walks on groups with superlinear divergent geodesics.
problem Existence of superlinear divergent geodesics in groups.
method Developed theory of superlinear divergence and applied Gouëzel's pivoting technique.
result Established a central limit theorem for random walks on groups with superlinear divergent geodesics.
The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In particular, even for those estimators whose MSE convergence rates are known, the asympt…
New PAC-Bayesian bounds for multi-view learning using Rényi divergence.
problem Applying PAC-Bayesian theory to multi-view learning.
method Introducing novel PAC-Bayesian bounds based on Rényi divergence for multi-view learning.
result Efficient optimization algorithms that align with theoretical bounds.
New variational formula for Rényi divergences improves neural network estimation in high dimensions.
problem Estimating Rényi divergences in high-dimensional systems.
method Derive and apply a variational formula for Rényi divergences over various function spaces.
result Neural network estimators of Rényi divergences are consistent under certain conditions.
This paper provides performance guarantees for neural estimation of statistical distances.
problem Developing performance guarantees for neural estimation of statistical distances.
method Non-asymptotic error bounds using function approximation theorems and empirical process theory.
result Established a fundamental tradeoff between approximation and estimation errors in neural estimation of statistical distances.
f-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler divergence, chi-squared divergence, squared Hellinger distance, total variation distance e…
GANs can generate realistic data without minimizing a divergence, contrary to current theory.
problem Current theory suggests GANs minimize a divergence to generate realistic data.
method Discussed various loss functions for G, showing they are not divergences and do not have the same equilibrium.
result GANs can use a wide range of loss functions, not just divergences, to generate realistic data.
Linking GANs to binary classification through divergences.
problem Training GANs and understanding their divergence properties.
method Revisiting the discriminator's role in computing f-divergences.
result Alternative training perspective for f-GANs by designing discriminator loss.
The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
We use invariance theory to compute the divergence term am+2,md+δ in the super trace for the twisted de Rham complex for a closed Riemannian manifold.
The so-called great divergence in the income per capita is described in the Unified Growth Theory as the mind-boggling and unresolved mystery about the growth process. This mystery has now been solved: the great divergence never happened. It was created by the manipulation of data. Economic growth in various regions is…
New approach to QFT divergences uses curved momentum space.
problem UV divergences in quantum field theory.
method Geodesic metric in curved momentum space.
result Intrinsic suppression of high-energy divergences.
Neural networks estimate statistical divergences with performance guarantees.
problem Estimating statistical divergences with theoretical performance guarantees.
method Parametrizing empirical variational form by a neural network and optimizing over parameter space.
result Established non-asymptotic absolute error bounds for neural estimators of four f-divergences. Paper proposes f-EBM for training deep EBMs using various f-divergences.
problem Training deep EBMs with intractable partition functions.
method Introduces f-EBM framework and optimization algorithm for any f-divergence.
result f-EBM outperforms contrastive divergence and other f-divergences.
The paper explores how information theory aids in statistical learning models.
problem Characterizing fundamental performance limits in statistical learning models.
method Introduces divergence measures and evidence lower bound (ELBO) in model training.
result Provides a systematic derivation for generative diffusion models.
New theory explains GAN's high quality but low diversity.
problem Lack of theoretical justification for non-saturating GAN training.
method Showed non-saturating GAN training approximately minimizes a specific f-divergence.
result Non-saturating GAN training minimizes a particular f-divergence.
GANs can learn without always minimizing a divergence, contrary to recent theories.
problem The need for GANs to minimize a divergence at every step is overly restrictive.
method Empirical counterexamples and gradient penalties.
result GANs can learn distributions without always minimizing a specific divergence.
This manuscript develops the theory of agglomerative clustering with Bregman divergences. Geometric smoothing techniques are developed to deal with degenerate clusters. To allow for cluster models based on exponential families with overcomplete representations, Bregman divergences are developed for nondifferentiable co…
We study the divergence theorem on pseudo-Finsler spaces and obtain a completely Finslerian version for spaces having a vanishing mean Cartan torsion. This result helps to clarify the problem of energy-momentum conservation in Finsler gravity theories.
New proof finds three divergence-free vector fields for any 3D manifold.
problem Proving the existence of divergence-free vector fields on 3D manifolds.
method Using geometric properties of eigenspinors in three dimensions.
result Found three divergence-free vector fields that are orthogonal and have the same length at every point.
The paper introduces a new risk assessment framework using φ-divergence.
problem Assessing risk and decision-making in uncertain conditions.
method Introduces a novel framework called the φ-Divergence Quadrangle.
result Provides a more nuanced understanding of risk through φ-divergence.
Study of generalized Csiszár divergences and their application to Cramér-Rao bounds.
problem Deriving lower bounds for estimator variance using generalized divergences.
method Applied Eguchi's theory to derive Fisher information metric and dual affine connections.
result More widely applicable Cramér-Rao inequality for escort distributions.
The paper explores statistical and topological properties of sliced probability divergences.
problem Understanding the topological, statistical, and computational consequences of slicing divergences.
method Deriving theoretical properties of sliced probability divergences, including metric axioms preservation and weak continuity.
result Sliced divergences share similar topological properties and have stable sample complexity.
New method improves GAN by reducing KL divergence.
problem Improving stability and effectiveness of GANs.
method Theory-based approach using functional gradient learning.
result KL divergence between real and generated data converges to zero.
Study on Lp affine surface areas and their inequalities for convex bodies.
problem Understanding weighted Lp affine surface areas in convex bodies. method Investigating valuations, isoperimetric inequalities, and connections to f divergences. result Established isoperimetric inequalities for weighted Lp affine surface areas. Estimates KL divergence with fairness considerations for sub-populations.
problem Fairly estimate KL divergence between distributions considering sub-populations.
method Proposes multi-group attribution for KL divergence estimation, derived from multi-calibration.
result Shows multi-group attribution provides better KL divergence estimates conditioned on sub-populations.
New PAC-Bayes bounds derived using Legendre transform and f-divergences.
problem Deriving PAC-Bayes bounds under various assumptions.
method Combining Legendre transform and Fenchel--Young inequality to derive change-of-measure inequalities.
result Extended PAC-Bayesian guarantees under tailored assumptions.
The paper addresses instability in KL divergence estimation using a neural network discriminator.
problem Unstable estimation of KL divergence due to discriminator complexity.
method Using a Reproducing Kernel Hilbert Space (RKHS) to control discriminator complexity.
result Theoretical bound on error probability of KL estimates based on discriminator complexity in RKHS.
A new tensor decomposition method that minimizes KL divergence.
problem Tensor reconstruction accuracy.
method Legendre decomposition, based on information geometry.
result Minimizes KL divergence and improves tensor reconstruction accuracy.
AES uses α-divergence to select informative points for BO, improving optimization performance.
problem Optimizing complex functions with limited evaluations.
method AES uses α-divergence to select points based on dependency with global maximum.
result AES outperforms other information-based acquisition functions in various experiments.
Automated feature selection is important for text categorization to reduce the feature size and to speed up the learning process of classifiers. In this paper, we present a novel and efficient feature selection framework based on the Information Theory, which aims to rank the features with their discriminative capacity…
Optimal transport with f-divergence regularization using generalized Sinkhorn algorithm.
problem Optimal transport with f-divergence regularization. method Generalized Sinkhorn algorithm for solving optimal transport problems with various f-divergences. result Strong duality holds, optimums are attained, and convergence to an optimal solution is guaranteed under certain conditions.
Unified framework for data-free sampling using Wasserstein gradient flows.
problem Efficient sampling from unnormalized distributions without data.
method Unified theoretical framework based on Wasserstein gradient flows.
result Unified form of velocity field for various f-divergences.
LDP is equivalent to contraction of E_γ-divergence, impacting privacy and utility.
problem Analyzing trade-offs between privacy and utility in estimation problems.
method Equivalence of LDP constraints to contraction coefficients of E_γ-divergence, using f-divergences and estimation-theoretic tools.
result LDP guarantees can be expressed in terms of contraction coefficients of arbitrary f-divergences.
Reduced sample complexity for group-invariant distributions.
problem Improving sample complexity for estimating divergences of group-invariant distributions.
method Quantified reduction in sample complexity for Wasserstein-1 metric and Lipschitz-regularized α-divergences under finite and infinite groups.
result Sample complexity reduction proportional to group size for finite groups, and convergence rate depends on intrinsic dimension for infinite groups.
New insights into Markov chain geometry via positive transition measures.
problem Lack of statistical meaning in the space of transition probabilities.
method Constructing an extension of the space of transition probabilities using Amari's theory of positive measures.
result Introduction of a new dually flat structure for the space of positive transition measures.
New framework for logarithmically divergent integrals on manifolds with corners.
problem Logarithmically divergent integrals on manifolds with corners.
method Introduces new geometric framework and morphisms in logarithmic geometry.
result Functorial characterization of regularized integration.
There has been a growing interest in mutual information measures due to their wide range of applications in Machine Learning and Computer Vision. In this paper, we present a generalized structured regression framework based on Shama-Mittal divergence, a relative entropy measure, which is introduced to the Machine Learn…
A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…
New variational bounds improve posterior covariances and likelihoods.
problem Improving variational inference with different divergence measures.
method Applying variational perturbation theory to construct new variational bounds.
result New variational bounds lead to more accurate posterior covariances and higher likelihoods.
Gauss-Green theorem proven for vector fields in stratified groups.
problem Establishing the Gauss-Green theorem for vector fields in noncommutative stratified Lie groups.
method Developed a new family of function spaces for divergence-measure fields and proved the Gauss-Green theorem.
result Gauss-Green theorem achieved for vector fields of low regularity on sets of finite perimeter in stratified groups.