arXiv research

Improved Bayesian inference using power priors with historical data.

Characterizes connections on multivariate normal distributions.

The paper evaluates biased methods for alpha-divergence minimization.

This work presents a parametrized family of divergences, namely Alpha-Beta Log- Determinant (Log-Det) divergences, between positive definite unitized trace class operators on a Hilbert space. This is a generalization of the Alpha-Beta Log-Determinant divergences between symmetric, positive definite matrices to the infi…

Characterizes connections on normal distributions manifold.

This work extends alpha-beta divergences to complex data and finds closed-form solutions.

The paper presents methods to improve uncertainty calibration in Bayesian Neural Networks.

We describe the underlying probabilistic interpretation of alpha and beta divergences. We first show that beta divergences are inherently tied to Tweedie distributions, a particular type of exponential family, known as exponential dispersion models. Starting from the variance function of a Tweedie model, we outline how…

Unbiased methods for alpha-divergence minimization struggle in high dimensions.

AlphaNet improves supernets training with alpha-divergence.

New insights into Markov chain geometry via positive transition measures.

We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …

Quantum connections replace metrics with operator inner products.

On the predual of a von Neumann algebra, we define a differentiable manifold structure and affine connections by embeddings into non-commutative L_p-spaces. Using the geometry of uniformly convex Banach spaces and duality of the L_p and L_q spaces for 1/p+1/q=1, we show that we can introduce the α-divergence, for αin (…

This paper introduces the variational Rényi bound (VR) that extends traditional variational inference to Rényi's alpha-divergences. This new family of variational methods unifies a number of existing approaches, and enables a smooth interpolation from the evidence lower-bound to the log (marginal) likelihood that is co…

The paper explores how information geometry impacts classical CR inequalities.

New $\alpha$-divergence loss function improves neural density ratio estimation.

We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two…

The paper develops divergences for Gaussian processes and RKHS settings.

To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed. Dropout variational inference (VI) for example has been used for machine vision and medical applications, but VI can severely underestimates model uncertainty. Alpha-divergences are alternative …

A method to compute divergences between decomposable models, useful in supervised learning.

This paper introduces a variational approximation framework using direct optimization of what is known as the {\it scale invariant Alpha-Beta divergence} (sAB divergence). This new objective encompasses most variational objectives that use the Kullback-Leibler, the R{é}nyi or the gamma divergences. It also gives access…

In Riemannian geometry geodesics are integral curves of the Riemannian distance gradient. We extend this classical result to the framework of Information Geometry. In particular, we prove that the rays of level-sets defined by a pseudo-distance are generated by the sum of two tangent vectors. By relying on these vector…

Paper studies statistical manifolds with logarithmic divergences.

A study on $\alpha$-GANs proving convergence and estimation guarantees.

Paper formalizes and analyzes a new bound for variational inference.

Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences. In this paper, we view BBVI with generalized divergences as a form of estimating the marginal likelihood via biased import…

In this paper, we introduce new classes of divergences by extending the definitions of the Bregman divergence and the skew Jensen divergence. These new divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy some properties similar to the Bregman or skew Jensen divergence. We show these g-diverge…

AES uses α-divergence to select informative points for BO, improving optimization performance.

This paper studies geometrical structure of the manifold of escort probability distributions and shows its new applicability to information science. In order to realize escort probabilities we use a conformal transformation that flattens so-called alpha-geometry of the space of discrete probability distributions, which…

Study of generalized Csiszár divergences and their application to Cramér-Rao bounds.

We study the family of $α$-connections of Amari-Chentsov on the homogeneous space $\mathcal{D}(M)/\mathcal{D}_μ(M)$ of diffeomorphisms modulo volume-preserving diffeomorphims of a compact manifold $M$. We show that in some cases their geodesic equations yield completely integrable Hamiltonian systems.

Defines vector Laplacian on statistical manifolds.

We consider the nonlinear Kalman filtering problem using Kullback-Leibler (KL) and $α$-divergence measures as optimization criteria. Unlike linear Kalman filters, nonlinear Kalman filters do not have closed form Gaussian posteriors because of a lack of conjugacy due to the nonlinearity in the likelihood. In this paper …

The Conant-Ashby theorem is verified for hypergraph observers, leading to unique learning rules.

The paper introduces natural α-embeddings for item representations.

Study curvature and torsion in Gaussian distribution's dual coordinate system.

Dual affine connections on Riemannian manifolds have played a central role in the field of information geometry since their introduction by Amari. Here I would like to extend the notion of dual connections to general vector bundles with an inner product, in the same way as a unitary connection generalizes a metric affi…

Optimized $\alpha$-posteriors reduce KL divergence from true posterior in parametric misspecification.

We develop a family of infinite-dimensional (non-parametric) manifolds of probability measures. The latter are defined on underlying Banach spaces, and have densities of class $C_b^k$ with respect to appropriate reference measures. The case $k=\infty$, in which the manifolds are modelled on Fréchet spaces, is included.…

Black-box alpha (BB-$α$) is a new approximate inference method based on the minimization of $α$-divergences. BB-$α$ scales to large datasets because it can be implemented using stochastic gradient descent. BB-$α$ can be applied to complex probabilistic models with little effort since it only requires as input the likel…

A new Weyl prior is proposed for Bayesian statistics, offering a more canonical choice for parameter α.

EGAB algorithms improve online portfolio selection.

Python package for SPD matrix distances, reproducible and extensible.

Alpha2 discovers logical formulaic alphas using deep reinforcement learning.

The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence measures. Surprisingly, an equally important problem, estimating an unknown Markov ch…

Alpha-GPT mines new trading signals with human-AI interaction.

We review basic notions in the field of information geometry such as Fisher metric on statistical manifold, $α$-connection and corresponding curvature following Amari's work . We show application of information geometry to asymptotic statistical inference.