arXiv research

Variational Bayes (VB) has become a widely-used tool for Bayesian inference in statistics and machine learning. Nonetheless, the development of the existing VB algorithms is so far generally restricted to the case where the variational parameter space is Euclidean, which hinders the potential broad application of VB me…

Improved VB algorithm for high-dimensional logistic regression with theoretical guarantees.

Variational Bayes (VB) inference is one of the most important algorithms in machine learning and widely used in engineering and industry. However, VB is known to suffer from the problem of local optima. In this Letter, we generalize VB by using quantum mechanics, and propose a new algorithm, which we call quantum annea…

Paper generalizes VB-FTRL for online learning of quantum states with logarithmic loss.

Quantum annealing improves VB inference, avoiding local minima.

A new algorithm improves topic model accuracy in small data sets.

Improved VB algorithm for NIG mixtures outperforms Gaussian mixtures for non-Gaussian data.

Variational Bayes (VB), also known as independent mean-field approximation, has become a popular method for Bayesian network inference in recent years. Its application is vast, e.g. in neural network, compressed sensing, clustering, etc. to name just a few. In this paper, the independence constraint in VB will be relax…

The EM training algorithm of the classical i-vector extractor is often incorrectly described as a maximum-likelihood method. The i-vector model is however intractable: the likelihood itself and the hidden-variable posteriors needed for the EM algorithm cannot be computed in closed form. We show here that the classical …

Paper proposes a VB method for TS-SBP mixture models with reduced computational cost.

The paper categorifies Lie and Courant algebroids, establishing correspondences and new constructions.

ALBU improves LDA performance on small datasets.

VB uses natural gradients in information geometry.

Let $VB_n$, resp. $WB_n$ denote the virtual, resp. welded, braid group on $n$ strands. We study their commutator subgroups $VB_n' = [VB_n, VB_n]$ and, $WB_n' = [WB_n, WB_n]$ respectively. We obtain a set of generators and defining relations for these commutator subgroups. In particular, we prove that $VB_n'$ is finitel…

Two novel distributed VB algorithms improve Bayesian inference in sensor networks.

Paper quantifies uncertainties in EIS spectra of SOFCs, proposing VB method for online monitoring.

Variational Bayes (VB) methods have emerged as a fast and computationally-efficient alternative to Markov chain Monte Carlo (MCMC) methods for scalable Bayesian estimation of mixed multinomial logit (MMNL) models. It has been established that VB is substantially faster than MCMC at practically no compromises in predict…

Variational Bayes (VB) is a scalable alternative to Markov chain Monte Carlo (MCMC) for Bayesian posterior inference. Though popular, VB comes with few theoretical guarantees, most of which focus on well-specified models. However, models are rarely well-specified in practice. In this work, we study VB under model missp…

This thesis studies deformations of VB-algebroids and VB-groupoids in Lie algebroid and groupoid categories.

A practical guide to Variational Bayes methods.

Introduces VB-structures for geometric objects on manifolds.

A key challenge for modern Bayesian statistics is how to perform scalable inference of posterior distributions. To address this challenge, variational Bayes (VB) methods have emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) methods. VB methods tend to be faster while achieving comparabl…

VB-groupoids can be thought of as vector bundle objects in the category of Lie groupoids. Just as Lie algebroids are the infinitesimal counterparts of Lie groupoids, VB-algebroids correspond to the infinitesimal version of VB-groupoids. In this work we address the problem of the existence of a VB-groupoid admitting a g…

A VB method for high-dimensional regression with student-t priors achieves nearly optimal performance and computational efficiency.

Stochastic VB improves nonlinear model inference speed and accuracy.

We study vector bundles over Lie groupoids, known as VB-groupoids, and their induced geometric objects over differentiable stacks. We establish a fundamental theorem that characterizes VB-Morita maps in terms of fiber and basic data, and use it to prove the Morita invariance of VB-cohomology, with implications to defor…

We introduce multiplicative differential forms on Lie groupoids with values in VB-groupoids. Our main result gives a complete description of these objects in terms of infinitesimal data. By considering split VB-groupoids, we are able to present a Lie theory for differential forms on Lie groupoids with values in 2-term …

We introduce the Variational Holder (VH) bound as an alternative to Variational Bayes (VB) for approximate Bayesian inference. Unlike VB which typically involves maximization of a non-convex lower bound with respect to the variational parameters, the VH bound involves minimization of a convex upper bound to the intract…

VB approach for dynamic network models improves efficiency and accuracy.

Variational Bayes (VB) is a recent approximate method for Bayesian inference. It has the merit of being a fast and scalable alternative to Markov Chain Monte Carlo (MCMC) but its approximation error is often unknown. In this paper, we derive the approximation error of VB in terms of mean, mode, variance, predictive den…

Paper defines PB-groupoids and their relation to VB-groupoids.

A VB-algebroid is a vector bundle object in the category of Lie algebroids. We attach to every VB-algebroid a differential graded Lie algebra and we show that it controls deformations of the VB-algebroid structure. Several examples and applications are discussed. This is the first in a series of papers devoted to defor…

Combines ADMM and VB for federated learning.

Robust VB framework for large datasets with outliers.

VB-groupoids and algebroids are vector bundle objects in the categories of Lie groupoids and Lie algebroids respectively, and they are related via the Lie functor. VB-groupoids and algebroids play a prominent role in Poisson and related geometries. Additionally, they can be seen as models for vector bundles over singul…

We define the virtual bridge number $vb(K)$ and the virtual unknotting number $vu(K)$ invariants for virtual knots. For ordinary knots $K$ they are closely related to the bridge number $b(K)$ and the unknotting number $u(K)$ and we have $vu(K)\leq u(K), vb(K)\leq b(K).$ There are no ordinary knots $K$ with $b(K)=1.$ We…

Quantum method speeds up VB estimation in machine learning.

In this paper, I introduce weak representations of a Lie groupoid $G$. I also show that there is an equivalence of categories between the categories of 2-term representations up to homotopy and weak representations of $G$. Furthermore, I show that any VB-groupoid is isomorphic to an action groupoid associated to a weak…

Defines duals of higher vector bundles for Lie 2-groupoids.

For $n \geq 2$ we describe an $O(l^3n)$-time algorithm that determines if a length $l$ virtual braid word in the standard presentation of the virtual braid group ${\mathcal VB}_n$ represents the trivial virtual braid.

We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived for the mean-field VB approximation, implying that it converges to the sparse tr…

A VB-algebroid is essentially defined as a Lie algebroid object in the category of vector bundles. There is a one-to-one correspondence between VB-algebroids and certain flat Lie algebroid superconnections, up to a natural notion of equivalence. In this setting, we are able to construct characteristic classes, which in…

A VB-groupoid is a Lie groupoid equipped with a compatible linear structure. In this paper, we describe a correspondence, up to isomorphism, between VB-groupoids and 2-term representations up to homotopy of Lie groupoids. Under this correspondence, the tangent bundle of a Lie groupoid G corresponds to the "adjoint repr…

This paper shows the equivalence of the categories of $N$-manifolds of degree $2$ with the category of double vector bundles endowed with a linear metric. Split Poisson $N$-manifolds of degree $2$ are shown to be equivalent to self-dual representations up to homotopy. As a consequence, the equivalence above induces an …

Identifies LA-groups via VB-group structure and complementary actions.

There is a well known injective homomorphism $φ:{\mathcal {B}}_n \rightarrow {\rm Aut}(F_n)$ from the classical braid group ${\mathcal {B}}_n$ into the automorphism group of the free group $F_n$, first described by Artin. This homomorphism induces an action of ${\mathcal {B}}_n$ on $F_n$ that can be recovered by consid…

We study VB-groupoids and VB-algebroids, which are vector bundles in the realm of Lie groupoids and Lie algebroids. Through a suitable reformulation of their definitions, we elucidate the Lie theory relating these objects, i.e., their relation via differentiation and integration. We also show how to extend our techniqu…

Black-box alpha (BB-$α$) is a new approximate inference method based on the minimization of $α$-divergences. BB-$α$ scales to large datasets because it can be implemented using stochastic gradient descent. BB-$α$ can be applied to complex probabilistic models with little effort since it only requires as input the likel…