arXiv research

The paper improves Gaussian process regression by optimizing hyperparameters.

VBphenoR uses variational Bayes for EHR-based patient phenotyping.

A new optimization algorithm for Gaussian Variational Inference on precision matrices.

Parameter estimation for model-based clustering using a finite mixture of normal inverse Gaussian (NIG) distributions is achieved through variational Bayes approximations. Univariate NIG mixtures and multivariate NIG mixtures are considered. The use of variational Bayes approximations here is a substantial departure fr…

A new method improves likelihood-free Bayesian inference by transforming summary statistics and using efficient Variational Bayes.

Variational Gaussian Processes solve linear inverse problems efficiently.

Variational Bayes (VB) inference is one of the most important algorithms in machine learning and widely used in engineering and industry. However, VB is known to suffer from the problem of local optima. In this Letter, we generalize VB by using quantum mechanics, and propose a new algorithm, which we call quantum annea…

Paper proposes a method to estimate total variation distance for synthetic data fidelity.

Paper proposes a VB method for TS-SBP mixture models with reduced computational cost.

We propose a family of variational approximations to Bayesian posterior distributions, called $α$-VB, with provable statistical guarantees. The standard variational approximation is a special case of $α$-VB with $α=1$. When $α\in(0,1]$, a novel class of variational inequalities are developed for linking the Bayes risk …

Bayesian surrogate models reduce uncertainty in high-dimensional design optimisation problems.

Paper uses optimal transport for Bayesian filtering, deriving new EnKF and FPF formulations.

The article addresses a long-standing open problem on the justification of using variational Bayes methods for parameter estimation. We provide general conditions for obtaining optimal risk bounds for point estimates acquired from mean-field variational Bayesian inference. The conditions pertain to the existence of cer…

Fast variational Bayes methods improve geospatial data analysis speed and accuracy.

Combines ADMM and VB for federated learning.

We extend Stochastic Gradient Variational Bayes to perform posterior inference for the weights of Stick-Breaking processes. This development allows us to define a Stick-Breaking Variational Autoencoder (SB-VAE), a Bayesian nonparametric version of the variational autoencoder that has a latent representation with stocha…

The standard state-of-the-art backend for text-independent speaker recognizers that use i-vectors or x-vectors, is Gaussian PLDA (G-PLDA), assisted by a Gaussianization step involving length normalization. G-PLDA can be trained with both generative or discriminative methods. It has long been known that heavy-tailed PLD…

In Bayesian machine learning, the posterior distribution is typically computationally intractable, hence variational inference is often required. In this approach, an evidence lower bound on the log likelihood of data is maximized during training. Variational Autoencoders (VAE) are one important example where variation…

Variational Bayes (VB) is a recent approximate method for Bayesian inference. It has the merit of being a fast and scalable alternative to Markov Chain Monte Carlo (MCMC) but its approximation error is often unknown. In this paper, we derive the approximation error of VB in terms of mean, mode, variance, predictive den…

State-space models have been successfully used for more than fifty years in different areas of science and engineering. We present a procedure for efficient variational Bayesian learning of nonlinear state-space models based on sparse Gaussian processes. The result of learning is a tractable posterior over nonlinear dy…

Robust VB framework for large datasets with outliers.

A scalable algorithm approximates Bayesian posteriors in RKHS with improved efficiency.

New method handles unknown task boundaries in continual learning.

Paper introduces VBG for Bayesian causal structure and mechanism learning.

Empirical Bayes rates via variational approximations and prior decomposition.

Proposes a model combining graph networks and variational Bayes for graph data.

A software library for constructing and learning probabilistic models is presented. The library offers a set of building blocks from which a large variety of static and dynamic models can be built. These include hierarchical models for variances of other variables and many nonlinear models. The underlying variational B…

We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a general framework for estimating the evidence lower bound (ELBO) in Variational Bayes. In this work, …

Adaptive variational Bayes framework improves inference adaptively.

The variational autoencoder (VAE) is a generative model with continuous latent variables where a pair of probabilistic encoder (bottom-up) and decoder (top-down) is jointly learned by stochastic gradient variational Bayes. We first elaborate Gaussian VAE, approximating the local covariance matrix of the decoder as an o…

Sharp inequality between TV and Hellinger distances for Gaussian mixtures.

Mean field variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, it is well known that a major failing of MFVB is that it underestimates the uncertainty of model variables (sometimes severely) and provides no information about model variable cova…

VAR-GPs solve continual learning by updating posteriors sequentially.

While stochastic variational inference is relatively well known for scaling inference in Bayesian probabilistic models, related methods also offer ways to circumnavigate the approximation of analytically intractable expectations. The key challenge in either setting is controlling the variance of gradient estimates: rec…

TM-VI uses flexible transformation models to approximate complex posteriors in Bayesian models.

Simplified Variational Bayes for easier inference.

A new method combines Laplace and Variational Bayes for scalable inference.

Unified empirical and variational Bayes for unnormalized densities.

A practical guide to Variational Bayes methods.

This paper presents studies on a deterministic annealing algorithm based on quantum annealing for variational Bayes (QAVB) inference, which can be seen as an extension of the simulated annealing for variational Bayes (SAVB) inference. QAVB is as easy as SAVB to implement. Experiments revealed QAVB finds a better local …

Clarifies EM algorithm and variational Bayesian inference concepts.

New clustering algorithm for time series data using RNN and variational Bayes.

We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the non-parametric flexibility of Gaussian processes. This model accommodates input dependent signal and noise correlations between multiple response variables,…

Flexible Bayesian approach for generalized linear models, especially for sparse logistic regression.

Variational Bayes (VB) has become a widely-used tool for Bayesian inference in statistics and machine learning. Nonetheless, the development of the existing VB algorithms is so far generally restricted to the case where the variational parameter space is Euclidean, which hinders the potential broad application of VB me…

A framework uses variational Bayes for solving inverse problems efficiently.

The study calculates the risk of semi-supervised multitask learning on Gaussian mixtures.

Unified approach to Bayesian inference with guarantees on covariance matrices.