arXiv research

Ad-SVGD optimizes kernel parameters for SVGD, improving inference performance.

Kernel SVGD improves high-dimensional inference with noise adaptation.

Stein variational gradient descent (SVGD) is a particle-based inference algorithm that leverages gradient information for efficient approximate inference. In this work, we enhance SVGD by leveraging preconditioning matrices, such as the Hessian and Fisher information matrix, to incorporate geometric information into SV…

New method uses multiple kernels to improve SVGD performance.

This paper shows equivalence between SVGD and BBVI using kernel gradient flows.

Stein variational gradient descent (SVGD) is a non-parametric inference algorithm that evolves a set of particles to fit a given distribution of interest. We analyze the non-asymptotic properties of SVGD, showing that there exists a set of functions, which we call the Stein matching set, whose expectations are exactly …

ASVGD accelerates SVGD for efficient sampling.

ASVGD accelerates SVGD for efficient sampling from Gaussian targets.

Paper analyzes SVGD algorithm for non-asymptotic convergence.

We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference. The method is to apply the Random Batch Method (RBM) for interacting particle systems proposed by Jin et al to the …

Two SVGD variants achieve fast convergence with provable guarantees.

Gaussian-SVGD dynamics converge to Gaussian distributions under certain conditions.

Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for complex distributions. However, the standard SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable. In this work, we develop a gradient-free variant …

Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reprod…

SVGD algorithm converges at rate 1/sqrt(log log n) for sub-Gaussian distributions.

Stein variational neural network ensembles improve diversity and uncertainty estimation.

We develop Riemannian Stein Variational Gradient Descent (RSVGD), a Bayesian inference method that generalizes Stein Variational Gradient Descent (SVGD) to Riemann manifold. The benefits are two-folds: (i) for inference tasks in Euclidean spaces, RSVGD has the advantage over SVGD of utilizing information geometry, and …

Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.

Stein transport improves Bayesian inference with faster convergence and reduced variance.

NVGD uses neural networks to infer distributions without kernel choices.

The paper analyzes rates for a modified gradient descent method using Stein variational gradients.

We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) to leverage the Markov dependency structure of the distribution of interest. Our approach combines SVGD with a set of structured local kernel functions defined on the Markov blanket…

Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.

Paper analyzes convergence rates of mean-field SVGD method.

New algorithms accelerate SVGD convergence using deep unfolding.

A method for efficient approximate inference on discrete distributions.

Improves SVGD for high-dimensional Bayesian inference by reducing variance collapse.

Stein variational gradient descent (SVGD) is a recently proposed particle-based Bayesian inference method, which has attracted a lot of interest due to its remarkable approximation ability and particle efficiency compared to traditional variational inference and Markov Chain Monte Carlo methods. However, we observed th…

A new method improves inference for complex Bayesian models.

New particle-based VI algorithm expands function class and improves scalability.

Inspired by dynamic programming, we propose Stochastic Virtual Gradient Descent (SVGD) algorithm where the Virtual Gradient is defined by computational graph and automatic differentiation. The method is computationally efficient and has little memory requirements. We also analyze the theoretical convergence properties …

SMI uses mixture models to improve SVGD's performance in Bayesian inference.

New algorithm improves convergence of Bayesian inference.

Efficiently samples and learns densities with symmetries using equivariant methods.

A new method de-randomizes MCMC dynamics using the Stein operator.

In Bayesian inference, the posterior distributions are difficult to obtain analytically for complex models such as neural networks. Variational inference usually uses a parametric distribution for approximation, from which we can easily draw samples. Recently discrete approximation by particles has attracted attention …

Accelerates MMLE using SVGD with Nesterov acceleration.

A regression-based BNN model is proposed to predict spatiotemporal quantities like hourly rider demand with calibrated uncertainties. The main contributions of this paper are (i) A feed-forward deterministic neural network (DetNN) architecture that predicts cyclical time series data with sensitivity to anomalous foreca…

We propose a novel adaptive importance sampling algorithm which incorporates Stein variational gradient decent algorithm (SVGD) with importance sampling (IS). Our algorithm leverages the nonparametric transforms in SVGD to iteratively decrease the KL divergence between our importance proposal and the target distributio…

A new method for SVGD reduces variance in high dimensions.

We propose a unifying view of two different Bayesian inference algorithms, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) and Stein Variational Gradient Descent (SVGD), leading to improved and efficient novel sampling schemes. We show that SVGD combined with a noise term can be framed as a multiple chain SG-MCM…

Particle-optimization-based sampling (POS) is a recently developed effective sampling technique that interactively updates a set of particles. A representative algorithm is the Stein variational gradient descent (SVGD). We prove, under certain conditions, SVGD experiences a theoretical pitfall, {\it i.e.}, particles te…

There has been recent interest in developing scalable Bayesian sampling methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm simulates samples from a discrete-time Markov chain to approximate a target distribution, thus samp…

Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the i…

This study improves uncertainty estimation in FWI using SVGD with annealing and PCA.

Stein variational gradient descent improves inference in Gaussian process models.

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has been increasingly popular in Bayesian learning due to its ability to deal with large data. A standard SG-MCMC algorithm simulates samples from a discretized-time Markov chain to approximate a target distribution. However, the samples are typically highly correl…

Scalable Bayesian sampling is playing an important role in modern machine learning, especially in the fast-developed unsupervised-(deep)-learning models. While tremendous progresses have been achieved via scalable Bayesian sampling such as stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD)…