arXiv research

The paper improves Stein importance sampling for Markov chain samples.

We propose a novel adaptive importance sampling algorithm which incorporates Stein variational gradient decent algorithm (SVGD) with importance sampling (IS). Our algorithm leverages the nonparametric transforms in SVGD to iteratively decrease the KL divergence between our importance proposal and the target distributio…

Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing

Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational co…

An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target an…

Improved sampling method using regularized Stein Variational Gradient Flow.

New method improves sample diversity and efficiency from complex distributions.

The paper introduces a novel method for training neural network Stein critics with staged $L^2$-regularization.

JSRT improves regression tree performance by incorporating global node information.

BSVGD improves sampling for multimodal distributions using branching.

New framework for scalable approximate inference tackles high-dimensional models and large datasets.

Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…

We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the data/distribution-dependent Efron-Stein (ES) estimate of its variance and they do not requ…

A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this e…

SteinGen generates diverse graph samples from a single example.

A new method de-randomizes MCMC dynamics using the Stein operator.

Stochastic Stein Discrepancies improve inference efficiency.

Efficiently samples and learns densities with symmetries using equivariant methods.

Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…

Researchers derive an analytic expression for Gaussian stochastic volatility models.

Stein variational gradient descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate given distributions, based on an efficient gradient-based update that guarantees to optimally decrease the KL divergence within a function space. This paper develops the first th…

In this paper, we show that the Ozsváth-Szabó contact invariant $c^+(ξ)\in HF^+(-Y)$ of a contact 3-manifold $(Y,ξ)$ can be calculated combinatorially if $Y$ is the boundary of a certain type of plumbing $X$, and $ξ$ is induced by a Stein structure on $X$. Our technique uses an algorithm of Ozsváth and Szabó to determi…

KSD Descent uses KSD to sample from a target distribution efficiently.

An important task in computational statistics and machine learning is to approximate a posterior distribution $p(x)$ with an empirical measure supported on a set of representative points $\{x_i\}_{i=1}^n$. This paper focuses on methods where the selection of points is essentially deterministic, with an emphasis on achi…

SDG uses optimal control to improve classifier guidance in low-density regions.

Stein shrinkage improves BN robustness against adversarial attacks.

Efficient event generation for collider phenomenology using parallel Langevin sampling and learned Stein diagnostics.

New method for sampling on constrained domains using orthogonal-space gradient flow.

A new method for assessing Bayesian sampling quality, PSD, is proposed and shown to be more powerful and efficient.

New Stein identity for q-Gaussians reduces gradient variance in machine learning.

SRF improves kernel approximation and GP regression performance.

ASVGD accelerates SVGD for efficient sampling.

Study optimizes KSD estimation from samples, revealing Hilbert-Schmidt vs trace scales.

New algorithm improves convergence of Bayesian inference.

Bayesian deep learning improves maintenance planning uncertainty quantification.

New samplers minimize KL divergence for constrained and non-Euclidean geometries.

Lower bounds on private estimation of Gaussian covariance matrices.

Stein's method for measuring convergence to a continuous target distribution relies on an operator characterizing the target and Stein factor bounds on the solutions of an associated differential equation. While such operators and bounds are readily available for a diversity of univariate targets, few multivariate targ…

A new sequential test for unnormalized densities.

We propose two novel samplers to generate high-quality samples from a given (un-normalized) probability density. Motivated by the success of generative adversarial networks, we construct our samplers using deep neural networks that transform a reference distribution to the target distribution. Training schemes are deve…

CSD improves goodness-of-fit testing for higher-order dependence.

Bayesian online changepoint detection (BOCPD) (Adams & MacKay, 2007) offers a rigorous and viable way to identify changepoints in complex systems. In this work, we introduce a Stein variational online changepoint detection (SVOCD) method to provide a computationally tractable generalization of BOCPD beyond the exponent…

Improved kernel Stein discrepancy for large-scale data.

New method controls error in low-dimensional marginals of spatial models.

Paper proposes approximate Stein classes for efficient truncated density estimation.

New method uses CNN for seismic inversion uncertainty quantification.

In Bayesian inference, the posterior distributions are difficult to obtain analytically for complex models such as neural networks. Variational inference usually uses a parametric distribution for approximation, from which we can easily draw samples. Recently discrete approximation by particles has attracted attention …

New algorithms accelerate SVGD convergence using deep unfolding.