Paper proposes approximate Stein classes for efficient truncated density estimation.
problem Difficulties in estimating truncated density models due to intractable normalising constants and boundary conditions.
method Adapts score matching to solve the problem, introduces approximate Stein classes and a novel discrepancy measure, TKSD.
result TKSD does not require a fixed weighting function and can be evaluated using only boundary samples, leading to improved accuracy.
Stochastic Stein Discrepancies improve inference efficiency.
problem Intractable computation of Stein discrepancies.
method Subsampled approximations of Stein operators.
result Stochastic Stein Discrepancies inherit convergence properties of standard SDs.
New Stein operator improves robustness in model inference.
problem Improving robustness in inference for unnormalized models.
method Density-power weighted Stein operator (γ-Stein operator). result Robust methods for goodness-of-fit testing and posterior approximation.
New samplers minimize KL divergence for constrained and non-Euclidean geometries.
problem Efficient sampling from constrained and non-Euclidean distributions.
method Stein Variational Mirror Descent and Mirrored Stein Variational Gradient Descent.
result New samplers converge more rapidly and accurately than prior methods.
Stein's method for measuring convergence to a continuous target distribution relies on an operator characterizing the target and Stein factor bounds on the solutions of an associated differential equation. While such operators and bounds are readily available for a diversity of univariate targets, few multivariate targ…
Bayesian neural networks speed up numerical integration.
problem Scalability of Bayesian quadrature methods.
method Bayesian Stein networks using neural networks and Laplace approximation.
result Orders of magnitude speed-up on benchmark functions and real-world problems.
An important task in machine learning and statistics is the approximation of a probability measure by an empirical measure supported on a discrete point set. Stein Points are a class of algorithms for this task, which proceed by sequentially minimising a Stein discrepancy between the empirical measure and the target an…
Ad-SVGD optimizes kernel parameters for SVGD, improving inference performance.
problem Efficiently approximating posterior distributions in Bayesian inference.
method Adaptive kernel selection for SVGD dynamics.
result Ad-SVGD outperforms standard heuristics in various tasks.
Study approximates weak error for specific stochastic models with rough and Gaussian mean-reverting volatility.
problem Approximating weak error for specific stochastic models with rough and Gaussian mean-reverting volatility.
method Used Euler type scheme with integrated kernels to study weak convergence rate.
result Obtained weak convergence rate of min(3α−1,1) for discretised rough Ornstein-Uhlenbeck process and stochastic rough volatility model. Improved neural network ensembles using Stein Variational Newton updates.
problem Lack of efficient second-order information in current ensemble methods.
method Proposes a novel approximate Bayesian inference method integrating Stein Variational Newton updates with scalable Hessian approximations.
result Significantly faster convergence and more accurate posterior distribution approximations.
An important task in computational statistics and machine learning is to approximate a posterior distribution p(x) with an empirical measure supported on a set of representative points {xi}i=1n. This paper focuses on methods where the selection of points is essentially deterministic, with an emphasis on achi…
Regularized Stein thinning improves MCMC output approximations.
problem Pathologies in Stein thinning leading to poor approximations.
method Theoretical analysis and regularization to improve KSD.
result Regularized Stein thinning alleviates pathologies and improves efficiency.
AgraSSt assesses graph generators using Stein operators and kernel discrepancies.
problem Assessing the quality of graph generators that are implicit or not in explicit form.
method AgraSSt uses Stein operators and kernel discrepancies to assess graph generators, providing interpretable criticisms.
result Theoretical guarantees and empirical validation for various graph models.
Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational co…
New method uses kernel Stein discrepancy for measure transport without strict continuity constraints.
problem Minimizing Kullback-Leibler divergence for posterior approximation.
method Proposes minimizing kernel Stein discrepancy instead of Kullback-Leibler divergence.
result Demonstrates consistency and competitiveness of the new method.
Stein transport improves Bayesian inference with faster convergence and reduced variance.
problem Efficiently approximating posterior distributions in Bayesian inference.
method A novel Bayesian inference method using Stein transport, which pushes particles along a curve of tempered distributions.
result Stein transport reaches posterior approximations faster and more accurately than Stein variational gradient descent (SVGD).
This paper analyzes Stein variational gradient descent for Bayesian inference.
problem Sampling or approximating high-dimensional probability distributions.
method Iterated steepest descent steps with a reproducing kernel Hilbert space norm.
result Performance gains of certain nondifferentiable kernels with adjusted tails.
Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
problem Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
method Reformulating Stein discrepancy construction as an explicit SNR^2 maximisation problem
result Avoiding exponential SNR^2 collapse and achieving stable SNR^2
SRF improves kernel approximation and GP regression performance.
problem Efficient kernel approximation and Bayesian kernel learning in large-scale regression problems.
method Stein variational gradient descent to generate high-quality random features and approximate spectral measure posteriors.
result SRF outperforms traditional approaches in kernel approximation and GP regression.
The paper analyzes rates for a modified gradient descent method using Stein variational gradients.
problem Improving the accuracy of gradient descent methods for complex target distributions.
method Derives finite-particle rates for regularized Stein variational gradient descent (R-SVGD).
result Establishes explicit non-asymptotic bounds for time-averaged empirical measures.
New open books solve a long-standing surface mapping class group question.
problem Understanding the mapping class group of surfaces with boundary.
method Constructing non-positive open books with once-punctured torus pages.
result Monoid of positive monodromies equals the monoid of monodromies supporting Stein-fillable contact structures if and only if the surface is planar.
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…
Improved sampling method using regularized Stein Variational Gradient Flow.
problem Improving the accuracy of sampling methods in machine learning.
method Proposed Regularized Stein Variational Gradient Flow to interpolate between SVGD and Wasserstein Gradient Flow.
result Established theoretical properties and provided preliminary numerical evidence of improved performance.
Stein variational gradient descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate given distributions, based on an efficient gradient-based update that guarantees to optimally decrease the KL divergence within a function space. This paper develops the first th…
CSD improves goodness-of-fit testing for higher-order dependence.
problem Insensitivity of standard KSDs to higher-order dependence features like tail dependence.
method Introduces Copula-Stein Discrepancy (CSD) that targets dependence geometry directly on copula density.
result CSD is sensitive to differences in tail dependence coefficients and metrizes weak convergence of copula distributions.
New KSDs control moments in approximations, improving diagnostics and tests.
problem Inability of standard KSDs to control moment convergence.
method Developed alternative diffusion KSDs under sufficient conditions.
result First KSDs to exactly characterize q-Wasserstein convergence.
We characterize the closed, oriented, Seifert fibered 3-manifolds which are oriented boundaries of Stein manifolds. We also show that for this class of 3-manifolds the existence of Stein fillings is equivalent to the existence of symplectic fillings.
New method uses multiple kernels to improve SVGD performance.
problem Sub-optimal performance of single kernel in SVGD.
method Combines multiple kernels to approximate optimal kernel, using Kernelized Stein Discrepancy (KSD) and constructing Multiple Kernel SVGD (MK-SVGD).
result Consistently matches or outperforms competing methods in experiments.
A new method improves inference for complex Bayesian models.
problem Bayesian inference for doubly intractable distributions is computationally challenging.
method Monte Carlo Stein variational gradient descent (MC-SVGD) approach.
result The method achieves substantial computational gains over existing algorithms.
A method for efficient approximate inference on discrete distributions.
problem Applying SVGD to discrete distributions.
method Transforming discrete distributions to piecewise continuous distributions for SVGD application.
result Outperforms traditional algorithms and ensemble methods on discrete graphical models.
We use contact handle decompositions and a stabilization process to compute the cylindrical contact homology of a subcritical Stein-fillable contact manifold with vanishing first Chern class, and show that it is completely determined by the homology of a subcritical Stein-filling of the contact manifold.
We give an algorithm which produces infinitely many pairwise exotic Stein fillings of the same contact 3-manifolds, applying positive allowable Lefschetz fibrations over the disk. As a corollary, for a large class of Stein fillings, we realize the topological invariants (i.e. fundamental group, homology group, homology…
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
Study Stein and Milnor fillings of links from surface singularities.
problem Comparing Stein and Milnor fillings of links from surface singularities.
method Analyzing the topology and obstructions of Stein fillings and Milnor fillings.
result Milnor fillings have bounded topology, while Stein fillings can be more varied.
A new framework improves kernel Stein discrepancy tests for validating distributions.
problem Improving goodness-of-fit testing for non-normal distributions.
method Introducing Sf-KSD, a unifying framework for studying Stein operators in KSD-based tests.
result Sf-KSD guides the development of new tests and outperforms existing methods.
SVGD algorithm converges at rate 1/sqrt(log log n) for sub-Gaussian distributions.
problem Approximating a probability distribution with particles.
method Stein variational gradient descent (SVGD) with finite particles and sub-Gaussian target distribution.
result SVGD achieves a convergence rate of 1/sqrt(log log n) for sub-Gaussian distributions.
The paper improves Stein importance sampling for Markov chain samples.
problem Improving the accuracy of sampling from complex distributions.
method Reproducing Stein kernels approach for post-hoc correction.
result Consistent estimators for target distributions using geometrically ergodic Markov chains.
The purpose of this paper is to synthesize the approaches taken by Chatterjee-Meckes and Reinert-Röllin in adapting Stein's method of exchangeable pairs for multivariate normal approximation. The more general linear regression condition of Reinert-Röllin allows for wider applicability of the method, while the method of…
KSD Descent uses KSD to sample from a target distribution efficiently.
problem Sampling from complex target distributions efficiently.
method Wasserstein gradient flow of KSD, using L-BFGS optimization.
result KSD Descent can sample from a target distribution using a set of particles.
New algorithm tunes SGMCMC hyperparameters for scalable Bayesian inference.
problem Tuning hyperparameters for SGMCMC is challenging due to lack of principled methods.
method Proposes a bandit-based algorithm using Stein discrepancies to tune hyperparameters.
result The method effectively tunes SGMCMC hyperparameters for various applications.
Paper analyzes SVGD algorithm for non-asymptotic convergence.
problem Optimizing a set of particles to approximate a target probability distribution.
method Finite time analysis of SVGD algorithm, providing descent lemma and convergence rates.
result SVGD algorithm decreases the objective at each iteration and converges to the target distribution.
Stein's method (Stein, 1973; 1981) is a powerful tool for statistical applications and has significantly impacted machine learning. Stein's lemma plays an essential role in Stein's method. Previous applications of Stein's lemma either required strong technical assumptions or were limited to Gaussian distributions with …
We develop Riemannian Stein Variational Gradient Descent (RSVGD), a Bayesian inference method that generalizes Stein Variational Gradient Descent (SVGD) to Riemann manifold. The benefits are two-folds: (i) for inference tasks in Euclidean spaces, RSVGD has the advantage over SVGD of utilizing information geometry, and …
New Stein identity for q-Gaussians reduces gradient variance in machine learning.
problem Improving gradient estimators for non-Gaussian distributions.
method Deriving a new Stein identity for bounded-support q-Gaussians and simplifying previous results.
result Gradient estimators for q-Gaussians have nearly identical forms to Gaussian ones, reducing variance.
New Stein fillings found for rational surface singularities.
problem Exploring Stein fillings of rational surface singularities.
method Using planar open books and Lefschetz fibrations, describe Stein fillings via symplectic disk arrangements.
result Many rational singularities admit Stein fillings not diffeomorphic to Milnor fibers.
We introduce a new generalization of Gompf nuclei and give applications. We construct infinitely many exotic smooth structures for a large class of compact 4-manifolds with boundary, regarding topological invariants. We prove that a large class of closed 3-manifolds (including disjoint unions of Stein fillable 3-manifo…
Stein variational gradient descent (SVGD) is a non-parametric inference algorithm that evolves a set of particles to fit a given distribution of interest. We analyze the non-asymptotic properties of SVGD, showing that there exists a set of functions, which we call the Stein matching set, whose expectations are exactly …
In this article, we prove a generalization of a theorem of Lisca-Matic to Stein cobordisms and develop a method for distinguishing certain Stein cobordisms using rotation numbers. Using these results along with standard techniques from convex surface theory and classifications of tight contact structures on certain 3-m…