Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
Improved SSD for faster and more accurate goodness-of-fit tests and model learning.
problem Optimal slicing directions for SSD are computationally expensive and sub-optimal.
method Relaxed optimal slicing requirement, active sub-space construction, spectral decomposition.
result 14-80x speed-up in goodness-of-fit tests compared to gradient-based alternatives.
Stochastic Stein Discrepancies improve inference efficiency.
problem Intractable computation of Stein discrepancies.
method Subsampled approximations of Stein operators.
result Stochastic Stein Discrepancies inherit convergence properties of standard SDs.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
Stein's method improves probabilistic inference and learning.
problem Improving probabilistic inference and learning methods.
method Constructing Stein discrepancies from Stein operators and Stein sets, discussing their properties.
result Connection between Stein operators and Stein variational gradient descent.
A new framework improves kernel Stein discrepancy tests for validating distributions.
problem Improving goodness-of-fit testing for non-normal distributions.
method Introducing Sf-KSD, a unifying framework for studying Stein operators in KSD-based tests.
result Sf-KSD guides the development of new tests and outperforms existing methods.
Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
problem Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
method Reformulating Stein discrepancy construction as an explicit SNR^2 maximisation problem
result Avoiding exponential SNR^2 collapse and achieving stable SNR^2
Inequalities linking entropy, Fisher info, Stein discrepancy, and Wasserstein distance on Riemannian manifolds.
problem Linking entropy, Fisher info, Stein discrepancy, and Wasserstein distance on Riemannian manifolds.
method Deriving inequalities linking these measures on Riemannian manifolds.
result Strengthening and extending existing inequalities to Riemannian manifolds.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
Bayesian inference uses Stein discrepancy for robustness in intractable likelihoods.
problem Intractable likelihoods in Bayesian inference.
method Generalised Bayesian inference with Stein discrepancy as the loss function.
result Robust generalised posteriors with closed form or accessible using MCMC.
Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational co…
When maximum likelihood estimation is infeasible, one often turns to score matching, contrastive divergence, or minimum probability flow to obtain tractable parameter estimates. We provide a unifying perspective of these techniques as minimum Stein discrepancy estimators, and use this lens to design new diffusion kerne…
New method uses kernel Stein discrepancy for measure transport without strict continuity constraints.
problem Minimizing Kullback-Leibler divergence for posterior approximation.
method Proposes minimizing kernel Stein discrepancy instead of Kullback-Leibler divergence.
result Demonstrates consistency and competitiveness of the new method.
CSD improves goodness-of-fit testing for higher-order dependence.
problem Insensitivity of standard KSDs to higher-order dependence features like tail dependence.
method Introduces Copula-Stein Discrepancy (CSD) that targets dependence geometry directly on copula density.
result CSD is sensitive to differences in tail dependence coefficients and metrizes weak convergence of copula distributions.
Improved kernel Stein discrepancy for large-scale data.
problem Efficiently testing probability distributions with kernel methods.
method Nyström approximation to reduce runtime complexity.
result Nyström-based KSD is n \sqrt{n} n -consistent and applicable for large datasets. Regularized Stein thinning improves MCMC output approximations.
problem Pathologies in Stein thinning leading to poor approximations.
method Theoretical analysis and regularization to improve KSD.
result Regularized Stein thinning alleviates pathologies and improves efficiency.
Paper proposes kernelized Stein tests for time-to-event data with censoring.
problem Testing goodness-of-fit for time-to-event data with censoring.
method Combining Stein's method and kernelized discrepancies for non-parametric testing.
result Proposed kernelized Stein discrepancy tests perform better than existing methods.
Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
A new sequential test for unnormalized densities.
problem Testing unnormalized densities with adaptive stopping.
method Sequential kernelized Stein discrepancy test, using non-uniform Stein kernels.
result Valid test with asymptotic lower bound for growth.
Stein importance sampling is a widely applicable technique based on kernelized Stein discrepancy, which corrects the output of approximate sampling algorithms by reweighting the empirical distribution of the samples. A general analysis of this technique is conducted for the previously unconsidered setting where samples…
New research sets the minimax lower bound for KSD estimation at sqrt(n).
problem Estimating goodness-of-fit using Kernel Stein Discrepancy (KSD) on high-dimensional spaces.
method Two complementary results proving the minimax lower bound of KSD estimation.
result The minimax lower bound of KSD estimation is n^(-1/2), indicating exponential difficulty with dimensionality.
New autoencoder improves latent space learning by optimizing sliced Gromov-Wasserstein discrepancies.
problem Improving inner discrepancy between prior and posterior distributions in autoencoders.
method Proposed spherical sliced fused Gromov Wasserstein (SSFG) and variants (MSSFG, PSSFG) to find important directions.
result New autoencoders achieve favorable performance in latent manifold learning, image generation, and reconstruction.
Paper proposes approximate Stein classes for efficient truncated density estimation.
problem Difficulties in estimating truncated density models due to intractable normalising constants and boundary conditions.
method Adapts score matching to solve the problem, introduces approximate Stein classes and a novel discrepancy measure, TKSD.
result TKSD does not require a fixed weighting function and can be evaluated using only boundary samples, leading to improved accuracy.
SVGD algorithm converges at rate 1/sqrt(log log n) for sub-Gaussian distributions.
problem Approximating a probability distribution with particles.
method Stein variational gradient descent (SVGD) with finite particles and sub-Gaussian target distribution.
result SVGD achieves a convergence rate of 1/sqrt(log log n) for sub-Gaussian distributions.
New method reduces computational cost for learning stationary diffusions.
problem Learning parameters of stationary diffusions efficiently.
method Stein-type discrepancy (SKDS) for estimating generator expectations.
result SKDS guarantees alignment with target stationary distribution.
AgraSSt assesses graph generators using Stein operators and kernel discrepancies.
problem Assessing the quality of graph generators that are implicit or not in explicit form.
method AgraSSt uses Stein operators and kernel discrepancies to assess graph generators, providing interpretable criticisms.
result Theoretical guarantees and empirical validation for various graph models.
A new test assesses how well observed networks fit a specified ERGM model.
problem Testing the goodness of fit for ERGMs with a single network observation.
method Kernel Stein discrepancy combined with a discrete Stein operator for ERGMs, Monte Carlo simulation.
result The test provides theoretical and practical support for assessing ERGM fit.
New conditions ensure MMDs separate and converge to target distributions.
problem Ensuring MMDs separate and converge to target distributions.
method Deriving new sufficient and necessary conditions for MMDs on separable metric spaces.
result First KSDs that exactly metrize weak convergence to P.
A new method for assessing Bayesian sampling quality, PSD, is proposed and shown to be more powerful and efficient.
problem Scalability and convergence assessment of Bayesian sampling algorithms, especially for high-dimensional problems.
method Polynomial Stein Discrepancy (PSD) for measuring discrepancy between samples and posterior distributions.
result PSD detects differences in the first r moments for Gaussian targets and is more powerful and efficient than competitors.
KSD Descent uses KSD to sample from a target distribution efficiently.
problem Sampling from complex target distributions efficiently.
method Wasserstein gradient flow of KSD, using L-BFGS optimization.
result KSD Descent can sample from a target distribution using a set of particles.
This work interprets diffusion score matching using normalizing flows for better model training and evaluations.
problem Limitations of diffusion score matching when dealing with certain types of distributions.
method The approach involves interpreting the diffusion matrix using normalizing flows to provide better interpretation and usage of diffusion score matching.
result Diffusion score matching is equivalent to the original score matching evaluated in the transformed space defined by the normalizing flow.
Improved KSD test for better detection of differences in distributions.
problem Low power of KSD test when distributions have same modes but different mixing proportions.
method Perturb the observed sample using Markov transition kernels to improve KSD test power.
result Perturbed KSD test can lead to substantially higher power than the original KSD test.
A new test for IRG models using KSD for small networks.
problem Testing goodness-of-fit for inhomogeneous random graph models.
method Kernelised Stein Discrepancy (KSD) test for IRG models.
result The test is effective for small networks and provides theoretical guarantees.
New KSDs control moments in approximations, improving diagnostics and tests.
problem Inability of standard KSDs to control moment convergence.
method Developed alternative diffusion KSDs under sufficient conditions.
result First KSDs to exactly characterize q-Wasserstein convergence.
Paper analyzes SVGD algorithm for non-asymptotic convergence.
problem Optimizing a set of particles to approximate a target probability distribution.
method Finite time analysis of SVGD algorithm, providing descent lemma and convergence rates.
result SVGD algorithm decreases the objective at each iteration and converges to the target distribution.
Study optimizes KSD estimation from samples, revealing Hilbert-Schmidt vs trace scales.
problem Optimizing estimation of Kernel Stein Discrepancy from samples.
method Identifying and comparing minimax scales for U-statistic and V-statistic.
result Hilbert-Schmidt norm of Stein covariance operator gives optimal scale.
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…
A new kernel Stein test assesses fit for variable-length sequential data.
problem Evaluating goodness of fit for varying-dimensional data like text documents of different lengths.
method Extends kernel Stein discrepancy (KSD) to variable-dimension settings by identifying appropriate Stein operators and proposing a novel KSD goodness-of-fit test.
result The proposed test performs well on discrete sequential data benchmarks.
Stein's method for measuring convergence to a continuous target distribution relies on an operator characterizing the target and Stein factor bounds on the solutions of an associated differential equation. While such operators and bounds are readily available for a diversity of univariate targets, few multivariate targ…
New method uses multiple kernels to improve SVGD performance.
problem Sub-optimal performance of single kernel in SVGD.
method Combines multiple kernels to approximate optimal kernel, using Kernelized Stein Discrepancy (KSD) and constructing Multiple Kernel SVGD (MK-SVGD).
result Consistently matches or outperforms competing methods in experiments.
We present a new method for evaluating and training unnormalized density models. Our approach only requires access to the gradient of the unnormalized model's log-density. We estimate the Stein discrepancy between the data density p ( x ) p(x) p ( x ) and the model density q ( x ) q(x) q ( x ) defined by a vector function of the data. We paramete…
Fourier representation improves KSD for infinite-dimensional data.
problem Applying KSD to infinite-dimensional data.
method Combining measure equations with kernel methods for a Fourier representation of KSD.
result KSD can separate measures in infinite-dimensional Hilbert spaces.
New findings show fixed-kernel discriminators are weaker than feature-learning ones.
problem Comparing performance of fixed-kernel and feature-learning discriminators.
method Using function classes F 2 \mathcal{F}_2 F 2 and F 1 \mathcal{F}_1 F 1 , constructing pairs of distributions, and linking IPMs with sliced Wasserstein distances. result Fixed-kernel IPM and SD cannot discriminate certain distributions that feature-learning IPM and SD can.
Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
We derive a new discrepancy statistic for measuring differences between two probability distributions based on combining Stein's identity with the reproducing kernel Hilbert space theory. We apply our result to test how well a probabilistic model fits a set of observations, and derive a new class of powerful goodness-o…
New algorithm tunes SGMCMC hyperparameters for scalable Bayesian inference.
problem Tuning hyperparameters for SGMCMC is challenging due to lack of principled methods.
method Proposes a bandit-based algorithm using Stein discrepancies to tune hyperparameters.
result The method effectively tunes SGMCMC hyperparameters for various applications.
We show that for any po sitive integer m m m , there exist order n n n Stein corks. The boundaries are cyclic branched covers of slice knots embedded in the boundary of corks. By applying these corks to generalized forms, we give a method producing examples of many finite order corks, which are possibly not Stein cork.
A new method for kernel tests without data splitting increases power.
problem Lack of power in kernel-based tests due to data splitting.
method Selective inference framework to learn hyperparameters and test on full sample.
result Empirically larger test power without data splitting, regardless of split proportion.