KSD Thinning uses KSD to thin MCMC samples efficiently.
problem Efficiently representing posterior distributions in Bayesian inference.
method KSD Thinning: retains only samples exceeding a KSD threshold.
result Established convergence and complexity tradeoffs for KSD Thinning.
New research sets the minimax lower bound for KSD estimation at sqrt(n).
problem Estimating goodness-of-fit using Kernel Stein Discrepancy (KSD) on high-dimensional spaces.
method Two complementary results proving the minimax lower bound of KSD estimation.
result The minimax lower bound of KSD estimation is n^(-1/2), indicating exponential difficulty with dimensionality.
New KSDs control moments in approximations, improving diagnostics and tests.
problem Inability of standard KSDs to control moment convergence.
method Developed alternative diffusion KSDs under sufficient conditions.
result First KSDs to exactly characterize q-Wasserstein convergence.
KSD Descent uses KSD to sample from a target distribution efficiently.
problem Sampling from complex target distributions efficiently.
method Wasserstein gradient flow of KSD, using L-BFGS optimization.
result KSD Descent can sample from a target distribution using a set of particles.
Improved KSD test for faster GoF testing.
problem Slow and computationally intractable KSD tests.
method Nyström acceleration for KSD estimation.
result Asymptotic properties preserved by Nyström acceleration.
Fourier representation improves KSD for infinite-dimensional data.
problem Applying KSD to infinite-dimensional data.
method Combining measure equations with kernel methods for a Fourier representation of KSD.
result KSD can separate measures in infinite-dimensional Hilbert spaces.
Improved KSD test for better detection of differences in distributions.
problem Low power of KSD test when distributions have same modes but different mixing proportions.
method Perturb the observed sample using Markov transition kernels to improve KSD test power.
result Perturbed KSD test can lead to substantially higher power than the original KSD test.
A new kernel Stein test assesses fit for variable-length sequential data.
problem Evaluating goodness of fit for varying-dimensional data like text documents of different lengths.
method Extends kernel Stein discrepancy (KSD) to variable-dimension settings by identifying appropriate Stein operators and proposing a novel KSD goodness-of-fit test.
result The proposed test performs well on discrete sequential data benchmarks.
Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target dist…
Improved kernel Stein discrepancy for large-scale data.
problem Efficiently testing probability distributions with kernel methods.
method Nyström approximation to reduce runtime complexity.
result Nyström-based KSD is n \sqrt{n} n -consistent and applicable for large datasets. A new framework improves kernel Stein discrepancy tests for validating distributions.
problem Improving goodness-of-fit testing for non-normal distributions.
method Introducing Sf-KSD, a unifying framework for studying Stein operators in KSD-based tests.
result Sf-KSD guides the development of new tests and outperforms existing methods.
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Ad-SVGD optimizes kernel parameters for SVGD, improving inference performance.
problem Efficiently approximating posterior distributions in Bayesian inference.
method Adaptive kernel selection for SVGD dynamics.
result Ad-SVGD outperforms standard heuristics in various tasks.
A new method improves Bayesian inference for multimodal posteriors.
problem Insensitivity to well-separated modes in multimodal posteriors.
method Weighted Kernel Stein Discrepancy method.
result Significantly improved mode sensitivity compared to standard KSD-Bayes.
A new test for IRG models using KSD for small networks.
problem Testing goodness-of-fit for inhomogeneous random graph models.
method Kernelised Stein Discrepancy (KSD) test for IRG models.
result The test is effective for small networks and provides theoretical guarantees.
Study evaluates RKHS choices for assessing graph models using KSD tests.
problem Effect of RKHS choice on KSD tests for graph model assessment.
method Investigated power performance and computational runtime of KSD tests for ERGMs and synthetic graph generators.
result Different RKHS choices affect KSD test performance and computational runtime.
KSDAgg combines multiple KSD tests to improve goodness-of-fit testing without splitting data.
problem Improving goodness-of-fit testing without data splitting.
method KSDAgg aggregates multiple KSD tests with different kernels to maximize power.
result KSDAgg achieves the smallest uniform separation rate of the collection, up to a logarithmic term.
Study optimizes KSD estimation from samples, revealing Hilbert-Schmidt vs trace scales.
problem Optimizing estimation of Kernel Stein Discrepancy from samples.
method Identifying and comparing minimax scales for U-statistic and V-statistic.
result Hilbert-Schmidt norm of Stein covariance operator gives optimal scale.
New conditions ensure MMDs separate and converge to target distributions.
problem Ensuring MMDs separate and converge to target distributions.
method Deriving new sufficient and necessary conditions for MMDs on separable metric spaces.
result First KSDs that exactly metrize weak convergence to P.
Unified framework for optimal kernel tests across MMD, HSIC, and KSD.
problem Optimal testing in kernel-based hypothesis testing frameworks.
method Unified derivation of minimax rates, adaptive kernel selection methods.
result Unified power results across MMD, HSIC, and KSD.
A new algorithm STE for model-based RL improves learning rates.
problem Sparse rewards and computational intractability of estimating information gain.
method Developed a novel algorithm based on Stein Information Directed Exploration (STE)E.
result Achieves sublinear Bayesian regret, outperforming prior approaches.
Kernel SIVI improves variational inference by avoiding lower-level optimization.
problem Intractable densities in semi-implicit variational distributions.
method Kernel SIVI-SM uses a minimax formulation and kernel tricks to avoid lower-level optimization.
result Kernel Stein discrepancy (KSD) objective is computable and leads to convergence guarantees.
New method uses kernel Stein discrepancy for measure transport without strict continuity constraints.
problem Minimizing Kullback-Leibler divergence for posterior approximation.
method Proposes minimizing kernel Stein discrepancy instead of Kullback-Leibler divergence.
result Demonstrates consistency and competitiveness of the new method.
Improved convergence rates for Stein Variational Gradient Descent in finite-particle settings.
problem Improving convergence rates for Stein Variational Gradient Descent in finite-particle settings.
method Analyzing the time derivative of relative entropy and splitting it into dominant and smaller parts.
result Finite-particle convergence rates of order 1/\sqrt{N} for Kernelized Stein Discrepancy and Wasserstein-2 metrics.
Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We …
High-dimensional U-statistics show surprising phase transitions, impacting kernel-based tests.
problem Understanding phase transitions in high-dimensional U-statistics.
method Proved a convergence theorem for U-statistics of degree two in high dimensions.
result High-dimensional U-statistics can have non-Gaussian limits with larger variance and asymmetry.
A new measure helps compute suboptimality in entropy-regularized methods.
problem Computing suboptimality in entropy-regularized variational objectives when unnormalised densities are unavailable.
method Introduced 'kernel gradient discrepancy' (KGD) to compute suboptimality explicitly.
result KGD characterizes kernel Stein discrepancy (KSD) in the standard Bayesian context and measures variational gradient size.
CSD improves goodness-of-fit testing for higher-order dependence.
problem Insensitivity of standard KSDs to higher-order dependence features like tail dependence.
method Introduces Copula-Stein Discrepancy (CSD) that targets dependence geometry directly on copula density.
result CSD is sensitive to differences in tail dependence coefficients and metrizes weak convergence of copula distributions.
Kernel semi-implicit variational inference improves variational inference without additional optimization.
problem Intractability of hierarchical semi-implicit distributions in variational inference.
method Kernel semi-implicit variational inference (KSIVI) using kernel methods to eliminate lower-level optimization.
result KSIVI reduces variational inference to kernel Stein discrepancy (KSD) optimization, improving expressiveness and tractability.
A new method for assessing Bayesian sampling quality, PSD, is proposed and shown to be more powerful and efficient.
problem Scalability and convergence assessment of Bayesian sampling algorithms, especially for high-dimensional problems.
method Polynomial Stein Discrepancy (PSD) for measuring discrepancy between samples and posterior distributions.
result PSD detects differences in the first r moments for Gaussian targets and is more powerful and efficient than competitors.
Deep neural networks can approximate any target probability distribution given certain conditions.
problem Approximating complex probability distributions with deep neural networks.
method Proving the existence of a deep neural network mapping that approximates a target distribution under various integral probability metrics.
result Upper bounds on the size of the neural network in terms of dimension and approximation error for different metrics.
The paper calibrates geophysical predictions using marginal distributions and machine learning.
problem Sensitivity to initial conditions in geophysical systems leads to large deviations in long-term forecasts.
method The method introduces a calibration algorithm based on normalization and Kernelized Stein Discrepancy (KSD) to enhance ML predictions.
result The method improves the fidelity of ML predictions to known physical distributions, ensuring consistency with non-local statistical structures.
Novel MBRL method for large-scale RL with reduced posterior complexity.
problem Theoretical guarantees for MBRL in large spaces with complex models.
method Kernelized Stein Discrepancy for compression of posterior estimate.
result Sublinear Bayesian regret and up to 50% reduction in training time.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable. The proposed test generalizes the recently proposed kernel Stein discrepanc…
A new method for kernel tests without data splitting increases power.
problem Lack of power in kernel-based tests due to data splitting.
method Selective inference framework to learn hyperparameters and test on full sample.
result Empirically larger test power without data splitting, regardless of split proportion.
Paper presents variational estimates for EBLVMs without structural assumptions.
problem Challenges in learning and evaluating EBLVMs due to intractable true posteriors and partition functions.
method Variational estimates of the score function and its gradient (VaES and VaGES) in a general EBLVM.
result The estimates can be applied to KSD and SM-based methods to learn EBLVMs and estimate Fisher divergence.
Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational co…
Regularized Stein thinning improves MCMC output approximations.
problem Pathologies in Stein thinning leading to poor approximations.
method Theoretical analysis and regularization to improve KSD.
result Regularized Stein thinning alleviates pathologies and improves efficiency.
New method uses multiple kernels to improve SVGD performance.
problem Sub-optimal performance of single kernel in SVGD.
method Combines multiple kernels to approximate optimal kernel, using Kernelized Stein Discrepancy (KSD) and constructing Multiple Kernel SVGD (MK-SVGD).
result Consistently matches or outperforms competing methods in experiments.
New robustness test for kernel goodness-of-fit tests.
problem Lack of robustness in existing kernel goodness-of-fit tests.
method Proposes a new robust kernel goodness-of-fit test using kernel Stein discrepancy (KSD) balls.
result First robust kernel goodness-of-fit test addressing both qualitative and quantitative robustness.
Paper proposes a new method to assess synthetic data generators.
problem Assessing the quality of implicit generative models.
method Kernelised Stein Statistic (KSD) test based on non-parametric Stein operator.
result Improved power performance compared to existing approaches.
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
Copula Discrepancy benchmarks sample dependence structure against known families.
problem Benchmarking sample dependence structure against known families.
method Copula Discrepancy (CD) statistic comparing target Kendall's tau with fitted parameter.
result CD reliably separates on-target and off-target copulas.
Unified score and distance-based GoF tests for model adequacy.
problem Difficulty in extending score-based GoF tests to nonparametric alternatives.
method Introducing semiparametric kernelized Stein discrepancy (SKSD) test.
result SKSD test is computationally efficient and universally consistent.
Efficient tests for various statistical problems using incomplete U-statistics.
problem Nonparametric tests for two-sample, independence, and goodness-of-fit problems.
method Proposes MMDAggInc, HSICAggInc, and KSDAggInc tests aggregating over multiple kernel bandwidths.
result Aggregated tests provide a solution to the kernel selection problem and achieve optimal rates.