A new method improves inference for complex Bayesian models.
problem Bayesian inference for doubly intractable distributions is computationally challenging.
method Monte Carlo Stein variational gradient descent (MC-SVGD) approach.
result The method achieves substantial computational gains over existing algorithms.
New MCMC methods use auxiliary variables to sample from intractable distributions.
problem Sampling from distributions with unknown normalizing constants.
method Unified Markov chain Monte Carlo framework with auxiliary variables.
result New algorithms outperform existing methods on synthetic and real datasets.
Neural network predicts short rate model steps accurately.
problem Predicting intractable short rate model steps.
method Proposes an algorithm using neural networks.
result Achieves superior outcomes compared to unbiased estimate.
A large number of statistical models are "doubly-intractable": the likelihood normalising term, which is a function of the model parameters, is intractable, as well as the marginal likelihood (model evidence). This means that standard inference techniques to sample from the posterior, such as Markov chain Monte Carlo (…
Bayesian inference for models that have an intractable partition function is known as a doubly intractable problem, where standard Monte Carlo methods are not applicable. The past decade has seen the development of auxiliary variable Monte Carlo techniques (Møller et al., 2006; Murray et al., 2006) for tackling this pr…
Bayesian inference uses Stein discrepancy for robustness in intractable likelihoods.
problem Intractable likelihoods in Bayesian inference.
method Generalised Bayesian inference with Stein discrepancy as the loss function.
result Robust generalised posteriors with closed form or accessible using MCMC.
Generative Bayesian Filtering improves inference in complex models without explicit density evaluations.
problem Performing posterior inference in complex nonlinear and non-Gaussian state-space models.
method Generative Bayesian Filtering (GBF) extends GBC to dynamic settings using deep neural networks for recursive posterior inference. Generative-Gibbs sampler bypasses density evaluations for parameter learning.
result GBF significantly outperforms likelihood-free approaches in accuracy and robustness for intractable state-space models.
Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.
problem Intractable densities in variational distributions hinder SIVI training.
method SIVI-SM uses score matching to handle intractable densities in a minimax formulation.
result SIVI-SM outperforms ELBO-based SIVI methods in Bayesian inference tasks.
Efficient Bayesian decision-making with intractable likelihoods.
problem Bayesian decision-making under intractable likelihoods.
method Learning surrogate models and using simulation-based inference and Bayesian optimization.
result Optimal actions can be learned with fewer simulations than posterior inference.
Markov chain Monte Carlo (MCMC) algorithms are widely used to sample from complicated distributions, especially to sample from the posterior distribution in Bayesian inference. However, MCMC is not directly applicable when facing the doubly intractable problem. In this paper, we discussed and compared two existing solu…
Paper establishes DRL for high-dimensional rewards.
problem Intractable reinforcement learning with high-dimensional rewards.
method Theoretical foundations and a novel DRL algorithm.
result Bellman operator contraction in high-dimensional spaces.
A new method for sampling from posterior distributions in Bayesian inverse problems.
problem Sampling from posterior distributions in Bayesian inverse problems is challenging due to intractable terms.
method Proposes a novel approach that decomposes the transitions, allowing a trade-off between complexity of guidance term and prior transitions.
result Validated through experiments on various inverse problems, including challenging cases with latent diffusion models as priors.
Top-performing machine learning systems, such as deep neural networks, large ensembles and complex probabilistic graphical models, can be expensive to store, slow to evaluate and hard to integrate into larger systems. Ideally, we would like to replace such cumbersome models with simpler models that perform equally well…
New method improves sample diversity and efficiency from complex distributions.
problem Sampling from intractable un-normalized distributions with high auto-correlation.
method Stein self-repulsive dynamics using a repulsive force to push samples away from past trajectories.
result Significantly decreases auto-correlation and increases effective sample size.
How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild dif…
Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function and its derivatives. Here we propose a new parameter estimation technique that does not require computing an intractable normalization factor or sampling from the equilibrium distribution of the model. T…
Investor maximizes utility from an unknown claim using robust optimization.
problem Maximizing utility from an unknown contingent claim.
method Robust optimization with quantile formulation and variational inequalities.
result Optimal trading strategy and utility indifference price determined.
Gradient learning optimises MCMC proposal distributions.
problem Intractable targets in MCMC sampling.
method Gradient-based optimisation of proposal distributions using a maximum entropy regularised objective function.
result Our method can outperform traditional MCMC algorithms, including Hamiltonian Monte Carlo.
New method uses neural exponential families for likelihood-free inference.
problem Bayesian Likelihood-Free Inference with intractable likelihood.
method Score Matching neural conditional exponential families for approximate likelihood.
result State-of-the-art performance in posterior sampling for intractable likelihood models.
Computing partition functions, the normalizing constants of probability distributions, is often hard. Variants of importance sampling give unbiased estimates of a normalizer Z, however, unbiased estimates of the reciprocal 1/Z are harder to obtain. Unbiased estimates of 1/Z allow Markov chain Monte Carlo sampling of "d…
This review article surveys data augmentation MCMC algorithms.
problem Sampling from intractable probability distributions.
method Comprehensive study of DA MCMC algorithms, their convergence properties, and acceleration strategies.
result Synthesizes recent developments and provides insights for researchers.
Privacy preserving mechanisms such as differential privacy inject additional randomness in the form of noise in the data, beyond the sampling mechanism. Ignoring this additional noise can lead to inaccurate and invalid inferences. In this paper, we incorporate the privacy mechanism explicitly into the likelihood functi…
This paper solves robust utility maximization with unknown claim dependencies.
problem Investor optimizes utility in the presence of an intractable contingent claim.
method Quantile optimization approach, transforming dynamic problem into static concave optimization.
result Optimal payoffs depend on ambiguity attitude, market conditions, and claim characteristics.
A new gradient estimator for categorical distributions reduces bias and variance.
problem Intractability of gradients for categorical distributions in discrete latent variable models.
method CatLog-Derivative trick and IndeCateR gradient estimator.
result IndeCateR reduces bias and variance of gradients for categorical distributions.
ABI bypasses likelihood intractability with nonparametric distribution matching.
problem Approximate Bayesian computation's inefficiency in high-dimensional settings and under diffuse priors.
method Adaptive Bayesian Inference (ABI) compares posterior distributions directly using nonparametric distribution matching and MSW distance.
result ABI significantly outperforms other methods in high-dimensional or dependent observation regimes.
We propose a general algorithm for approximating nonstandard Bayesian posterior distributions. The algorithm minimizes the Kullback-Leibler divergence of an approximating distribution to the intractable posterior distribution. Our method can be used to approximate any posterior distribution, provided that it is given i…
Optimizes experimental designs for intractable models using mutual information bounds.
problem Finding optimal experimental designs for models with intractable data-generating distributions.
method Maximizes mutual information lower bounds parametrized by neural networks, updating network parameters and designs simultaneously.
result Framework enables experimental design for various tasks including parameter estimation and model discrimination.
Many methods for machine learning rely on approximate inference from intractable probability distributions. Variational inference approximates such distributions by tractable models that can be subsequently used for approximate inference. Learning sufficiently accurate approximations requires a rich model family and ca…
Likelihood-free methods such as approximate Bayesian computation (ABC) have extended the reach of statistical inference to problems with computationally intractable likelihoods. Such approaches perform well for small-to-moderate dimensional problems, but suffer a curse of dimensionality in the number of model parameter…
Markov Chain Monte Carlo is repeatedly used to analyze the properties of intractable distributions in a convenient way. In this paper we derive conditions for geometric ergodicity of a general class of nonparametric stochastic volatility models with skewness driven by hidden Markov Chain with switching.
Paper develops a scalable distributed inference algorithm for sensor networks.
problem Efficient inference in intelligent sensor networks for location, tracking, and mapping.
method Distributed variational inference algorithm for continuous variables and large-scale data.
result Derives a separable lower bound for distributed variational inference with one-hop communication.
The mean field methods, which entail approximating intractable probability distributions variationally with distributions from a tractable family, enjoy high efficiency, guaranteed convergence, and provide lower bounds on the true likelihood. But due to requirement for model-specific derivation of the optimization equa…
Variational Prediction simplifies Bayesian inference without test time costs.
problem Bayesian inference's computational costs and posterior predictive distribution marginalization.
method Variational Prediction learns a variational approximation to the posterior predictive distribution using a variational bound.
result Directly learns a variational approximation to the posterior predictive distribution without test time marginalization costs.
A new method improves Bayesian inference for multimodal posteriors.
problem Insensitivity to well-separated modes in multimodal posteriors.
method Weighted Kernel Stein Discrepancy method.
result Significantly improved mode sensitivity compared to standard KSD-Bayes.
Bayesian approach for inhomogeneous Poisson process intensity estimation.
problem Intractable integral in likelihood of Gaussian Cox process.
method Joint modeling of intensity and cumulative intensity as transformed Gaussian process; exact MCMC sampler.
result Exact posterior inference without approximations.
We derive and approximate the conjugate prior of Dirichlet and beta distributions.
problem Intractability of conjugate prior for Dirichlet and beta distributions.
method Derive conjugate prior, define closed-form approximation, and provide algorithm.
result Closed-form approximation enables fully tractable Bayesian treatment.
Score matching offers efficient estimation for certain distributions.
problem Estimating probability distributions with intractable constants.
method Score matching as an alternative to maximum likelihood.
result Score matching is computationally and statistically efficient for certain distributions.
This paper improves conditional sampling for VAEs by overcoming structural issues.
problem Computational intractability of conditional sampling in VAEs.
method Proposes two methods to address pitfalls in Metropolis-within-Gibbs (MWG) for VAEs.
result Improved performance on sampling tasks.
We develop a general method for estimating a finite mixture of non-normalized models. Here, a non-normalized model is defined to be a parametric distribution with an intractable normalization constant. Existing methods for estimating non-normalized models without computing the normalization constant are not applicable …
ConDiSim uses diffusion models to approximate complex system posteriors efficiently.
problem Simulation-based inference of systems with intractable likelihoods.
method Conditional diffusion model with forward and reverse processes.
result Effective posterior approximation across various benchmark and real-world problems.
Posterior sampling from diffusion models is computationally hard.
problem Sampling from posterior distributions in diffusion models is intractable.
method Analyzes the computational complexity of posterior sampling in diffusion models.
result Posterior sampling is computationally intractable under cryptographic assumptions.
Algorithm simulates counterfactuals for fairness analysis.
problem Analytical intractability of counterfactuals in conditional distributions.
method Proposes an algorithm using particle filtering for discrete and continuous variables.
result Asymptotically valid inference for counterfactuals.
New method for density estimation without approximating posterior distributions.
problem Challenges in non-smooth data distributions for Bayesian density estimation.
method Autoregressive likelihood decomposition and Gaussian process prior in a quasi-Bayesian framework.
result Achieves state-of-the-art results in small-data regimes.
A new method uses mixture approximations to improve diffusion models for Bayesian inverse problems.
problem Approximating posterior distributions in Bayesian inverse problems with intractable likelihoods.
method Proposes a mixture-based approximation of intermediate posterior distributions and uses Gibbs sampling for practical sampling.
result Validated the approach on image inverse problems and audio source separation, demonstrating improved performance.
We derive a new discrepancy statistic for measuring differences between two probability distributions based on combining Stein's identity with the reproducing kernel Hilbert space theory. We apply our result to test how well a probabilistic model fits a set of observations, and derive a new class of powerful goodness-o…
Thompson sampling (TS) is a class of algorithms for sequential decision-making, which requires maintaining a posterior distribution over a model. However, calculating exact posterior distributions is intractable for all but the simplest models. Consequently, efficient computation of an approximate posterior distributio…
Python package for fast simulation-based inference.
problem Intractable likelihood functions in Bayesian inference.
method Uses neural networks as surrogate models for Bayesian inference.
result Highly efficient and user-friendly for constructing SBI estimators.
The exchange algorithm is studied for its convergence and asymptotic variance.
problem Theoretical limitations of the exchange algorithm in sampling from doubly-intractable distributions.
method Theoretical analysis of the exchange algorithm's convergence speed and asymptotic variance.
result The exchange algorithm converges at a geometric rate and satisfies a Central Limit Theorem.