This work improves neural likelihood surrogates for stochastic models with a score-augmented loss.
problem Efficient parameter inference for stochastic models with computationally expensive likelihood functions.
method Score-augmented loss function for neural network likelihood surrogates.
result Improves surrogate quality at a lower computational cost compared to generating more data.
Deep Gaussian Processes improve likelihood-free inference for complex distributions.
problem Limited flexibility of Bayesian Optimization with GPs for multimodal distributions.
method Proposes Deep Gaussian Processes (DGPs) as a surrogate model for likelihood-free inference.
result DGPs outperform GPs on multimodal distributions while maintaining comparable performance on unimodal cases.
Gaussian surrogates improve Poisson imaging performance at low doses.
problem Improving Poisson imaging performance at low doses.
method Analysis of Poisson and Gaussian surrogate reconstruction objectives under Poisson noise.
result Gaussian surrogates can achieve MSE comparable to Poisson MAP at low doses.
We extend recent work (Brehmer, et. al., 2018) that use neural networks as surrogate models for likelihood-free inference. As in the previous work, we exploit the fact that the joint likelihood ratio and joint score, conditioned on both observed and latent variables, can often be extracted from an implicit generative m…
We consider Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained. This occurs for example when complex simulator-based statistical models are fitted to data, and synthetic likelihood (SL) method is used to form the noisy log-likelihood estimates using computationally costly …
A new method extends Bayesian optimization to more models and utilities.
problem Extending Bayesian optimization to a broader class of models and utilities.
method Likelihood-free Bayesian Optimization (LFBO) which directly models the acquisition function without separate inference.
result LFBO outperforms state-of-the-art black-box optimization methods on real-world problems.
We propose a novel method for maximum likelihood-based parameter inference in nonlinear and/or non-Gaussian state space models. The method is an iterative procedure with three steps. At each iteration a particle filter is used to estimate the value of the log-likelihood function at the current parameter iterate. Using …
Neural surrogate predicts SPN rates from token trajectories.
problem Challenging parameter estimation in SPNs with covariates.
method 1D Convolutional Residual Network trained on Gillespie-simulated SPN realizations.
result Surrogate predicts rate-function coefficients with RMSE = 0.043.
Gaussian process regression helps approximate Bayesian inverse problems efficiently.
problem Computational intractability of Bayesian posterior distributions in inverse problems.
method Gaussian process regression to build a surrogate model for the likelihood.
result Error between true and approximate posterior can be bounded by weighted L2-norm error between true and approximate likelihood. This study proposes an efficient surrogate for Darcy flow inverse problems.
problem Efficiently constructing accurate surrogate models for high-dimensional complex inverse problems.
method Sequential Bayesian design strategy to acquire a locally accurate surrogate model focusing on high-probability regions.
result The proposed method accelerates inversion accuracy and computational speed.
A new method combines variational inference and MCMC for efficient data subsampling.
problem Combining variational inference and MCMC for data subsampling.
method Introduces a surrogate likelihood to learn jointly with variational parameters.
result Permits an intuitive trade-off between inference fidelity and computational cost.
NoFAS combines variational inference and adaptive surrogate models for efficient inference of computationally expensive models.
problem Efficient inference of parameters from data with computationally expensive models.
method Variational inference with normalizing flow and adaptive surrogate model training.
result NoFAS reduces computational cost without sacrificing inferential accuracy.
Efficient Bayesian decision-making with intractable likelihoods.
problem Bayesian decision-making under intractable likelihoods.
method Learning surrogate models and using simulation-based inference and Bayesian optimization.
result Optimal actions can be learned with fewer simulations than posterior inference.
Simulators often provide the best description of real-world phenomena. However, they also lead to challenging inverse problems because the density they implicitly define is often intractable. We present a new suite of simulation-based inference techniques that go beyond the traditional Approximate Bayesian Computation …
Paper proposes a surrogate model for efficient experience rating in large insurance portfolios.
problem Inexpensive and transparent computation of Bayesian premiums for large insurance portfolios.
method Surrogate modeling approach using likelihood-based summary statistics.
result Reduced computational burden and provided a transparent way of computing Bayesian premiums.
Estimates log-likelihood of interacting particle systems using virtual particles.
problem Inconsistent estimation of finite-particle log-likelihood in large particle systems.
method Stochastic gradient estimate using continuous trajectory and virtual particle systems.
result Convergence to stationary points of limiting mean-field system's log-likelihood.
VBMC+VIQR outperforms noisy models in Bayesian inference.
problem Bayesian inference with noisy likelihoods in complex models.
method Gaussian process surrogates, expected information gain, variational interquantile range.
result VBMC+VIQR achieves state-of-the-art performance in noisy inference benchmarks.
Likelihood-free inference refers to inference when a likelihood function cannot be explicitly evaluated, which is often the case for models based on simulators. Most of the literature is based on sample-based `Approximate Bayesian Computation' methods, but recent work suggests that approaches based on deep neural condi…
ITF improves DSR but inflates curvature, while marginal likelihood reduces it, affecting QoIs.
problem Curvature mismatch between teacher forcing and marginal likelihood in chaotic dynamical systems.
method Comparing objective-induced curvatures of ITF and marginal likelihood in a probabilistic switching augmentation of AL-RNNs.
result Curvature inflation by ITF and reduction by marginal likelihood affect dynamical quantities of interest.
A new method uses natural gradients for efficient distribution optimization.
problem Challenges in computing natural gradients for many distributions.
method Reframe optimization as a surrogate distribution with easy natural gradient computation.
result Expands set of distributions efficiently targetable with natural gradients.
We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For…
ML surrogates speed up Bayesian inverse problem solving.
problem Infer source location from noisy acoustic wave equation data.
method Use neural network as surrogate for PDE, apply MCMC to posterior.
result Accurately infers source location from noisy data.
Survey of Monte Carlo methods for noisy, costly densities in reinforcement learning and ABC.
problem Dealing with intractable, costly, and noisy densities in real-world scenarios.
method Classification and description of Monte Carlo methodologies using surrogate models.
result Unified scheme and numerical comparisons of different methodologies.
New method speeds up Bayesian inverse problem solving with neural operators.
problem Solving infinite-dimensional Bayesian inverse problems with high computational cost.
method Delayed-acceptance geometric MCMC driven by derivative-informed neural operator surrogates.
result Significant speedup in generating posterior samples (3-9 times faster).
In likelihood-free settings where likelihood evaluations are intractable, approximate Bayesian computation (ABC) addresses the formidable inference task to discover plausible parameters of simulation programs that explain the observations. However, they demand large quantities of simulation calls. Critically, hyperpara…
SGE-Kriging reduces high-dimensional surrogate modelling costs.
problem High-dimensional function approximation for expensive models.
method Splitting training data into slices, using sliced likelihood function, and learning hyper-parameters from sensitivity indices.
result SGE-Kriging achieves comparable accuracy to standard GE-Kriging but with lower training costs.
A new meta-learning BO approach that bypasses surrogate models and directly learns task utility.
problem Scalability issues and sensitivity to task similarity in existing meta-learning BO methods.
method Directly learns the utility of queries across tasks, models task uncertainty, and includes an auxiliary model for robust adaptation.
result Demonstrates strong anytime performance and outperforms state-of-the-art methods in various benchmarks.
Traditional structured prediction models try to learn the conditional likelihood, i.e., p(y|x), to capture the relationship between the structured output y and the input features x. For many models, computing the likelihood is intractable. These models are therefore hard to train, requiring the use of surrogate objecti…
When learning from a batch of logged bandit feedback, the discrepancy between the policy to be learned and the off-policy training data imposes statistical and computational challenges. Unlike classical supervised learning and online learning settings, in batch contextual bandit learning, one only has access to a colle…
Bayesian method refines surrogate models for accurate full waveform inversion.
problem Complex input/output relations in full waveform inversion make accurate surrogate models difficult.
method Iterative refinement of surrogate models using MCMC samples and progressively expanding frequency bandwidth.
result Highly accurate surrogate model across full bandwidth enables accurate final MCMC inversion.
Bayesian optimization has recently attracted the attention of the automatic machine learning community for its excellent results in hyperparameter tuning. BO is characterized by the sample efficiency with which it can optimize expensive black-box functions. The efficiency is achieved in a similar fashion to the learnin…
PDMP samplers improve Bayesian PDE coefficient inference.
problem Efficient Bayesian inference in non-linear inverse problems with expensive likelihoods.
method Piecewise deterministic Markov process (PDMP) with surrogate-assisted thinning.
result PDMP samplers achieve higher accuracy and efficiency than traditional methods.
Directly estimates Fisher score for likelihood maximization.
problem Intractable likelihood functions with model simulations.
method Gradient-based optimization using local score matching and linear parameterization.
result Efficient approximation of Fisher score improves likelihood maximization.
Self-consistency improves the accuracy of model comparison methods.
problem Improving the accuracy of model comparison methods when simulation models are misspecified.
method Supplement traditional simulation-based training with a self-consistency loss on unlabeled real data.
result Self-consistency training improves model comparison accuracy, especially in open-world scenarios.
When trained effectively, the Variational Autoencoder (VAE) is both a powerful language model and an effective representation learning framework. In practice, however, VAEs are trained with the evidence lower bound (ELBO) as a surrogate objective to the intractable marginal data likelihood. This approach to training yi…
New GP kernel handles mixed-categorical data, improving model accuracy.
problem Improving Gaussian process models for mixed-categorical data.
method Extends continuous exponential kernels to handle mixed-categorical variables.
result The proposed GP model gives higher likelihood and smaller residual error.
We propose a novel method for gradient-based optimization of black-box simulators using differentiable local surrogate models. In fields such as physics and engineering, many processes are modeled with non-differentiable simulators with intractable likelihoods. Optimization of these forward models is particularly chall…
Generative AI improves surrogate models by blending LF and HF data.
problem Data scarcity between high-fidelity and low-fidelity simulations.
method Probabilistic multi-fidelity surrogate framework using generative transfer learning.
result The model achieves HF accuracy with fewer HF evaluations.
Bayesian method improves approximate model posteriors.
problem Poor uncertainty quantification in approximate Bayesian inference.
method Optimizing a transformation of the approximate posterior to maximize a scoring rule.
result Significant reduction in bias and improvement in posterior coverage properties.
A deep-learning-based surrogate model is developed and applied for predicting dynamic subsurface flow in channelized geological models. The surrogate model is based on deep convolutional and recurrent neural network architectures, specifically a residual U-Net and a convolutional long short term memory recurrent networ…
New method improves simulation-based inference by avoiding model misspecification.
problem Inefficient parameter estimation for models with intractable likelihoods.
method Proposes a robust SNL method with additional adjustment parameters.
result Demonstrates more accurate point estimates and uncertainty quantification.
We consider Bayesian inference problems with computationally intensive likelihood functions. We propose a Gaussian process (GP) based method to approximate the joint distribution of the unknown parameters and the data. In particular, we write the joint density approximately as a product of an approximate posterior dens…
Engine for Likelihood-Free Inference (ELFI) is a Python software library for performing likelihood-free inference (LFI). ELFI provides a convenient syntax for arranging components in LFI, such as priors, simulators, summaries or distances, to a network called ELFI graph. The components can be implemented in a wide vari…
A new framework reduces inconsistencies in chaotic surrogate modeling.
problem Consistency issues between probabilistic objectives and dynamical system dynamics.
method KAFFEE (Kalman-Aware Framework For Ergodic Emulation), a differentiable extended Kalman filter.
result KAFFEE mitigates the dynamic-probabilistic consistency gap, improving reconstruction and predictive scores.
The complex and computationally expensive nature of landscape evolution models pose significant challenges in the inference and optimisation of unknown parameters. Bayesian inference provides a methodology for estimation and uncertainty quantification of unknown model parameters. In our previous work, we developed para…
This paper develops embeddings that preserve likelihood-based statistical inference.
problem Modern machine learning embeddings destroy the geometric structure required for likelihood-based inference.
method Developed a rigorous theory of likelihood-preserving embeddings and introduced the Likelihood-Ratio Distortion metric.
result Controlling the distortion Δn is necessary and sufficient for preserving inference. When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the …
Developed efficient distributed logistic regression for large datasets.
problem Communication inefficiency and sparsity issues in distributed training of large-scale models.
method Iterative local optimization of a surrogate likelihood to improve initial solutions, handling sparsity and diverging updates.
result Learned a communication-efficient distributed logistic regression model for millions of features.