Neural networks estimate spatial process likelihoods efficiently.
problem Challenges in estimating spatial processes with slow or intractable likelihoods.
method Convolutional neural networks trained on a classification task to learn likelihood function.
result Neural likelihood surfaces provide fast and accurate parameter estimation.
Bayesian Quadrature improves ensembling for neural networks with dispersed likelihood peaks.
problem Ensembling neural networks struggles with dispersed, narrow peaks in likelihood surfaces.
method Uses Bayesian Quadrature to construct weighted ensembles of architectures.
result Empirically outperforms state-of-the-art baselines in test likelihood, accuracy, and expected calibration error.
Performing inference over simulators is generally intractable as their runtime means we cannot compute a marginal likelihood. We develop a likelihood-free inference method to infer parameters for a cardiac simulator, which replicates electrical flow through the heart to the body surface. We improve the fit of a state-o…
ManiFlow models manifold data by optimizing NFs on perturbed data.
problem Capturing manifold data with NFs' invertibility constraint.
method Train NFs on perturbed data to implicitly represent manifold.
result NFs implicitly model manifold in regions of maximum likelihood.
Mixed likelihood GPs improve model performance in human-in-the-loop experiments.
problem Lack of auxiliary information in traditional GPs for human responses.
method Propose mixed likelihood variational GPs to leverage auxiliary information.
result Modeling performance improvements across diverse applications.
In this paper we present an algorithm for rapid Bayesian analysis that combines the benefits of nested sampling and artificial neural networks. The blind accelerated multimodal Bayesian inference (BAMBI) algorithm implements the MultiNest package for nested sampling as well as the training of an artificial neural netwo…
Improved MCMC sampling for expensive, irregular likelihoods.
problem Bayesian inference challenges with irregular, expensive likelihoods.
method Adapt subset samplers, introduce data-driven proxies, adaptive controller.
result Improved HINTS algorithm achieves best sampling error in fixed budget.
Knot-based, sparse Gaussian processes have enjoyed considerable success as scalable approximations to full Gaussian processes. Problems can occur, however, when knot selection is done by optimizing the marginal likelihood. For example, the marginal likelihood surface is highly multimodal, which can cause suboptimal kno…
Neural point estimators improve parameter estimation from replicated data.
problem Making inference from replicated data in weakly-identified and highly-parameterised models.
method Permutation-invariant neural networks for likelihood-free parameter estimation.
result Neural point estimators can quickly and optimally estimate parameters.
We present an improved Bayesian framework for performing inference of affine transformations of constrained functions. We focus on quadrature with nonnegative functions, a common task in Bayesian inference. We consider constraints on the range of the function of interest, such as nonnegativity or boundedness. Although …
In a range of fields including the geosciences, molecular biology, robotics and computer vision, one encounters problems that involve random variables on manifolds. Currently, there is a lack of flexible probabilistic models on manifolds that are fast and easy to train. We define an extremely flexible class of exponent…
Adapting a construction of D Salamon involving the U(1) vortex equations, we explore the properties of a Floer theory for 3-manifolds that fiber over S^1 which exhibits several parallels with monopole Floer homology, and in all likelihood coincides with it. The theory fits into a restricted analogue of a TQFT in which …
The expectation-maximization (EM) algorithm is an iterative method for finding maximum likelihood estimates when data are incomplete or are treated as being incomplete. The EM algorithm and its variants are commonly used for parameter estimation in applications of mixture models for clustering and classification. This …
Networks, which represent agents and interactions between them, arise in myriad applications throughout the sciences, engineering, and even the humanities. To understand large-scale structure in a network, a common task is to cluster a network's nodes into sets called "communities", such that there are dense connection…
Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
Generative AI decodes quantum codes without labeled data.
problem Efficient decoding of quantum error-correcting codes.
method Generative Transformers learn logical operators from unsupervised syndromes.
result Significantly better decoding accuracy than traditional methods.
Due to the intractable partition function, the exact likelihood function for a Markov random field (MRF), in many situations, can only be approximated. Major approximation approaches include pseudolikelihood and Laplace approximation. In this paper, we propose a novel way of approximating the likelihood function throug…
A new method for training diffusion models using likelihood matching.
problem Training efficient and accurate diffusion models.
method Likelihood Matching approach, quasi-likelihood approximation, score and Hessian estimation.
result Consistent matching of first two transitional moments between diffusion steps.
Proposes a new prior for VAEs to improve out-of-distribution detection.
problem Probabilistic generative models struggle with out-of-distribution detection.
method Introduces an exponentially tilted Gaussian prior for VAEs.
result Achieves state-of-the-art results on ROC-AUC metric.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
Proposes efficient Gaussian approximations for non-Gaussian likelihoods.
problem Computational challenges in learning and inference with non-Gaussian likelihoods.
method Variational inference and moment matching in transformed bases.
result Good approximation quality for binary and multiclass classification.
Test log-likelihood comparisons can be misleading.
problem Misinterpretation of test log-likelihood in model comparison.
method Simple examples of model comparison and forecast accuracy.
result Test log-likelihood does not always correlate with model accuracy.
New analysis shows entropy term cancels out in likelihood-based OOD detection.
problem Curious likelihood values for out-of-distribution data.
method Decomposed average likelihood into KL divergence and entropy terms.
result Entropy term explains OOD behaviour and cancels out in expectation.
Neural networks approximate likelihood ratios for complex models.
problem Difficulty in computing likelihood ratios for modern models.
method Applying the likelihood ratio trick with neural network classifiers.
result Different neural network setups can approximate likelihood ratios with varying performance.
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…
Proposes LFGP for likelihood-free Gaussian process regression.
problem Inability to set likelihood functions in unknown probability models.
method Clusters and approximates likelihood using asymptotic normality.
result Reduces assumptions and computational costs for scalable problems.
Likelihood-free (a.k.a. simulation-based) inference problems are inverse problems with expensive, or intractable, forward models. ODE inverse problems are commonly treated as likelihood-free, as their forward map has to be numerically approximated by an ODE solver. This, however, is not a fundamental constraint but jus…
Develops methods for constructing likelihoods and priors for Bayesian networks.
problem Learning parameters and structure of Bayesian networks from limited data.
method Introduces assumptions for constructing likelihoods and priors from small assessments.
result Allows construction of likelihoods and priors for a wide range of network structures.
We study the parameter estimation problem in mixture models with observational nonidentifiability: the full model (also containing hidden variables) is identifiable, but the marginal (observed) model is not. Hence global maxima of the marginal likelihood are (infinitely) degenerate and predictions of the marginal likel…
New approach combines likelihood and adversarial losses for better precipitation predictions.
problem Spatially inconsistent precipitation projections from likelihood-based models.
method Fuses likelihood-based and adversarial losses for generative models.
result Improves spatial consistency in precipitation downscaling.
Bayesian autoencoders improve OOD detection by addressing Bernoulli likelihood issues.
problem Out-of-distribution (OOD) detection fails with Bernoulli likelihood for certain datasets.
method Proposes Bayesian autoencoders and alternative likelihood models to fix the issue.
result Bayesian autoencoders and alternative likelihood models improve OOD detection accuracy.
This paper develops embeddings that preserve likelihood-based statistical inference.
problem Modern machine learning embeddings destroy the geometric structure required for likelihood-based inference.
method Developed a rigorous theory of likelihood-preserving embeddings and introduced the Likelihood-Ratio Distortion metric.
result Controlling the distortion Δn is necessary and sufficient for preserving inference. This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
New method uses path signatures for efficient likelihood estimation in time-series data.
problem Intractable likelihood functions in complex dynamic models.
method Kernel classifier based on path signatures for sequential data.
result Path signatures yield highly performant classifiers, even with low sample numbers.
Bayesian evidence helps compare models but can overfit.
problem Comparing hypotheses consistent with observations.
method Marginal likelihood, Occam's razor, PAC-Bayes bounds.
result Marginal likelihood can negatively correlate with generalization.
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
R package for Bayesian empirical likelihood sampling using HMC.
problem Sampling from non-convex Bayesian empirical likelihood posteriors.
method Hamiltonian Monte Carlo (HMC) algorithm for numerical integration.
result Efficient HMC sampling from BayesEL posteriors.
A method merges two pretrained diffusion experts to improve image quality and likelihood.
problem Trade-off between image quality and data likelihood in diffusion models.
method Combining two pretrained diffusion experts by switching between them along the denoising trajectory.
result The merged model consistently matches or outperforms its base components, improving or preserving both likelihood and sample quality.
In likelihood-free settings where likelihood evaluations are intractable, approximate Bayesian computation (ABC) addresses the formidable inference task to discover plausible parameters of simulation programs that explain the observations. However, they demand large quantities of simulation calls. Critically, hyperpara…
A new algorithm is developed to tackle the issue of sampling non-Gaussian model parameter posterior probability distributions that arise from solutions to Bayesian inverse problems. The algorithm aims to mitigate some of the hurdles faced by traditional Markov Chain Monte Carlo (MCMC) samplers, through constructing pro…
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
Proposes Likelihood Regret for VAEs to improve OOD detection.
problem VAEs can assign high likelihoods to OOD samples, making traditional likelihood thresholds unreliable.
method Introduces Likelihood Regret, a new OOD score for VAEs.
result Empirical results show Likelihood Regret outperforms existing methods for VAEs.
Resultant improves likelihood-based U-OOD detection across various tasks.
problem Improving likelihood-based U-OOD detection performance.
method Resultant combines post-hoc prior and dataset entropy-mutual calibration techniques.
result Resultant achieves new state-of-the-art U-OOD detection performance.
The paper extends MS models with TVTP to U.S. Treasury yields, finding reliable regime dynamics but challenging TVTP identification.
problem Identifying time-varying transition probabilities in Markov-switching models for U.S. Treasury yields.
method Developed a comprehensive MS model with TVTP, including simulations and an R package for estimation.
result Regime means, variances, and transition probabilities are reliably identified, but TVTP coefficients are harder to estimate.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
Direct neural ratio estimator for likelihood-free inference.
problem Efficient likelihood estimation for complex models.
method Amortized likelihood ratio estimation using neural networks.
result DNRE often outperforms previous ratio estimators.
This research improves neural likelihood approximation for Bayesian inverse problems.
problem Challenges in modeling and inference for high-dimensional Bayesian inverse problems.
method Develops a strictly convex approximation framework for neural likelihood.
result Empirical minimizers converge to the true likelihood as sample size increases.
Maximum likelihood estimation fails to be well-posed in Gaussian process regression.
problem Establishing well-posedness of maximum likelihood estimation in Gaussian process regression.
method Analyzing the conditions under which maximum likelihood estimation is not Lipschitz in the data with respect to the Hellinger distance.
result Maximum likelihood estimation is not well-posed in the noiseless data setting for any Gaussian process with a stationary covariance function whose lengthscale parameter is estimated using maximum likelihood.