ITF improves DSR but inflates curvature, while marginal likelihood reduces it, affecting QoIs.
problem Curvature mismatch between teacher forcing and marginal likelihood in chaotic dynamical systems.
method Comparing objective-induced curvatures of ITF and marginal likelihood in a probabilistic switching augmentation of AL-RNNs.
result Curvature inflation by ITF and reduction by marginal likelihood affect dynamical quantities of interest.
The study examines MCMC methods for arbitrary objectives and finds likelihood sharpness impacts performance and regularization.
problem Limitations of MCMC methods for arbitrary objective functions.
method Two-block MCMC framework with Metropolis-Hastings and Gibbs sampling, exploring likelihood curvature and sharpness.
result Likelihood sharpness governs in-sample performance and regularization inferred by training data.
Estimates curvature of network manifolds to understand community structure.
problem Understanding the geometry of network models to infer community structure.
method Develops hypothesis tests to determine manifold type, dimension, and curvature from noisy distance matrices.
result Consistently estimates manifold type, dimension, and curvature from Riemannian manifolds of constant curvature.
Variational Laplace improves Bayesian neural networks performance.
problem Improving Bayesian neural networks performance.
method Develops variational Laplace for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms other inference methods.
We consider a problem of ecological inference, in which individual-level covariates are known, but labeled data is available only at the aggregate level. The intended application is modeling voter preferences in elections. In Rosenman and Viswanathan (2018), we proposed modeling individual voter probabilities via a log…
PEP improves deep network performance and calibration by perturbing optimal parameters.
problem Improving deep network performance and calibration.
method Parameter Ensembling by Perturbation (PEP) constructs an ensemble of parameter values as random perturbations of the optimal set, maximizing log-likelihood on validation data.
result PEP provides a small to substantial improvement in calibration and log-likelihood, and in some cases, classification accuracy.
Variational Laplace improves Bayesian neural network performance without sampling.
problem Improving Bayesian neural network performance and calibration.
method Develops a new variational Laplace method for BNNs, exploiting curvature of likelihood.
result Variational Laplace outperforms standard VI methods in test performance and calibration.
ELU algorithm improves on EM for over-specified Gaussian mixtures.
problem Slow convergence of EM in over-specified Gaussian mixtures.
method Developed ELU algorithm for two-component mixtures, combining exponential location update and gradient descent.
result ELU converges to final statistical radius after logarithmic iterations, resolving open question.
Corrects pseudo log-likelihood method issues in various applications.
problem Log-likelihood function unbounded issues in pseudo log-likelihood methods.
method Provided a counterexample and corrected algorithms in previous literature.
result Ensured well-definedness of maximum pseudo log-likelihood estimation.
Due to the intractable partition function, the exact likelihood function for a Markov random field (MRF), in many situations, can only be approximated. Major approximation approaches include pseudolikelihood and Laplace approximation. In this paper, we propose a novel way of approximating the likelihood function throug…
A new method for training diffusion models using likelihood matching.
problem Training efficient and accurate diffusion models.
method Likelihood Matching approach, quasi-likelihood approximation, score and Hessian estimation.
result Consistent matching of first two transitional moments between diffusion steps.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
Bayesian models' singular fluctuation is shown to be akin to specific heat, influencing model complexity and generalization.
problem Understanding the thermodynamic interpretation of singular fluctuation in Bayesian models.
method Showed singular fluctuation as the curvature of Bayesian free energy and variance of log-likelihood observable under a Gibbs posterior.
result Singular fluctuation is the statistical analogue of specific heat, controlling model complexity and generalization.
Neural networks estimate spatial process likelihoods efficiently.
problem Challenges in estimating spatial processes with slow or intractable likelihoods.
method Convolutional neural networks trained on a classification task to learn likelihood function.
result Neural likelihood surfaces provide fast and accurate parameter estimation.
New method uses Gaussian ODE filtering to approximate likelihoods for fast ODE inverse problems.
problem Intractable forward models in likelihood-free inference, especially for ODEs.
method Gaussian ODE filtering to construct local Gaussian likelihood approximations.
result New solvers outperform standard likelihood-free approaches on benchmark systems.
Proposes efficient Gaussian approximations for non-Gaussian likelihoods.
problem Computational challenges in learning and inference with non-Gaussian likelihoods.
method Variational inference and moment matching in transformed bases.
result Good approximation quality for binary and multiclass classification.
Test log-likelihood comparisons can be misleading.
problem Misinterpretation of test log-likelihood in model comparison.
method Simple examples of model comparison and forecast accuracy.
result Test log-likelihood does not always correlate with model accuracy.
New analysis shows entropy term cancels out in likelihood-based OOD detection.
problem Curious likelihood values for out-of-distribution data.
method Decomposed average likelihood into KL divergence and entropy terms.
result Entropy term explains OOD behaviour and cancels out in expectation.
New method resolves nonidentifiability in mixture models.
problem Nonidentifiability in marginal models of mixture models.
method Introducing an effective temperature to generalize the marginal likelihood.
result Maximization of the generalized likelihood leads to unique results.
Neural networks approximate likelihood ratios for complex models.
problem Difficulty in computing likelihood ratios for modern models.
method Applying the likelihood ratio trick with neural network classifiers.
result Different neural network setups can approximate likelihood ratios with varying performance.
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to explicit models trained by maximum likelihood. Yet, GANs sidestep the characterization of an explici…
Proposes LFGP for likelihood-free Gaussian process regression.
problem Inability to set likelihood functions in unknown probability models.
method Clusters and approximates likelihood using asymptotic normality.
result Reduces assumptions and computational costs for scalable problems.
Develops methods for constructing likelihoods and priors for Bayesian networks.
problem Learning parameters and structure of Bayesian networks from limited data.
method Introduces assumptions for constructing likelihoods and priors from small assessments.
result Allows construction of likelihoods and priors for a wide range of network structures.
New approach combines likelihood and adversarial losses for better precipitation predictions.
problem Spatially inconsistent precipitation projections from likelihood-based models.
method Fuses likelihood-based and adversarial losses for generative models.
result Improves spatial consistency in precipitation downscaling.
Bayesian autoencoders improve OOD detection by addressing Bernoulli likelihood issues.
problem Out-of-distribution (OOD) detection fails with Bernoulli likelihood for certain datasets.
method Proposes Bayesian autoencoders and alternative likelihood models to fix the issue.
result Bayesian autoencoders and alternative likelihood models improve OOD detection accuracy.
This paper develops embeddings that preserve likelihood-based statistical inference.
problem Modern machine learning embeddings destroy the geometric structure required for likelihood-based inference.
method Developed a rigorous theory of likelihood-preserving embeddings and introduced the Likelihood-Ratio Distortion metric.
result Controlling the distortion Δn is necessary and sufficient for preserving inference. This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
New method uses path signatures for efficient likelihood estimation in time-series data.
problem Intractable likelihood functions in complex dynamic models.
method Kernel classifier based on path signatures for sequential data.
result Path signatures yield highly performant classifiers, even with low sample numbers.
Bayesian evidence helps compare models but can overfit.
problem Comparing hypotheses consistent with observations.
method Marginal likelihood, Occam's razor, PAC-Bayes bounds.
result Marginal likelihood can negatively correlate with generalization.
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
R package for Bayesian empirical likelihood sampling using HMC.
problem Sampling from non-convex Bayesian empirical likelihood posteriors.
method Hamiltonian Monte Carlo (HMC) algorithm for numerical integration.
result Efficient HMC sampling from BayesEL posteriors.
A method merges two pretrained diffusion experts to improve image quality and likelihood.
problem Trade-off between image quality and data likelihood in diffusion models.
method Combining two pretrained diffusion experts by switching between them along the denoising trajectory.
result The merged model consistently matches or outperforms its base components, improving or preserving both likelihood and sample quality.
In likelihood-free settings where likelihood evaluations are intractable, approximate Bayesian computation (ABC) addresses the formidable inference task to discover plausible parameters of simulation programs that explain the observations. However, they demand large quantities of simulation calls. Critically, hyperpara…
A new method normalizes EBM training by introducing a learnable parameter.
problem Training energy-based models with maximum likelihood is challenging due to intractable normalisation constants.
method Proposes a self-normalised log-likelihood (SNL) objective that introduces a learnable parameter representing the normalisation constant.
result The SNL objective is a lower bound of the log-likelihood and can be directly optimised using stochastic gradient techniques.
The inference of deep hierarchical models is problematic due to strong dependencies between the hierarchies. We investigate a specific transformation of the model parameters based on the multivariate distributional transform. This transformation is a special form of the reparametrization trick, flattens the hierarchy a…
Unified contrastive learning for likelihood-free inference.
problem Parameter inference in models with intractable likelihood.
method Unified contrastive learning scheme for both density ratio and direct posterior estimation.
result Unified approach clarifies method selection and comparison.
Proposes Likelihood Regret for VAEs to improve OOD detection.
problem VAEs can assign high likelihoods to OOD samples, making traditional likelihood thresholds unreliable.
method Introduces Likelihood Regret, a new OOD score for VAEs.
result Empirical results show Likelihood Regret outperforms existing methods for VAEs.
Resultant improves likelihood-based U-OOD detection across various tasks.
problem Improving likelihood-based U-OOD detection performance.
method Resultant combines post-hoc prior and dataset entropy-mutual calibration techniques.
result Resultant achieves new state-of-the-art U-OOD detection performance.
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of th…
Direct neural ratio estimator for likelihood-free inference.
problem Efficient likelihood estimation for complex models.
method Amortized likelihood ratio estimation using neural networks.
result DNRE often outperforms previous ratio estimators.
This research improves neural likelihood approximation for Bayesian inverse problems.
problem Challenges in modeling and inference for high-dimensional Bayesian inverse problems.
method Develops a strictly convex approximation framework for neural likelihood.
result Empirical minimizers converge to the true likelihood as sample size increases.
Maximum likelihood estimation fails to be well-posed in Gaussian process regression.
problem Establishing well-posedness of maximum likelihood estimation in Gaussian process regression.
method Analyzing the conditions under which maximum likelihood estimation is not Lipschitz in the data with respect to the Hellinger distance.
result Maximum likelihood estimation is not well-posed in the noiseless data setting for any Gaussian process with a stationary covariance function whose lengthscale parameter is estimated using maximum likelihood.
Geometrically, high-likelihood regions in DGMs are unlikely to generate OOD data.
problem The paradox of high-likelihood OOD detection in deep generative models.
method Local intrinsic dimension estimation to identify high-likelihood regions that do not generate OOD data.
result A method pairing likelihoods and LID estimates for reliable OOD detection.
A new VIS approach improves log-likelihood estimation in latent variable models.
problem Challenges in achieving high log-likelihood with VI for complex posterior distributions.
method Uses forward χ2 divergence to optimize proposal distribution for better log-likelihood estimation. result Consistently outperforms state-of-the-art baselines in log-likelihood and parameter estimation.
New method uses neural exponential families for likelihood-free inference.
problem Bayesian Likelihood-Free Inference with intractable likelihood.
method Score Matching neural conditional exponential families for approximate likelihood.
result State-of-the-art performance in posterior sampling for intractable likelihood models.
In many fields of science, generalized likelihood ratio tests are established tools for statistical inference. At the same time, it has become increasingly common that a simulator (or generative model) is used to describe complex processes that tie parameters θ of an underlying theory and measurement apparatus to hig…
Two synthetic likelihood methods learn EBM of likelihood from simulator data for SBI.
problem Conduct inference from experimental observations using high-fidelity simulators.
method Learn conditional EBM of likelihood using synthetic data conditioned on parameters.
result Learned likelihood combined with prior yields posterior estimate for sampling.
Directly estimates Fisher score for likelihood maximization.
problem Intractable likelihood functions with model simulations.
method Gradient-based optimization using local score matching and linear parameterization.
result Efficient approximation of Fisher score improves likelihood maximization.