This paper improves Bayesian inference for predictive models with limited data.
problem Effective uncertainty quantification for training predictive models with limited data.
method Entropy-regularized gradient estimators to approximate the Bayesian posterior.
result The method generates diverse samples from the posterior distribution efficiently.
The paper develops fast Bayesian methods for estimating huge PVARs with competitive forecasts.
problem Computational and statistical issues in estimating PVARs with many parameters.
method Integrated rotated Gaussian approximations, exploiting domestic over international information, and fast approximations for international coefficients.
result Produces competitive forecasts quickly using a huge world economy model.
This study uses neural networks to approximate Bayesian filtering problems.
problem Estimating latent time-series signal statistics from observation sequences.
method Formulated a generic recurrent neural network framework to learn recursive mappings directly.
result Approximation error bounds for filtering in non-compact domains and strong time-uniform bounds.
GPNs use unlabeled data to estimate uncertainty in Bayesian problems.
problem Limited training data in high-dimensional problems.
method Generative Posterior Networks (GPNs) that approximate the posterior distribution using unlabeled data.
result GPNs improve epistemic uncertainty estimation and scalability.
A novel Laplace-approximated Bayesian Tensor Network Kernel Machine (LA-TNKM) provides principled uncertainty estimates.
problem How to provide principled uncertainty estimates for tensor network kernel machines.
method Employing a linearized Laplace approximation for Bayesian inference.
result Consistently matches or surpasses Gaussian Processes and BNNs across diverse UCI regression benchmarks.
New method for density estimation without approximating posterior distributions.
problem Challenges in non-smooth data distributions for Bayesian density estimation.
method Autoregressive likelihood decomposition and Gaussian process prior in a quasi-Bayesian framework.
result Achieves state-of-the-art results in small-data regimes.
Bayesian model averaging improves causal effect estimation by averaging over multiple models.
problem Estimating causal effects under linear Structural Causal Models (SCMs).
method Bayesian model averaging using Gaussian scale mixture distributions for computational efficiency.
result Bayesian model averaging is optimal for causal effect estimation.
Bayesian meta-reinforcement learning improves over point estimates with Laplace approximation.
problem Improving meta-reinforcement learning by providing full posterior distributions.
method Augmenting point estimates with Laplace approximation for full posterior distributions.
result Our method performs similarly to variational baselines with fewer parameters.
The point estimates of ReLU classification networks---arguably the most widely used neural network architecture---have been shown to yield arbitrarily high confidence far away from the training data. This architecture, in conjunction with a maximum a posteriori estimation scheme, is thus not calibrated nor robust. Appr…
Stochastic Volatility in Mean models with heavy-tailed distributions using Hidden Markov Models
problem Accurate inference for Stochastic Volatility in Mean models with heavy-tailed distributions
method Numerically stable estimation procedure and parallel computing
result Significant reduction in computational times
Estimates expected information gain using density approximations and dimension reduction.
problem Estimating expected information gain in nonlinear and non-Gaussian settings.
method Flexible transport-based schemes for EIG estimation, optimal sample allocation, and gradient-based upper bounds on mutual information.
result Optimal sample allocation and dimension reduction schemes improve EIG estimation accuracy and convergence rate.
Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty estimates. Classical Monte Carlo methods, particularly Markov Chain Monte Carlo, rem…
We develop a scalable method for Bayesian neural networks with stochastic differential equations.
problem Uncertainty quantification in deep neural networks.
method Gradient-based stochastic variational inference in continuous-depth Bayesian neural networks.
result Gradient estimator with zero variance as the approximation improves.
The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation. In this article, we consider Gaussian latent variable models where the integration over the latent values is approximated using the Laplace method or expectation propagation (EP). We study the properties of several B…
Simulation-based inference methods can produce unreliable posterior approximations.
problem Reliability of simulation-based inference methods for scientific use cases.
method Benchmarked algorithms including Neural Posterior Estimation, Neural Ratio Estimation, Sequential Neural Likelihood, and Approximate Bayesian Computation.
result Ensembling posterior surrogates provides more reliable approximations.
Bayesian deep learning method using subnetwork inference.
problem Improving deep neural networks' calibration and efficiency.
method Perform inference over a subset of model weights, keeping others as point estimates.
result Subnetwork inference enables accurate predictive posteriors without full network approximations.
Generative approach speeds hyperparameter tuning for machine learning models.
problem Computational infeasibility of cross-validation and difficulty of fully Bayesian hyper-parameter learning.
method Combines optimization-based approximations and amortization techniques.
result Rapid evaluation of hyper-parameters over grids or ranges, supporting predictive tuning and uncertainty quantification.
Develops a new Gaussian process method for efficient Bayesian inference of plant root parameters in the Richards equation.
problem Estimating unknown parameters in nonlinear PDEs for agricultural studies.
method Gaussian process collocation with importance sampling and Bayesian optimization.
result Our method yields robust estimates with uncertainty quantification for plant root parameters.
New method improves ABC for Bayesian model comparison.
problem Comparing complex models with observed data.
method Approximate Bayesian Computation with posterior density estimation.
result Efficiently assigns high posterior probabilities to ground-truth models.
Novel method recursively partitions sample space for density estimation.
problem Estimating complex density functions efficiently and accurately.
method Recursive partitioning of the sample space, asymptotically exact.
result Asymptotically exact approximation of any density function.
QLA improves Bayesian uncertainty estimation for DNNs without increasing computational cost.
problem Overconfident out-of-distribution predictions from DNNs.
method Proposes Quadratic Laplace Approximation (QLA) to improve Bayesian uncertainty quantification.
result QLA yields modest yet consistent uncertainty estimation improvements over Linearized Laplace Approximation (LLA) on five regression datasets.
New variational inference approach using Hilbert space for robotic state estimation.
problem Robotic state estimation with high-dimensional data.
method Variational inference reformulated in a Bayesian Hilbert space, using iterative projection.
result Variational inference can be seen as iterative projection in Euclidean space.
Bayesian inference using stochastic neural networks ensembles.
problem Approximating Bayesian posterior distributions.
method Formulate stochastic ensembles of neural networks, train with variational inference, and evaluate using Monte Carlo dropout.
result Stochastic ensembles provide more accurate posterior estimates than other methods.
A new method reduces dimensionality for better likelihood-free parameter estimation.
problem Estimating parameters from data with no closed-form likelihood.
method Combines reconstruction map estimation with dimension-reduction techniques.
result The proposed method outperforms existing techniques in accuracy and efficiency.
We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. Th…
New methods improve statistical accuracy of complex models without high computational cost.
problem Improving statistical accuracy of complex models without high computational cost.
method Neural posterior and likelihood estimation (NPE and NLE) methods.
result NPE and NLE methods have similar theoretical guarantees to ABC and BSL, but achieve accuracy at a reduced computational cost.
Approximate Bayesian computation (ABC) is a method for Bayesian inference when the likelihood is unavailable but simulating from the model is possible. However, many ABC algorithms require a large number of simulations, which can be costly. To reduce the computational cost, Bayesian optimisation (BO) and surrogate mode…
Bayesian model selection via mean-field variational approximation improves efficiency and accuracy.
problem Bayesian model selection under model mis-specification and latent variables.
method Mean-field variational approximation with non-asymptotic properties and geometric convergence.
result ELBO tends to select models closer to the true model than BIC as sample size increases.
Bayesian state and parameter estimation for nonlinear models using variational methods.
problem Estimating states and parameters for nonlinear state-space models.
method Variational approach to approximate the intractable Bayesian distribution, resulting in an optimisation problem.
result The proposed method efficiently computes Bayesian estimates for nonlinear models, outperforming Hamiltonian Monte Carlo in numerical examples.
Bayesian model averaging fails under covariate shift, affecting neural networks' performance.
problem Bayesian model averaging's failure in neural networks under covariate shift.
method Explained the issue and proposed novel priors to improve robustness.
result Bayesian model averaging is problematic under covariate shift, especially with linear feature dependencies.
Bayesian neural networks speed up numerical integration.
problem Scalability of Bayesian quadrature methods.
method Bayesian Stein networks using neural networks and Laplace approximation.
result Orders of magnitude speed-up on benchmark functions and real-world problems.
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
This paper optimizes Bayesian estimation for log-concave models using Langevin Monte-Carlo.
problem Optimizing Bayesian estimators for log-concave models with Langevin Monte-Carlo.
method Quantitative statistical bounds and numerical approximation of Gibbs measures.
result Established optimal numerical strategy and its cost for Bayesian posterior mean approximation.
Approximate Bayesian inference on the basis of summary statistics is well-suited to complex problems for which the likelihood is either mathematically or computationally intractable. However the methods that use rejection suffer from the curse of dimensionality when the number of summary statistics is increased. Here w…
Neural networks are popular state-of-the-art models for many different tasks.They are often trained via back-propagation to find a value of the weights that correctly predicts the observed data. Although back-propagation has shown good performance in many applications, it cannot easily output an estimate of the uncerta…
Generalized linear models (GLMs) -- such as logistic regression, Poisson regression, and robust regression -- provide interpretable models for diverse data types. Probabilistic approaches, particularly Bayesian ones, allow coherent estimates of uncertainty, incorporation of prior information, and sharing of power acros…
The study compares Bayesian and frequentist approaches in deep learning.
problem Comparing Bayesian and frequentist inference in deep learning.
method Conducts a comparative analysis of point and posterior estimators across various settings.
result Amortized point estimators generally outperform posterior inference, though posterior inference remains competitive in some low-dimensional problems.
Method estimates parameters for disease spread models robustly.
problem Estimating parameters for disease spread models.
method Statistical Learning applied to Approximate Bayesian Computation.
result Qualitative properties of disease evolution can be assessed.
Improved MMD estimator for likelihood-free inference.
problem Computational challenges in estimating MMD for likelihood-free inference.
method Optimally-weighted MMD estimator with improved sample complexity.
result Significantly improved sample complexity for accurate MMD estimation.
The PAC-Bayesian approach is a powerful set of techniques to derive non- asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately intractable. One may sample from it using Markov chain Monte Carlo, but this is often too slow…
Estimating the predictive uncertainty of a Bayesian learning model is critical in various decision-making problems, e.g., reinforcement learning, detecting adversarial attack, self-driving car. As the model posterior is almost always intractable, most efforts were made on finding an accurate approximation the true post…
UA-SABI uses surrogates to speed up Bayesian inference for expensive models.
problem Inference for computationally expensive models is slow and uncertain.
method Combines surrogate modeling with Amortized Bayesian Inference (ABI) to propagate uncertainties.
result Reliable, fast, and repeated Bayesian inference for expensive models is achieved.
Two novel distributed VB algorithms improve Bayesian inference in sensor networks.
problem Efficient inference in Bayesian frameworks for sensor networks.
method Two novel distributed VB algorithms for general Bayesian inference, using stochastic natural gradient and ADMM.
result Distributed algorithms perform nearly as well as centralized ones, demonstrating excellent performance.
Many statistical models can be simulated forwards but have intractable likelihoods. Approximate Bayesian Computation (ABC) methods are used to infer properties of these models from data. Traditionally these methods approximate the posterior over parameters by conditioning on data being inside an ε-ball around the obs…
Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms---which build a sm…
We consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likel…
Bayesian approach improves performance in Gaussian process models.
problem Scalable posterior estimation in Gaussian process models.
method Revisiting variational inference techniques with Bayesian treatment of inducing variables and hyper-parameters.
result State-of-the-art performance demonstrated across various regression and classification problems.
QEM uses parallel importance weighting for fast approximate Bayesian inference.
problem Bayesian inference challenges in large models with many observations and latent variables.
method Expectation Maximization (EM) with massively parallel importance weighting.
result QEM is faster and more scalable than RWS and VI.