Meta-learn Bayesian inference for task-specific BNNs using amortised inference.
problem Efficiently learning Bayesian inference for small-scale probabilistic meta-learning.
method Replace global inducing points with actual data to create a set of approximate likelihoods, train a meta-model to learn these parameters across related datasets.
result Meta-learned inference can be applied to task-specific BNNs, improving efficiency and scalability.
A new method for efficient probabilistic meta-learning.
problem High-quality predictions with well-calibrated uncertainty estimates require large amounts of data.
method Amortised Inference in Bayesian Neural Networks (APOVI-BNN)
result The APOVI-BNN produces high-quality predictions with well-calibrated uncertainty estimates using significantly less data.
Probabilistic programming languages (PPLs) are powerful modelling tools which allow to formalise our knowledge about the world and reason about its inherent uncertainty. Inference methods used in PPL can be computationally costly due to significant time burden and/or storage requirements; or they can lack theoretical g…
Capsule models enforce object pose relationships for robustness, explored with probabilistic generative and variational methods.
problem Enforcing object pose relationships for robustness to viewpoint changes.
method Probabilistic generative model with variational bound, exploring capsule assumptions and inference mechanisms.
result Unified objective and test time optimisation demonstrated for capsule models.
We propose a novel approximate inference algorithm that approximates a target distribution by amortising the dynamics of a user-selected MCMC sampler. The idea is to initialise MCMC using samples from an approximation network, apply the MCMC operator to improve these samples, and finally use the samples to update the a…
New method improves generative model performance by fully conditioning variational posteriors.
problem Inaccurate inference due to partial conditioning of variational posteriors in sequential LVMs.
method Introduces fully-conditioned approximate posteriors to improve generative model performance.
result Improves generative modelling and multi-step prediction performance.
Image super-resolution (SR) is an underdetermined inverse problem, where a large number of plausible high-resolution images can explain the same downsampled image. Most current single image SR methods use empirical risk minimisation, often with a pixel-wise mean squared error (MSE) loss. However, the outputs from such …
The paper reclassifies RL algorithms using inference concepts.
problem To provide a unified perspective on RL algorithms.
method Using the control as inference framework, the paper classifies RL algorithms as amortised and iterative inference.
result A wide range of RL algorithms can be classified in this new manner.
Unified reinforcement learning methods using hybrid inference.
problem Combining model-based and model-free reinforcement learning approaches.
method Control as Hybrid Inference (CHI) framework.
result CHI algorithm balances model-based and model-free learning.
New method improves robustness and efficiency of inference from complex models.
problem Inability of existing methods to handle outliers in simulator-based models.
method Generalised Bayesian inference with neural approximation of weighted score-matching loss.
result Provable robustness to outliers and computational efficiency.
Paper introduces RVNP to improve SBI in misspecified models.
problem Misspecification in simulation-based inference leads to unreliable posterior estimation.
method RVNP uses variational inference and error modeling to bridge the simulation-to-reality gap.
result RVNP can recover robust posterior inference without hyperparameters or priors.
Probabilistic graphical models are powerful tools which allow us to formalise our knowledge about the world and reason about its inherent uncertainty. There exist a considerable number of methods for performing inference in probabilistic graphical models; however, they can be computationally costly due to significant t…
We introduce a flexible, scalable Bayesian inference framework for nonlinear dynamical systems characterised by distinct and hierarchical variability at the individual, group, and population levels. Our model class is a generalisation of nonlinear mixed-effects (NLME) dynamical systems, the statistical workhorse for ma…
New method learns priors for Bayesian neural networks from datasets.
problem Lack of prior beliefs in Bayesian deep learning.
method Amortised variational inference to learn priors from datasets.
result Flexible Bayesian neural networks for meta-learning and within-task minibatching.
A new method learns latent variable updates directly, not approximating the posterior.
problem Intractable maximum-likelihood learning for complex latent-variable models.
method Amortised learning using wake-sleep Monte-Carlo strategy.
result Demonstrated effectiveness on various complex models.
New method uses path signatures for efficient likelihood estimation in time-series data.
problem Intractable likelihood functions in complex dynamic models.
method Kernel classifier based on path signatures for sequential data.
result Path signatures yield highly performant classifiers, even with low sample numbers.
POLAR learns efficient data acquisition policies using pretrained belief representations.
problem Challenges in learning effective policies for adaptive data acquisition.
method POLAR decouples representation learning from policy learning by leveraging pretrained predictive foundation models as belief-state encoders.
result POLAR outperforms state-of-the-art methods across diverse tasks while requiring fewer training samples.
Develops SGP-VAE for efficient sparse GP inference in multi-dimensional datasets.
problem Sparse GP approximations and missing data in multi-dimensional spatio-temporal datasets.
method Leverages partial inference networks for sparse GP approximations and amortized variational inference.
result Outperforms multi-output GPs and structured VAEs in various experiments.
A new method for Bayesian inference using diffusion models.
problem Bayesian inference in simulator-based models.
method Score-based diffusion models trained with a sequential training procedure.
result Comparable or superior performance compared to existing methods.
New methods improve uncertainty explanations for models.
problem Improving interpretation of uncertainty estimates from probabilistic models.
method Developed new methods to generate diverse and global explanations for uncertain model predictions.
result Generated diverse and global explanations for uncertain model predictions, addressing previous limitations.
Paper proposes a new method for sampling from complex distributions.
problem Sampling from unnormalised density functions in complex distributions.
method Combines amortised and particle-based methods with reinforcement learning.
result Improves sampling from complex distributions compared to existing methods.
GATSBI uses GANs for SBI, improving posterior estimation in high dimensions.
problem Statistical inference on stochastic models without likelihoods.
method Adversarial approach to variational objective, amortized inference, implicit priors.
result GATSBI returns well-calibrated posterior estimates in high dimensions.
VMoER improves uncertainty quantification in MoE layers for scalable foundation models.
problem Uncertainty quantification in large-scale models like MoE layers.
method Structured Bayesian approach with amortized variational inference over routing logits and temperature parameter inference.
result Improves routing stability, reduces calibration error, and increases AUROC by 12%.
Generative latent-variable models are emerging as promising tools in robotics and reinforcement learning. Yet, even though tasks in these domains typically involve distinct objects, most state-of-the-art generative models do not explicitly capture the compositional nature of visual scenes. Two recent exceptions, MONet …
New method uses neural networks to infer dark matter subhalo abundance from stellar streams.
problem Constrain warm dark matter mass using stellar streams.
method Amortized Approximate Likelihood Ratios (AALR) for likelihood-free Bayesian inference.
result Demonstrates effectiveness of new method for estimating dark matter subhalo abundance.
NPE improves scalability and efficiency for ERGMs.
problem Scalability and efficiency issues in Bayesian ERGM estimation.
method Neural posterior estimation (NPE) for ERGMs using neural network density estimation.
result NPE provides more efficient and scalable inference for ERGMs.
Neural point estimators improve parameter estimation from replicated data.
problem Making inference from replicated data in weakly-identified and highly-parameterised models.
method Permutation-invariant neural networks for likelihood-free parameter estimation.
result Neural point estimators can quickly and optimally estimate parameters.
Divide-and-conquer framework speeds up black-box inference for large data.
problem Computational intractability of uncertainty quantification for expensive data simulation.
method Divide data into partitions, train on a subset, bootstrap on partitions, combine results.
result Feasibility of estimating max-stable process parameters with tens of thousands of locations.
Proposes a method to learn conditional VAEs from datasets with missing covariates.
problem Learning conditional VAEs from datasets with missing covariates.
method Augments conditional VAEs with a prior distribution for missing covariates and estimates their posterior using amortised variational inference.
result The proposed method outperforms previous methods in learning conditional VAEs from non-temporal, temporal, and longitudinal datasets.
Flexible model tackles high-dimensional, missing data, and stochastic processes.
problem High-dimensional longitudinal data with structured missingness and unknown measurement time points.
method Latent variable model using Gaussian processes and variational autoencoder.
result Competitive performance on simulated and real datasets.
We introduce a methodology for efficiently computing a lower bound to empowerment, allowing it to be used as an unsupervised cost function for policy learning in real-time control. Empowerment, being the channel capacity between actions and states, maximises the influence of an agent on its near future. It has been sho…
The paper speeds up hyperparameter optimisation in Gaussian processes.
problem Scaling hyperparameter optimisation to large datasets.
method Improvements to linear system solvers (pathwise gradient, warm starting, early stopping).
result Speed-ups of up to 72x and residual norm decreases of up to 7x.
Warm starts improve Gaussian process regression by up to 16x.
problem Optimizing hyperparameters for Gaussian processes.
method Iterative Gaussian processes with warm start optimization.
result Warm starts achieve the same results as conventional methods but significantly speed up computations.
Discrete diffusion samplers improve sampling from unnormalised densities.
problem Sampling from discrete unnormalised densities efficiently.
method Introduce off-policy training techniques and data-to-energy Schrödinger bridge training for discrete diffusion samplers.
result Improved performance on synthetic and new benchmarks.
Italian banks use swaps to hedge against rising interest rates, offsetting losses on debt securities.
problem Interest rate risk on Italian banks' debt securities.
method Analysis of granular regulatory data on euro interest rate swap trades.
result Swaps can offset losses on debt securities, reducing interest rate exposure.
New analysis for learning and applying preconditioners in MCMC improves efficiency.
problem Improving efficiency of MCMC algorithms by modifying them with preconditioners.
method Analyzes and compares computational costs of MCMC schemes with and without preconditioners.
result Establishes non-asymptotic guarantees for MCMC algorithms that learn and use preconditioners.
Analyzes learning and applying preconditioners in MCMC for efficiency.
problem Improving efficiency of MCMC algorithms.
method Non-asymptotic analysis of schemes that learn preconditioners.
result Established non-asymptotic guarantees for preconditioned ULA.
Graph neural networks extend neural Bayes estimators to irregular spatial data.
problem Estimating parameters from irregular spatial data with computational efficiency.
method Employing graph neural networks to approximate Bayes estimators for irregular spatial data.
result Extending neural Bayes estimation to irregular spatial data with computational benefits.
In 1979 following a decade of hyperinflation, Iceland introduced Verðtryggð lán, negatively amortised, index-linked loans whose outstanding principal is increased by the rate of the consumer price inflation index(CPI). The loans were part of a general government policy which used indexation to the CPI to address the ec…
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
Approximate probabilistic inference algorithms are central to many fields. Examples include sequential Monte Carlo inference in robotics, variational inference in machine learning, and Markov chain Monte Carlo inference in statistics. A key problem faced by practitioners is measuring the accuracy of an approximate infe…
This work frames active inference through control as inference, offering robust control algorithms.
problem Active inference framework lacks practical sensorimotor control algorithms.
method Frame active inference through control as inference, presenting trajectory optimization as inference.
result AI may be framed as partially-observed CaI when the cost function is defined in observation states.
Simformer uses transformer models to perform flexible Bayesian inference.
problem Current simulation-based inference methods are inflexible and require fixed priors.
method Trains a probabilistic diffusion model with transformer architectures.
result Outperforms state-of-the-art methods on various benchmarks.
Improved Bayesian inference for neuronal ensemble inference reduces computational cost.
problem Efficient inference of neuronal ensembles from activity data.
method Modified MCMC algorithm with simulated annealing for hyperparameter control.
result Our method reduces computational cost while maintaining or improving inference accuracy.
Adding metadata abruptly changes network inference outcomes.
problem Understanding the impact of metadata on network inference.
method Investigated the effect of metadata on network inference problems.
result Metadata causes abrupt transitions in inference outcomes.
SNVI combines likelihood estimation with variational inference for efficient Bayesian inference.
problem Bayesian inference in models with intractable likelihoods.
method Sequential Neural Variational Inference (SNVI) that combines likelihood-estimation with variational inference.
result SNVI is more computationally efficient than previous algorithms without sacrificing accuracy.
Paper introduces a diagnostic for approximate inference methods.
problem Estimating errors in probabilistic inference algorithms, especially for approximate methods.
method Repeatedly simulate datasets from the prior and perform inference on each, estimating a symmetric KL-divergence.
result A diagnostic for approximate inference methods can be estimated using symmetric KL-divergence.
Probabilistic inference procedures are usually coded painstakingly from scratch, for each target model and each inference algorithm. We reduce this effort by generating inference procedures from models automatically. We make this code generation modular by decomposing inference algorithms into reusable program-to-progr…