Adaptive workflow combines fast amortized inference with MCMC for many datasets.
problem Trade-off between computational speed and sampling accuracy in Bayesian inference.
method Adaptive workflow integrating amortized inference and MCMC with principled diagnostics.
result Efficiency gains with high posterior quality on tens of thousands of datasets.
The variational autoencoder (VAE) is a popular model for density estimation and representation learning. Canonically, the variational principle suggests to prefer an expressive inference model so that the variational approximation is accurate. However, it is often overlooked that an overly-expressive inference model ca…
This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
Simformer uses transformer models to perform flexible Bayesian inference.
problem Current simulation-based inference methods are inflexible and require fixed priors.
method Trains a probabilistic diffusion model with transformer architectures.
result Outperforms state-of-the-art methods on various benchmarks.
This paper examines how neural architectures support amortized Bayesian inference and its performance under varying conditions.
problem Understanding and evaluating amortized inference under signal-to-noise variation and distribution shift.
method Statistical analysis of neural architectures including feedforward networks, Deep Sets, and Transformers.
result Neural architectures support amortized Bayesian inference, offering controlled generalization error and robustness under varying conditions.
BayesFlow trains neural networks for fast Bayesian inference.
problem Fast Bayesian inference for complex models.
method Amortized neural networks for intractable posterior distributions.
result Fast inference through pre-trained neural networks.
Adversarial robustness of amortized Bayesian inference is studied, showing it can be improved.
problem Adversarial robustness of amortized Bayesian inference.
method Simulation-based estimation, regularization scheme based on Fisher information.
result Adversarial robustness can be improved with a regularization scheme.
ASPIRE improves amortized posterior inference for Bayesian inverse problems.
problem Bayesian inverse problems are computationally challenging due to uncertainty quantification.
method Iterative refinement of amortized posteriors using physics-based and summary statistics.
result ASPIRE achieves better posterior approximations with minimal extra computations.
Improved diffusion sampling for inverse problems with faster and more robust inference.
problem High computational cost and lack of robustness in diffusion posterior sampling.
method Amortized variational inference with explicit likelihood guidance.
result Improved trade-off between inference speed and robustness to unseen degradations.
Amortized VI for DGPs learns efficient inference.
problem Expressive limitations in GP approximations.
method Amortized variational inference for DGPs.
result Improved expressive prior and posterior for DGPs.
New method improves generalization of VAEs by reducing overfitting.
problem Generalization issues in VAEs overfitting to training data.
method Proposed a new training objective to improve amortized inference.
result Improved performance in image modeling and lossless compression.
Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any posterior without leveraging the structures of the generative models. In this paper, we propose Amorti…
A new recursive mixture estimation algorithm improves VAE inference efficiency and accuracy.
problem Inaccurate posterior approximation in traditional VAEs.
method Recursive mixture estimation algorithm using functional gradient approach for iterative component selection.
result Significantly higher test data likelihood compared to state-of-the-art methods on benchmark datasets.
Neural networks speed up statistical inference.
problem Efficient statistical inference for complex models.
method Neural networks for learning complex mappings.
result Amortized inference speeds up inference processes.
New method learns causal models from data efficiently.
problem Learning Structural Causal Models from data is challenging.
method Amortized inference via Conditional Fixed-Point Iterations with transformer embeddings.
result Single model predicts causal mechanisms conditioned on data and graph.
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…
UA-SABI uses surrogates to speed up Bayesian inference for expensive models.
problem Inference for computationally expensive models is slow and uncertain.
method Combines surrogate modeling with Amortized Bayesian Inference (ABI) to propagate uncertainties.
result Reliable, fast, and repeated Bayesian inference for expensive models is achieved.
A-VI can approximate F-VI under certain conditions, improving inference in some models.
problem Approximate Bayesian inference using factorized distributions.
method Amortized variational inference (A-VI) learns a common inference function for latent variables.
result A-VI can close the amortization gap in simple hierarchical models.
This paper compares amortized methods for Bayesian posterior estimation.
problem Bayesian inference difficulties with iterative routines.
method Amortized in-context Bayesian posterior estimation using transformers and normalizing flows.
result Reverse KL estimator superior for predictive problems.
Probabilistic programming languages (PPLs) are a powerful modeling tool, able to represent any computable probability distribution. Unfortunately, probabilistic program inference is often intractable, and existing PPLs mostly rely on expensive, approximate sampling-based methods. To alleviate this problem, one could tr…
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…
New method uses neural networks to efficiently approximate Bayesian inference for complex models.
problem Efficiently approximating Bayesian inference for complex models with varying temperatures.
method Fully amortized neural posterior estimator trained on a single forward pass.
result Achieves competitive posterior approximations across various temperatures and benchmarks.
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
Proposes an amortized variational framework for Deep Q Networks.
problem Efficient exploration in deep reinforcement learning.
method Amortized variational inference for action value function approximation.
result Significantly less learning parameters and better performance.
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
Current approaches to amortizing Bayesian inference focus solely on approximating the posterior distribution. Typically, this approximation is, in turn, used to calculate expectations for one or more target functions - a computational pipeline which is inefficient when the target function(s) are known upfront. In this …
SurvivalPFN simplifies survival analysis through amortized Bayesian inference.
problem Selecting appropriate survival analysis methods requires expertise and can be time-consuming.
method SurvivalPFN uses a prior-data fitted network for in-context Bayesian inference.
result SurvivalPFN achieves strong predictive performance across diverse datasets.
New methods make Bayesian inference feasible for complex cognitive models.
problem Bayesian inference is hard for complex, unknown models.
method Amortized neural network estimators for simulation-based inference.
result Effort in training networks amortizes over multiple uses.
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models such as variational autoencoders (VAE), recent empirical work suggests that inference networks can produce suboptimal variational parameter…
PAVI speeds up Bayesian inference for large datasets.
problem Challenges in Bayesian inference for large population studies.
method Plate-amortized Variational Inference (PAVI) that shares parameterization across i.i.d. variables.
result Significant speedup in training variational distributions, orders of magnitude faster.
We introduce the variational filtering EM algorithm, a simple, general-purpose method for performing variational inference in dynamical latent variable models using information from only past and present variables, i.e. filtering. The algorithm is derived from the variational objective in the filtering setting and cons…
Improved state estimation in nonlinear models using amortized backward variational inference.
problem State estimation in general state-space models.
method Amortized backward variational inference with neural network parameters.
result Linear growth of variational approximation error in number of observations.
This paper speeds up inference in large hierarchical models.
problem Inference in large hierarchical models is slow and difficult.
method Amortized variational inference with shared parameters.
result Amortized inference is as accurate as a full-rank Gaussian but much faster.
A new method learns causal structure from data using amortized inference.
problem Causal structure learning is a combinatorial search problem that is costly and difficult to design suitable scores or tests.
method Train a variational inference model to predict causal structure from data.
result Our inference model generalizes well to larger problem instances and outperforms existing algorithms, especially in genomics.
ALINE optimizes Bayesian inference and data acquisition by strategically querying informative data.
problem Strategic acquisition of informative data for Bayesian inference in challenging tasks.
method Unified framework combining amortized Bayesian inference and active data acquisition using a transformer architecture trained via reinforcement learning.
result ALINE delivers both instant and accurate inference along with efficient selection of informative points.
ACE improves GBI for simulators by approximating cost functions, making inference more efficient.
problem Inference for misspecified simulators is overly restrictive.
method Amortized cost estimation (ACE) for Generalized Bayesian Inference (GBI).
result ACE provides accurate cost predictions and more efficient inference.
Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amor…
LazyDINO efficiently solves high-dimensional Bayesian inverse problems with fast and scalable solutions.
problem High-dimensional nonlinear Bayesian inverse problems with expensive parameter-to-observable maps.
method LazyDINO combines derivative-informed neural surrogates and lazy map variational inference for efficient posterior approximation.
result Significant cost reduction in amortized Bayesian inversion, achieving one to two orders of magnitude improvement.
Amortized Causal Discovery learns to infer causal graphs from time-series data, improving performance.
problem Inference of causal graphs from time-series data is inefficient due to fitting new models for each sample.
method Proposes Amortized Causal Discovery, a variational model that leverages shared dynamics across samples with different causal graphs.
result Significant improvements in causal discovery performance demonstrated experimentally.
Proposes efficient sensitivity analysis for complex Bayesian models.
problem Inefficiency of sensitivity analyses in complex Bayesian models.
method SA-ABI: weight sharing and neural network rapid inference.
result Efficiently integrates sensitivity analyses into Bayesian inference.
A new method decouples set representation learning from posterior modeling for efficient amortized inference.
problem Efficient inference for large sets of observations with shared factors.
method Train a mean-pool Deep Set on sets of size at most two, then finetune the inference head on pre-aggregated embeddings.
result Matches or outperforms standard baselines at a fraction of the compute cost for large N.
Naive approaches to amortized inference in probabilistic programs with unbounded loops can produce estimators with infinite variance. This is particularly true of importance sampling inference in programs that explicitly include rejection sampling as part of the user-programmed generative procedure. In this paper we de…
PriorGuide adapts diffusion models to new priors at test time.
problem Limited applicability of prior distributions in diffusion-based inference.
method PriorGuide uses a guidance approximation to adapt diffusion models to new priors at test time.
result Enhances the versatility of pre-trained inference models by allowing flexible adaptation to new priors.
Neural processes approximate Gaussian process inference, revealing three key costs.
problem Approximating Gaussian process inference with neural processes.
method Bounding KL divergence into three components: label contamination, information bottleneck, and amortization error.
result Characterization of three costs of amortizing Gaussian process inference with neural processes.
Improved variational inference for geophysical inverse problems with data correction.
problem High computational cost and accuracy issues in Bayesian inference for geophysical inverse problems.
method Amortized variational inference with latent distribution correction using physics-based priors.
result Improved robustness of amortized variational inference under data distribution shifts.
Delta-AI speeds up inference in sparse PGMs by local credit assignment.
problem Efficient inference in sparse probabilistic graphical models.
method Local credit assignment in agent's policy learning objective.
result Trained sampler recovers marginals and conditional distributions.
Paper tackles scalable VFL with data augmentation and amortized inference.
problem Collaborative model estimation across multiple clients with distinct covariates.
method Data augmentation, amortized variational approximation, factorized likelihoods.
result Scalable Bayesian VFL framework for various models.