This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
Adaptive workflow combines fast amortized inference with MCMC for many datasets.
problem Trade-off between computational speed and sampling accuracy in Bayesian inference.
method Adaptive workflow integrating amortized inference and MCMC with principled diagnostics.
result Efficiency gains with high posterior quality on tens of thousands of datasets.
The variational autoencoder (VAE) is a popular model for density estimation and representation learning. Canonically, the variational principle suggests to prefer an expressive inference model so that the variational approximation is accurate. However, it is often overlooked that an overly-expressive inference model ca…
A new method optimizes projection directions for sliced Wasserstein distances.
problem Finding informative projecting directions for sliced Wasserstein distances is computationally expensive.
method Amortized projection optimization to predict directions efficiently.
result Proposed amortized models improve generative modeling performance.
BayesFlow trains neural networks for fast Bayesian inference.
problem Fast Bayesian inference for complex models.
method Amortized neural networks for intractable posterior distributions.
result Fast inference through pre-trained neural networks.
AQL uses amortized inference to handle high-dimensional action spaces in Q-learning.
problem Difficulty in maximizing over large action spaces in Q-learning.
method Replace expensive maximization over all actions with a maximization over a small subset sampled from a learned proposal distribution.
result AQL outperforms existing methods on continuous control tasks with up to 21 dimensional actions.
ASPIRE improves amortized posterior inference for Bayesian inverse problems.
problem Bayesian inverse problems are computationally challenging due to uncertainty quantification.
method Iterative refinement of amortized posteriors using physics-based and summary statistics.
result ASPIRE achieves better posterior approximations with minimal extra computations.
This paper examines how neural architectures support amortized Bayesian inference and its performance under varying conditions.
problem Understanding and evaluating amortized inference under signal-to-noise variation and distribution shift.
method Statistical analysis of neural architectures including feedforward networks, Deep Sets, and Transformers.
result Neural architectures support amortized Bayesian inference, offering controlled generalization error and robustness under varying conditions.
Amortized VI for DGPs learns efficient inference.
problem Expressive limitations in GP approximations.
method Amortized variational inference for DGPs.
result Improved expressive prior and posterior for DGPs.
The recognition network in deep latent variable models such as variational autoencoders (VAEs) relies on amortized inference for efficient posterior approximation that can scale up to large datasets. However, this technique has also been demonstrated to select suboptimal variational parameters, often resulting in consi…
Novel approach to Bayesian experimental design for non-exchangeable data.
problem Optimal experimental design for non-exchangeable data.
method Inside-Out SMC2 algorithm embedded in particle Markov chain Monte Carlo framework. result Efficacy demonstrated on a set of dynamical systems.
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
PABBO optimizes user utility learning from preferential feedback, significantly faster than traditional methods.
problem Computational inefficiency in Preferential Bayesian Optimization (PBO) methods.
method Amortized Bayesian Optimization (PABBO) using transformer neural processes and reinforcement learning.
result Significantly faster performance compared to traditional Gaussian process-based methods.
Paper tackles scalable VFL with data augmentation and amortized inference.
problem Collaborative model estimation across multiple clients with distinct covariates.
method Data augmentation, amortized variational approximation, factorized likelihoods.
result Scalable Bayesian VFL framework for various models.
Posterior inference with an intractable likelihood is becoming an increasingly common task in scientific domains which rely on sophisticated computer simulations. Typically, these forward models do not admit tractable densities forcing practitioners to make use of approximations. This work introduces a novel approach t…
PriorGuide adapts diffusion models to new priors at test time.
problem Limited applicability of prior distributions in diffusion-based inference.
method PriorGuide uses a guidance approximation to adapt diffusion models to new priors at test time.
result Enhances the versatility of pre-trained inference models by allowing flexible adaptation to new priors.
InfoMax-VAE enhances VAEs for better representation learning.
problem Lack of meaningful learned representations in VAEs.
method Combines VAEs with mutual information maximization.
result Significantly boosts quality of learned high-level representations.
Current approaches to amortizing Bayesian inference focus solely on approximating the posterior distribution. Typically, this approximation is, in turn, used to calculate expectations for one or more target functions - a computational pipeline which is inefficient when the target function(s) are known upfront. In this …
Improves point-cloud reconstruction by optimizing projections with self-attention.
problem Inefficient and non-metric projection methods for sliced Wasserstein distances.
method Proposes distributional sliced Wasserstein distance with self-attention for permutation-invariant and metric optimization.
result Self-attention amortized distributional projection optimization achieves better performance in point-cloud reconstruction.
New option type preserves fungibility by amortizing payments over time.
problem Traditional installment options destroy fungibility and lapse when payments stop.
method Introduces amortizing perpetual options (AmPOs) with an implicit payment scheme.
result Valuation of AmPOs reduces to vanilla perpetual American options.
ADAVI tackles variational inference for large HBM models in neuroimaging.
problem Large, pyramidally-organized HBM models in neuroimaging studies.
method Automatic dual amortized variational inference using neural networks and attention-based hierarchical encoding.
result Significantly reduced parameterization of the variational family, maintaining expressivity.
Despite the recent success in probabilistic modeling and their applications, generative models trained using traditional inference techniques struggle to adapt to new distributions, even when the target distribution may be closely related to the ones seen during training. In this work, we present a doubly-amortized var…
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
Neural clustering learns time series affinity from statistical features.
problem Challenging time series clustering with unknown cluster shapes and structures.
method Amortized neural inference using statistical features.
result Competitive clustering accuracy without manual specification of cluster shapes.
Framework creates fast, interpretable surrogates for stochastic simulators with unbounded randomness.
problem Creating accurate and fast approximations for stochastic simulators with unbounded randomness.
method Probabilistic surrogate networks that retain structure of reference simulators and enable amortized inference.
result Surrogates accurately model stochastic programs with unbounded random variables and significantly speed up inference.
Improved diffusion sampling for inverse problems with faster and more robust inference.
problem High computational cost and lack of robustness in diffusion posterior sampling.
method Amortized variational inference with explicit likelihood guidance.
result Improved trade-off between inference speed and robustness to unseen degradations.
Efficiently predicts optimal transport plans using sliced potentials.
problem Predicting optimal transport plans across multiple measure pairs efficiently.
method Regression-based and objective-based amortization strategies using sliced optimal transport potentials.
result Efficient and accurate prediction of optimal transport plans for various tasks.
Inspired by the seminal work on Stein Variational Inference and Stein Variational Policy Gradient, we derived a method to generate samples from the posterior variational parameter distribution by \textit{explicitly} minimizing the KL divergence to match the target distribution in an amortize fashion. Consequently, we a…
Simformer uses transformer models to perform flexible Bayesian inference.
problem Current simulation-based inference methods are inflexible and require fixed priors.
method Trains a probabilistic diffusion model with transformer architectures.
result Outperforms state-of-the-art methods on various benchmarks.
The paper corrects Bayesian neural network approximations to improve decision quality.
problem Inaccurate posterior approximations in Bayesian neural networks lead to suboptimal decisions.
method Develops methods to calibrate approximate posterior predictive distributions for better decision making.
result Empirically produces higher quality decisions compared to previous methods.
New method improves generalization of VAEs by reducing overfitting.
problem Generalization issues in VAEs overfitting to training data.
method Proposed a new training objective to improve amortized inference.
result Improved performance in image modeling and lossless compression.
Adversarial robustness of amortized Bayesian inference is studied, showing it can be improved.
problem Adversarial robustness of amortized Bayesian inference.
method Simulation-based estimation, regularization scheme based on Fisher information.
result Adversarial robustness can be improved with a regularization scheme.
New method reduces infinite variance in probabilistic programs with rejection sampling.
problem Infinite variance in naive importance sampling for programs with rejection sampling.
method Developed a new amortized importance sampling estimator with finite variance proof.
result Empirically demonstrated efficiency and correctness compared to existing alternatives.
The paper tackles efficient computation of optimal transport by approximating conjugates with amortized optimization.
problem Efficient computation of convex conjugates in optimal transport is challenging and limits the quality of transport maps.
method The approach combines amortized approximations of conjugates with a fine-tuning solver to improve transport map quality.
result The method significantly improves the quality of transport maps for the Wasserstein-2 benchmark and models many 2D couplings and flows.
Neural processes approximate Gaussian process inference, revealing three key costs.
problem Approximating Gaussian process inference with neural processes.
method Bounding KL divergence into three components: label contamination, information bottleneck, and amortization error.
result Characterization of three costs of amortizing Gaussian process inference with neural processes.
Self-consistency improves the accuracy of model comparison methods.
problem Improving the accuracy of model comparison methods when simulation models are misspecified.
method Supplement traditional simulation-based training with a self-consistency loss on unlabeled real data.
result Self-consistency training improves model comparison accuracy, especially in open-world scenarios.
Improved community detection in graphs with probabilistic models.
problem Lack of probabilistic formulation and fixed number of communities in GNN-based methods.
method Combines GNNs with amortized clustering for variable numbers of clusters.
result Improved performance on synthetic and real datasets compared to previous methods.
Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any posterior without leveraging the structures of the generative models. In this paper, we propose Amorti…
Improved phylogenetic inference with normalizing flows.
problem Limitations of current diagonal Lognormal branch length approximation in VBPI.
method Proposes VBPI-NF using normalizing flows to handle non-Euclidean branch length space.
result Significantly improves phylogenetic posterior estimation on real data.
Proposes an amortized variational framework for Deep Q Networks.
problem Efficient exploration in deep reinforcement learning.
method Amortized variational inference for action value function approximation.
result Significantly less learning parameters and better performance.
This paper compares amortized methods for Bayesian posterior estimation.
problem Bayesian inference difficulties with iterative routines.
method Amortized in-context Bayesian posterior estimation using transformers and normalizing flows.
result Reverse KL estimator superior for predictive problems.
New method generates realistic adversarial images with smaller perturbations.
problem Generating realistic adversarial images with small perturbations.
method Imposes spatial distortions (scaling, rotation, shear, translation) in addition to perturbations.
result Produces visually more realistic attacks that deceive classifiers without affecting human predictions.
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of eith…
New method learns causal models from data efficiently.
problem Learning Structural Causal Models from data is challenging.
method Amortized inference via Conditional Fixed-Point Iterations with transformer embeddings.
result Single model predicts causal mechanisms conditioned on data and graph.
A new recursive mixture estimation algorithm improves VAE inference efficiency and accuracy.
problem Inaccurate posterior approximation in traditional VAEs.
method Recursive mixture estimation algorithm using functional gradient approach for iterative component selection.
result Significantly higher test data likelihood compared to state-of-the-art methods on benchmark datasets.
DAD learns to design experiments quickly, outperforming traditional methods.
problem Real-time decision-making in sequential Bayesian experimental design.
method Amortized design network trained with contrastive information bounds.
result DAD outperforms alternative strategies on various problems.
This paper tackles sampling issues in latent space EBMs by introducing diffusion-based amortization.
problem Degenerate MCMC sampling quality hinders latent space EBM learning and generation quality.
method Introduces diffusion-based amortization for long-run MCMC sampling.
result The learned amortization of MCMC is a valid long-run MCMC sampler.
LazyDINO efficiently solves high-dimensional Bayesian inverse problems with fast and scalable solutions.
problem High-dimensional nonlinear Bayesian inverse problems with expensive parameter-to-observable maps.
method LazyDINO combines derivative-informed neural surrogates and lazy map variational inference for efficient posterior approximation.
result Significant cost reduction in amortized Bayesian inversion, achieving one to two orders of magnitude improvement.