A new method optimizes projection directions for sliced Wasserstein distances.
problem Finding informative projecting directions for sliced Wasserstein distances is computationally expensive.
method Amortized projection optimization to predict directions efficiently.
result Proposed amortized models improve generative modeling performance.
This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
ASPIRE improves amortized posterior inference for Bayesian inverse problems.
problem Bayesian inverse problems are computationally challenging due to uncertainty quantification.
method Iterative refinement of amortized posteriors using physics-based and summary statistics.
result ASPIRE achieves better posterior approximations with minimal extra computations.
The variational autoencoder (VAE) is a popular model for density estimation and representation learning. Canonically, the variational principle suggests to prefer an expressive inference model so that the variational approximation is accurate. However, it is often overlooked that an overly-expressive inference model ca…
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any posterior without leveraging the structures of the generative models. In this paper, we propose Amorti…
Proposes an amortized variational framework for Deep Q Networks.
problem Efficient exploration in deep reinforcement learning.
method Amortized variational inference for action value function approximation.
result Significantly less learning parameters and better performance.
The paper tackles efficient computation of optimal transport by approximating conjugates with amortized optimization.
problem Efficient computation of convex conjugates in optimal transport is challenging and limits the quality of transport maps.
method The approach combines amortized approximations of conjugates with a fine-tuning solver to improve transport map quality.
result The method significantly improves the quality of transport maps for the Wasserstein-2 benchmark and models many 2D couplings and flows.
Amortized VI for DGPs learns efficient inference.
problem Expressive limitations in GP approximations.
method Amortized variational inference for DGPs.
result Improved expressive prior and posterior for DGPs.
This paper tackles sampling issues in latent space EBMs by introducing diffusion-based amortization.
problem Degenerate MCMC sampling quality hinders latent space EBM learning and generation quality.
method Introduces diffusion-based amortization for long-run MCMC sampling.
result The learned amortization of MCMC is a valid long-run MCMC sampler.
New method learns causal models from data efficiently.
problem Learning Structural Causal Models from data is challenging.
method Amortized inference via Conditional Fixed-Point Iterations with transformer embeddings.
result Single model predicts causal mechanisms conditioned on data and graph.
New method improves generalization of VAEs by reducing overfitting.
problem Generalization issues in VAEs overfitting to training data.
method Proposed a new training objective to improve amortized inference.
result Improved performance in image modeling and lossless compression.
VAEs help in learning latent variables for cryo-EM applications.
problem Learning latent variables for cryo-EM data.
method Used VAEs for latent variable learning, focusing on the encoder's role.
result The encoder of the VAE in cryo-EM applications resembles traditional latent variable representations.
This paper examines how neural architectures support amortized Bayesian inference and its performance under varying conditions.
problem Understanding and evaluating amortized inference under signal-to-noise variation and distribution shift.
method Statistical analysis of neural architectures including feedforward networks, Deep Sets, and Transformers.
result Neural architectures support amortized Bayesian inference, offering controlled generalization error and robustness under varying conditions.
Probabilistic programming languages (PPLs) are a powerful modeling tool, able to represent any computable probability distribution. Unfortunately, probabilistic program inference is often intractable, and existing PPLs mostly rely on expensive, approximate sampling-based methods. To alleviate this problem, one could tr…
Efficiently predicts optimal transport plans using sliced potentials.
problem Predicting optimal transport plans across multiple measure pairs efficiently.
method Regression-based and objective-based amortization strategies using sliced optimal transport potentials.
result Efficient and accurate prediction of optimal transport plans for various tasks.
DAD learns to design experiments quickly, outperforming traditional methods.
problem Real-time decision-making in sequential Bayesian experimental design.
method Amortized design network trained with contrastive information bounds.
result DAD outperforms alternative strategies on various problems.
SurvivalPFN simplifies survival analysis through amortized Bayesian inference.
problem Selecting appropriate survival analysis methods requires expertise and can be time-consuming.
method SurvivalPFN uses a prior-data fitted network for in-context Bayesian inference.
result SurvivalPFN achieves strong predictive performance across diverse datasets.
Amortized Causal Discovery learns to infer causal graphs from time-series data, improving performance.
problem Inference of causal graphs from time-series data is inefficient due to fitting new models for each sample.
method Proposes Amortized Causal Discovery, a variational model that leverages shared dynamics across samples with different causal graphs.
result Significant improvements in causal discovery performance demonstrated experimentally.
Applying Q-learning to high-dimensional or continuous action spaces can be difficult due to the required maximization over the set of possible actions. Motivated by techniques from amortized inference, we replace the expensive maximization over all actions with a maximization over a small subset of possible actions sam…
Improved community detection in graphs with probabilistic models.
problem Lack of probabilistic formulation and fixed number of communities in GNN-based methods.
method Combines GNNs with amortized clustering for variable numbers of clusters.
result Improved performance on synthetic and real datasets compared to previous methods.
Current approaches to amortizing Bayesian inference focus solely on approximating the posterior distribution. Typically, this approximation is, in turn, used to calculate expectations for one or more target functions - a computational pipeline which is inefficient when the target function(s) are known upfront. In this …
A-VI can approximate F-VI under certain conditions, improving inference in some models.
problem Approximate Bayesian inference using factorized distributions.
method Amortized variational inference (A-VI) learns a common inference function for latent variables.
result A-VI can close the amortization gap in simple hierarchical models.
Neural processes approximate Gaussian process inference, revealing three key costs.
problem Approximating Gaussian process inference with neural processes.
method Bounding KL divergence into three components: label contamination, information bottleneck, and amortization error.
result Characterization of three costs of amortizing Gaussian process inference with neural processes.
Improves point-cloud reconstruction by optimizing projections with self-attention.
problem Inefficient and non-metric projection methods for sliced Wasserstein distances.
method Proposes distributional sliced Wasserstein distance with self-attention for permutation-invariant and metric optimization.
result Self-attention amortized distributional projection optimization achieves better performance in point-cloud reconstruction.
PABBO optimizes user utility learning from preferential feedback, significantly faster than traditional methods.
problem Computational inefficiency in Preferential Bayesian Optimization (PBO) methods.
method Amortized Bayesian Optimization (PABBO) using transformer neural processes and reinforcement learning.
result Significantly faster performance compared to traditional Gaussian process-based methods.
New option type preserves fungibility by amortizing payments over time.
problem Traditional installment options destroy fungibility and lapse when payments stop.
method Introduces amortizing perpetual options (AmPOs) with an implicit payment scheme.
result Valuation of AmPOs reduces to vanilla perpetual American options.
A new method learns causal structure from data using amortized inference.
problem Causal structure learning is a combinatorial search problem that is costly and difficult to design suitable scores or tests.
method Train a variational inference model to predict causal structure from data.
result Our inference model generalizes well to larger problem instances and outperforms existing algorithms, especially in genomics.
Amortizes MIPS by training neural networks to predict optimal keys.
problem Efficiently solving Maximum Inner Product Search for repeated queries.
method Regression-based SupportNet and KeyNet models trained on support functions.
result Significant improvement in IVF match rates for document embeddings.
Paper tackles scalable VFL with data augmentation and amortized inference.
problem Collaborative model estimation across multiple clients with distinct covariates.
method Data augmentation, amortized variational approximation, factorized likelihoods.
result Scalable Bayesian VFL framework for various models.
Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amor…
Adaptive workflow combines fast amortized inference with MCMC for many datasets.
problem Trade-off between computational speed and sampling accuracy in Bayesian inference.
method Adaptive workflow integrating amortized inference and MCMC with principled diagnostics.
result Efficiency gains with high posterior quality on tens of thousands of datasets.
ABMLL adapts LLMs to multiple datasets efficiently.
problem Limited generalization across multiple datasets in LLMs.
method Amortized Bayesian Meta-Learning for LoRA (ABMLL).
result ABMLL supports effective generalization across datasets and scales to large models.
A new recursive mixture estimation algorithm improves VAE inference efficiency and accuracy.
problem Inaccurate posterior approximation in traditional VAEs.
method Recursive mixture estimation algorithm using functional gradient approach for iterative component selection.
result Significantly higher test data likelihood compared to state-of-the-art methods on benchmark datasets.
APO optimizes neural network parameters by amortizing proximal point methods.
problem Optimizing neural network parameters online and adaptively.
method APO framework that meta-learns proximal point parameters.
result APO can recover and outperform existing optimizers and schedules.
Transformer-based method improves causal discovery from observational data.
problem Causal discovery from observational data requires explicit assumptions.
method CSIvA transformer architecture trained on synthetic data.
result Transformer-based methods adhere to identifiability theory.
ALINE optimizes Bayesian inference and data acquisition by strategically querying informative data.
problem Strategic acquisition of informative data for Bayesian inference in challenging tasks.
method Unified framework combining amortized Bayesian inference and active data acquisition using a transformer architecture trained via reinforcement learning.
result ALINE delivers both instant and accurate inference along with efficient selection of informative points.
Neural networks speed up statistical inference.
problem Efficient statistical inference for complex models.
method Neural networks for learning complex mappings.
result Amortized inference speeds up inference processes.
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
Adaptive teacher improves sample efficiency and mode coverage in sampling tasks.
problem Efficient exploration and mode coverage in sampling tasks with amortized inference.
method An adaptive training distribution (teacher) guides the training of the primary sampler (student).
result Improves sample efficiency and mode coverage across various tasks.
Inference models are a key component in scaling variational inference to deep latent variable models, most notably as encoder networks in variational auto-encoders (VAEs). By replacing conventional optimization-based inference with a learned model, inference is amortized over data examples and therefore more computatio…
A new method decouples set representation learning from posterior modeling for efficient amortized inference.
problem Efficient inference for large sets of observations with shared factors.
method Train a mean-pool Deep Set on sets of size at most two, then finetune the inference head on pre-aggregated embeddings.
result Matches or outperforms standard baselines at a fraction of the compute cost for large N.
Improved diffusion sampling for inverse problems with faster and more robust inference.
problem High computational cost and lack of robustness in diffusion posterior sampling.
method Amortized variational inference with explicit likelihood guidance.
result Improved trade-off between inference speed and robustness to unseen degradations.
FoundCause: Causal Discovery with Latent Confounders from Observational Data
problem Causal discovery from observational data
method FoundCause, an amortized causal discovery model trained on synthetic data
result FoundCause outperforms classical and amortized methods on real-world datasets
We introduce "Search with Amortized Value Estimates" (SAVE), an approach for combining model-free Q-learning with model-based Monte-Carlo Tree Search (MCTS). In SAVE, a learned prior over state-action values is used to guide MCTS, which estimates an improved set of state-action values. The new Q-estimates are then used…
BayesFlow trains neural networks for fast Bayesian inference.
problem Fast Bayesian inference for complex models.
method Amortized neural networks for intractable posterior distributions.
result Fast inference through pre-trained neural networks.
Efficiently learns from partial labels using variational inference.
problem Learning from noisy and ambiguous partial labels in crowdsourcing.
method Amortized variational inference for probabilistic posterior approximation.
result Achieves state-of-the-art performance in accuracy and efficiency.