Paper proposes ASR framework to improve image generation by incorporating human knowledge.
problem Deep generative models struggle to capture rich structures and relations in multi-object images.
method Introduces amortized structural regularization (ASR) framework using posterior regularization (PR) to embed human knowledge.
result Empirical results show ASR significantly outperforms DGM baselines in inference accuracy and sample quality.
The variational autoencoder (VAE) is a popular model for density estimation and representation learning. Canonically, the variational principle suggests to prefer an expressive inference model so that the variational approximation is accurate. However, it is often overlooked that an overly-expressive inference model ca…
Adversarial robustness of amortized Bayesian inference is studied, showing it can be improved.
problem Adversarial robustness of amortized Bayesian inference.
method Simulation-based estimation, regularization scheme based on Fisher information.
result Adversarial robustness can be improved with a regularization scheme.
Improved community detection in graphs with probabilistic models.
problem Lack of probabilistic formulation and fixed number of communities in GNN-based methods.
method Combines GNNs with amortized clustering for variable numbers of clusters.
result Improved performance on synthetic and real datasets compared to previous methods.
Recent efforts on combining deep models with probabilistic graphical models are promising in providing flexible models that are also easy to interpret. We propose a variational message-passing algorithm for variational inference in such models. We make three contributions. First, we propose structured inference network…
Efficiently predicts optimal transport plans using sliced potentials.
problem Predicting optimal transport plans across multiple measure pairs efficiently.
method Regression-based and objective-based amortization strategies using sliced optimal transport potentials.
result Efficient and accurate prediction of optimal transport plans for various tasks.
A new method learns causal structure from data using amortized inference.
problem Causal structure learning is a combinatorial search problem that is costly and difficult to design suitable scores or tests.
method Train a variational inference model to predict causal structure from data.
result Our inference model generalizes well to larger problem instances and outperforms existing algorithms, especially in genomics.
New method improves causal structure discovery with Prior-Fitted Networks.
problem Errors in likelihood estimation limit proper causal structure discovery.
method Amortized causal discovery with Prior-Fitted Networks.
result Significant gains in structure recovery compared to baselines.
New method learns causal models from data efficiently.
problem Learning Structural Causal Models from data is challenging.
method Amortized inference via Conditional Fixed-Point Iterations with transformer embeddings.
result Single model predicts causal mechanisms conditioned on data and graph.
Proposes ACP for efficient inference in noisy-or models.
problem Efficient inference in noisy-or models.
method Hybrid approach combining classical and modern variational inference.
result ACP outperforms or matches other approaches in noisy-or models.
FoundCause: Causal Discovery with Latent Confounders from Observational Data
problem Causal discovery from observational data
method FoundCause, an amortized causal discovery model trained on synthetic data
result FoundCause outperforms classical and amortized methods on real-world datasets
AQL uses amortized inference to handle high-dimensional action spaces in Q-learning.
problem Difficulty in maximizing over large action spaces in Q-learning.
method Replace expensive maximization over all actions with a maximization over a small subset sampled from a learned proposal distribution.
result AQL outperforms existing methods on continuous control tasks with up to 21 dimensional actions.
This paper examines how neural architectures support amortized Bayesian inference and its performance under varying conditions.
problem Understanding and evaluating amortized inference under signal-to-noise variation and distribution shift.
method Statistical analysis of neural architectures including feedforward networks, Deep Sets, and Transformers.
result Neural architectures support amortized Bayesian inference, offering controlled generalization error and robustness under varying conditions.
LazyDINO efficiently solves high-dimensional Bayesian inverse problems with fast and scalable solutions.
problem High-dimensional nonlinear Bayesian inverse problems with expensive parameter-to-observable maps.
method LazyDINO combines derivative-informed neural surrogates and lazy map variational inference for efficient posterior approximation.
result Significant cost reduction in amortized Bayesian inversion, achieving one to two orders of magnitude improvement.
Improved VAEs learn consistent posterior distributions from missing data.
problem Learning VAEs from data with missing values, especially in the encoder.
method Formal definition of posterior consistency and regularization approach.
result Regularization leads to improved performance in reconstruction and downstream tasks.
Inference amortization methods share information across multiple posterior-inference problems, allowing each to be carried out more efficiently. Generally, they require the inversion of the dependency structure in the generative model, as the modeller must learn a mapping from observations to distributions approximatin…
This paper speeds up inference in large hierarchical models.
problem Inference in large hierarchical models is slow and difficult.
method Amortized variational inference with shared parameters.
result Amortized inference is as accurate as a full-rank Gaussian but much faster.
APO optimizes neural network parameters by amortizing proximal point methods.
problem Optimizing neural network parameters online and adaptively.
method APO framework that meta-learns proximal point parameters.
result APO can recover and outperform existing optimizers and schedules.
Probabilistic programming languages (PPLs) are a powerful modeling tool, able to represent any computable probability distribution. Unfortunately, probabilistic program inference is often intractable, and existing PPLs mostly rely on expensive, approximate sampling-based methods. To alleviate this problem, one could tr…
New approach avoids meta-amortization error in few-shot learning.
problem Task ambiguity in few-shot learning makes learning new tasks difficult.
method Cyclical annealing schedule and maximum mean discrepancy criterion.
result Substantially outperforms standard meta-learning algorithms.
AMCI improves Monte Carlo integration by amortizing over both datasets and target functions.
problem Inefficiency in approximating expectations for known target functions using current approaches.
method Introduces AMCI, a method for amortizing Monte Carlo integration directly, producing three distinct amortized proposals.
result AMCI can theoretically produce arbitrarily small errors for any integrable target function using only a single sample from each proposal at runtime.
Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amor…
New method improves ABI for sequential data, reducing forgetting and improving accuracy.
problem Performance degradation of ABI under model misspecification and distribution shifts.
method Decouples simulation-based pre-training from unsupervised SC fine-tuning, using memory buffer and elastic weight consolidation.
result Significant mitigation of forgetting and improved posterior estimates compared to standard simulation-based training.
Delta-AI speeds up inference in sparse PGMs by local credit assignment.
problem Efficient inference in sparse probabilistic graphical models.
method Local credit assignment in agent's policy learning objective.
result Trained sampler recovers marginals and conditional distributions.
CogFormer trains a transformer to estimate parameters across various cognitive models.
problem Difficulty in fitting complex cognitive models and iterating over varying assumptions.
method Meta-amortized framework using a transformer to estimate parameters across multiple models.
result CogFormer accurately estimates parameters across different model families with minimal retraining.
This paper reviews recent advancements in amortized Variational Inference.
problem Scalability and efficiency issues in traditional Variational Inference.
method Systematic review of various Variational Inference techniques, focusing on amortized approaches.
result Amortized Variational Inference improves scalability and efficiency for generative modeling tasks.
Improves point-cloud reconstruction by optimizing projections with self-attention.
problem Inefficient and non-metric projection methods for sliced Wasserstein distances.
method Proposes distributional sliced Wasserstein distance with self-attention for permutation-invariant and metric optimization.
result Self-attention amortized distributional projection optimization achieves better performance in point-cloud reconstruction.
New option type preserves fungibility by amortizing payments over time.
problem Traditional installment options destroy fungibility and lapse when payments stop.
method Introduces amortizing perpetual options (AmPOs) with an implicit payment scheme.
result Valuation of AmPOs reduces to vanilla perpetual American options.
Develops theory for conditional optimal transport in infinite-dimensional spaces.
problem Bayesian inference with functional parameters in infinite-dimensional spaces.
method Theory of constrained optimal transport for block-triangular maps.
result Regularity estimates on conditioning maps from prior to posterior.
A new method learns posterior and predictive distributions together, reducing computational cost.
problem Sequential two-stage Bayesian inference is computationally expensive.
method Amortized variational inference targeting posterior-predictive distribution.
result Efficient online inference with more accurate predictive distributions.
A new method optimizes projection directions for sliced Wasserstein distances.
problem Finding informative projecting directions for sliced Wasserstein distances is computationally expensive.
method Amortized projection optimization to predict directions efficiently.
result Proposed amortized models improve generative modeling performance.
Adaptive workflow combines fast amortized inference with MCMC for many datasets.
problem Trade-off between computational speed and sampling accuracy in Bayesian inference.
method Adaptive workflow integrating amortized inference and MCMC with principled diagnostics.
result Efficiency gains with high posterior quality on tens of thousands of datasets.
Transformer-based method improves causal discovery from observational data.
problem Causal discovery from observational data requires explicit assumptions.
method CSIvA transformer architecture trained on synthetic data.
result Transformer-based methods adhere to identifiability theory.
ASPIRE improves amortized posterior inference for Bayesian inverse problems.
problem Bayesian inverse problems are computationally challenging due to uncertainty quantification.
method Iterative refinement of amortized posteriors using physics-based and summary statistics.
result ASPIRE achieves better posterior approximations with minimal extra computations.
Framework creates fast, interpretable surrogates for stochastic simulators with unbounded randomness.
problem Creating accurate and fast approximations for stochastic simulators with unbounded randomness.
method Probabilistic surrogate networks that retain structure of reference simulators and enable amortized inference.
result Surrogates accurately model stochastic programs with unbounded random variables and significantly speed up inference.
Proposes efficient sensitivity analysis for complex Bayesian models.
problem Inefficiency of sensitivity analyses in complex Bayesian models.
method SA-ABI: weight sharing and neural network rapid inference.
result Efficiently integrates sensitivity analyses into Bayesian inference.
PAVI speeds up Bayesian inference for large datasets.
problem Challenges in Bayesian inference for large population studies.
method Plate-amortized Variational Inference (PAVI) that shares parameterization across i.i.d. variables.
result Significant speedup in training variational distributions, orders of magnitude faster.
Deriving and applying Proximal Policy Optimization to GFlowNets for efficient training of discrete sampling policies
problem Training stochastic policies to sample from structured discrete probability distributions
method Deriving policy gradient algorithms for GFlowNets and applying Proximal Policy Optimization
result Improved convergence speed and data efficiency compared to standard GFlowNet training objectives
JANA trains networks to approximate Bayesian models efficiently.
problem Intractable likelihood functions and posterior densities in Bayesian models.
method End-to-end training of three networks: summary, posterior, and likelihood networks.
result JANA provides accurate amortized marginal likelihood and posterior predictive estimation.
APG samplers use neural suff stats to improve deep model inference.
problem Efficient inference in deep generative models.
method Amortized population Gibbs, neural suff stats, KL divergence minimization.
result Significant improvement in inference accuracy.
Improved diffusion sampling for inverse problems with faster and more robust inference.
problem High computational cost and lack of robustness in diffusion posterior sampling.
method Amortized variational inference with explicit likelihood guidance.
result Improved trade-off between inference speed and robustness to unseen degradations.
BayesFlow trains neural networks for fast Bayesian inference.
problem Fast Bayesian inference for complex models.
method Amortized neural networks for intractable posterior distributions.
result Fast inference through pre-trained neural networks.
New method finds all thin film structures from reflectometry data.
problem Computational prohibitive for standard algorithms, leading to unreliable analysis.
method Prior-Amortized Neural Posterior Estimation (PANPE) combining simulation-based inference and adaptive priors.
result Identifies all realistic structures in seconds, setting new standards in reflectometry.
Neural clustering learns time series affinity from statistical features.
problem Challenging time series clustering with unknown cluster shapes and structures.
method Amortized neural inference using statistical features.
result Competitive clustering accuracy without manual specification of cluster shapes.
Simformer uses transformer models to perform flexible Bayesian inference.
problem Current simulation-based inference methods are inflexible and require fixed priors.
method Trains a probabilistic diffusion model with transformer architectures.
result Outperforms state-of-the-art methods on various benchmarks.
AR-DAE approximates entropy gradient for machine learning models.
problem Intractable computation of entropy gradient for continuous distributions.
method Amortized residual denoising autoencoder (AR-DAE) to approximate entropy gradient.
result AR-DAE provides an unbiased gradient approximation for entropy.
New method improves generalization of VAEs by reducing overfitting.
problem Generalization issues in VAEs overfitting to training data.
method Proposed a new training objective to improve amortized inference.
result Improved performance in image modeling and lossless compression.
New method reduces infinite variance in probabilistic programs with rejection sampling.
problem Infinite variance in naive importance sampling for programs with rejection sampling.
method Developed a new amortized importance sampling estimator with finite variance proof.
result Empirically demonstrated efficiency and correctness compared to existing alternatives.