CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Annealed Importance Sampling”
mAIS improves free energy evaluation efficiency.
problem Computational infeasibility of exact free energy evaluation.
method mAIS, a marginalized version of AIS.
result mAIS is more efficient under certain conditions.
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimationstat.ML
We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a sequential factorization of the marginal likelihood in such settings by using stochastic gradient Markov Chain Monte Carlo techniques. This appro…
Kernel methods have revolutionized the fields of pattern recognition and machine learning. Their success, however, critically depends on the choice of kernel parameters. Using Gaussian process (GP) classification as a working example, this paper focuses on Bayesian inference of covariance (kernel) parameters using Mark…
New complexity analysis for estimating normalizing constants in high dimensions.
problem Estimating the normalizing constant of unnormalized probability densities in high dimensions.
method Analyze and derive the oracle complexity of annealed importance sampling.
result Oracle complexity of $\widetilde{O}\left(\frac{dβ^2{\mathcal{A}}^2}{\varepsilon^4}
ight)$ for estimating within relative error.
Optimizes AIS hyperparameters for efficient marginal likelihood estimation.
problem Limited computation budget affects AIS performance.
method Flexible intermediary distributions defined by residual density, parameter sharing, and fix linear schedule.
result Optimized-Path AIS reduces sampling iterations and improves performance.
Annealed importance sampling (AIS) is a common algorithm to estimate partition functions of useful stochastic models. One important problem for obtaining accurate AIS estimates is the selection of an annealing schedule. Conventionally, an annealing schedule is often determined heuristically or is simply set as a linear…
New method improves performance of Hamiltonian MCMC for log Z estimation.
problem Estimating tight bounds on log Z for unnormalized distributions.
method Uncorrected Hamiltonian Annealing (UHA) using reparameterization gradients.
result Better performance and easier parameter tuning compared to existing methods.
New methods improve Monte Carlo estimation of partition functions.
problem Estimating the normalization constant of complex distributions.
method Annealing through paths of distributions to estimate partition functions.
result Optimal path for estimation is arithmetic, improving efficiency.
A new method combines AIS and SMCI for efficient evaluation of Ising models.
problem Efficiently evaluating expectations on Ising models under various temperatures.
method Combining Annealed Importance Sampling (AIS) and Spatial Monte Carlo Integration (SMCI).
result The proposed method performs efficiently in both high- and low-temperature regions.
Paper proposes an ensemble-based AIS for multimodal sampling.
problem Sampling from multimodal distributions is challenging.
method Combines AIS with population-based Monte Carlo methods.
result Improves efficiency through ensemble interaction.
AFT combines AIS, SMC, and NFs for better Monte Carlo estimates.
problem Estimating normalizing constants of complex probability distributions.
method Annealed Flow Transport (AFT) integrates AIS, SMC, and normalizing flows.
result AFT improves Monte Carlo estimates of normalizing constants and expectations.
Improved variational inference for GPLVMs using AIS.
problem Challenges in generating effective proposal distributions for high-dimensional or complex data.
method Annealed Importance Sampling (AIS) combined with reparameterization.
result Our method achieves tighter variational bounds and higher log-likelihoods.
AIS uses a suboptimal extended target distribution, which this paper improves using SGM.
problem Improving the efficiency of Annealed Importance Sampling for marginal likelihood estimation.
method Leveraging score-based generative modeling to approximate the optimal extended target distribution.
result Demonstrated novel, differentiable AIS procedures on synthetic and real-world data.
Bootstrap Your Flowcs.LG
FAB combines flows with AIS to approximate complex distributions.
problem Challenges in flow-based methods, especially on complex targets.
method Combines flows with AIS, using α-divergence for training.
result FAB produces accurate approximations to complex distributions.
Restricted Boltzmann Machine (RBM) is a bipartite graphical model that is used as the building block in energy-based deep generative models. Due to numerical stability and quantifiability of the likelihood, RBM is commonly used with Bernoulli units. Here, we consider an alternative member of exponential family RBM with…
Variational inference (VI) and Markov chain Monte Carlo (MCMC) are two main approximate approaches for learning deep generative models by maximizing marginal likelihood. In this paper, we propose using annealed importance sampling for learning deep generative models. Our proposed approach bridges VI with MCMC. It gener…
DAIS minimizes symmetrized KL divergence between initial and target distributions.
problem Optimizing over initial distributions in importance sampling.
method Differentiable annealed importance sampling (DAIS) minimizing symmetrized KL divergence.
result DAIS minimizes symmetrized KL divergence between initial and target distributions.
Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) in…
DAIS improves AIS for differentiable marginal likelihood estimation.
problem Differentiable marginal likelihood estimation for complex models.
method Proposes Differentiable Annealed Importance Sampling (DAIS) to make AIS differentiable.
result DAIS achieves convergence and consistency in Bayesian linear regression.
EWFM trains continuous flows with only energy evaluations, improving sample quality with fewer computations.
problem Efficiently sampling from complex, high-dimensional Boltzmann distributions using only energy evaluations.
method Energy-Weighted Flow Matching (EWFM) using importance sampling and iterative/annealed training.
result Improved sample quality with up to 3 orders of magnitude fewer energy evaluations compared to existing methods.
AIS method improves estimation of RBM partition function with reduced computational cost.
problem Efficiently estimating partition function of RBMs for large systems.
method Annealed Importance Sampling (AIS) with optimized initialization.
result Good estimation of partition function Z with reduced computational cost.
Develops inference combinators for probabilistic programs using neural networks.
problem Creating efficient proposals for probabilistic program inference.
method Inference combinators using neural network parameterization of proposals.
result Correct by construction variational methods tailored to specific models.
DAIS improves AIS by resampling, avoiding gradient issues.
problem Low effective sample size in DAIS.
method DAIS with resampling step to improve efficiency.
result Resampling step avoids gradient variance issues.
D-Wave quantum annealing fails to improve sampling quality from RBMs compared to Gibbs sampling.
problem Improving sampling quality from RBMs using D-Wave quantum annealing.
method Comparison of D-Wave quantum annealing and Gibbs sampling for RBM sampling.
result D-Wave sampling does not significantly improve the number of local valleys compared to Gibbs sampling.
Flow AIS Bootstrap improves flow training by generating samples in hard-to-reach regions.
problem Training flows with high variance and mode-seeking behavior.
method Augment flows with AIS and minimize -divergence with .
result FAB learns Boltzmann distribution of alanine dipeptide without MD samples.
New MI bounds improve estimation in deep generative models.
problem Estimating mutual information without density information is intractable.
method Importance sampling, Annealed Importance Sampling, Generalized IWAE, MINE-AIS.
result Improved bounds for estimating mutual information in deep models.
New method uses Rashomon sets to improve Bayesian inference in factorial designs.
problem Combustion of model uncertainty in factorial designs leads to multimodal posterior and convergence issues.
method Rashomon-seeded annealing, integrating high-performing models as warm start for AIS.
result Restores full posterior inference without exhaustive enumeration of model space.
We study the problem of training machine learning models incrementally with batches of samples annotated with noisy oracles. We select each batch of samples that are important and also diverse via clustering and importance sampling. More importantly, we incorporate model uncertainty into the sampling probability to com…
Introduces q-paths for generalizing geometric annealing paths in machine learning.
problem Limited applicability of existing path methods in machine learning.
method Develops a family of paths derived from a generalized mean, including geometric and arithmetic mixtures.
result Empirical gains in Bayesian inference and generative model evaluation.
Study improves sampling from complex distributions using annealed Langevin Monte Carlo.
problem Sampling from non-log-concave and multimodal distributions.
method Annealed Langevin Monte Carlo algorithm with theoretical guarantees.
result Oracle complexity of O(dβ²A²/ε⁶) for achieving ε² accuracy in Kullback-Leibler divergence.
New analysis of annealing paths in sampling and estimation.
problem Sampling from complex distributions and estimating normalization constants.
method Extending known results on Bregman divergence to quasi-arithmetic means under monotonic embedding.
result Analogous result for quasi-arithmetic means, highlighting the interplay between means, parametric families, and divergence functionals.
The estimation of normalizing constants is a fundamental step in probabilistic model comparison. Sequential Monte Carlo methods may be used for this task and have the advantage of being inherently parallelizable. However, the standard choice of using a fixed number of particles at each iteration is suboptimal because s…
Nested Variational Inferencestat.ML
Develops a method for learning proposals in nested importance samplers.
problem Improving sampling quality in complex distributions.
method Nested Variational Inference (NVI) using forward or reverse KL divergence.
result Optimizing nested objectives leads to improved sample quality.
LFIS uses a time-dependent velocity field to sample from complex distributions.
problem Sampling from unnormalized density functions.
method LFIS learns a time-dependent velocity field to transport samples from a simple initial distribution to a complex target distribution.
result LFIS achieves state-of-the-art performance on various benchmark problems.
Paper improves VAEs using Monte Carlo methods.
problem Improving the Evidence Lower Bound (ELBO) for VAEs.
method Uses Monte Carlo techniques to improve ELBO, specifically Sequential Importance Sampling (SIS) with carefully chosen kernels.
result Demonstrates improved performance on various applications.
The study optimizes simulated annealing's cooling schedule for better performance.
problem Designing optimal cooling schedules for simulated annealing to improve its performance.
method Analyzed the cooling schedule's impact on simulated annealing's performance and provided sample and simulation complexity results.
result Optimal cooling schedules can be found with a small number of samples, improving the algorithm's runtime or success rate.
Markov random fields (MRFs) are difficult to evaluate as generative models because computing the test log-probabilities requires the intractable partition function. Annealed importance sampling (AIS) is widely used to estimate MRF partition functions, and often yields quite accurate results. However, AIS is prone to ov…
Self-regulating annealing improves sampling from heavy-tailed datasets.
problem Sampling from heavy-tailed distributions using diffusion models.
method Proposed an SDE-based sampler with a state-dependent diffusion coefficient.
result State dependence induces a self-regulating annealing mechanism.
Maximum likelihood estimation (MLE) is one of the most important methods in machine learning, and the expectation-maximization (EM) algorithm is often used to obtain maximum likelihood estimates. However, EM heavily depends on initial configurations and fails to find the global optimum. On the other hand, in the field …
Markov chain Monte Carlo (MCMC) is one of the main workhorses of probabilistic inference, but it is notoriously hard to measure the quality of approximate posterior samples. This challenge is particularly salient in black box inference methods, which can hide details and obscure inference failures. In this work, we ext…
Study shows annealing with adaptive schedule reduces mode collapse in NFs for parameter estimation.
problem Mode collapse in normalizing flows for multimodal distributions.
method Annealing with an adaptive schedule based on effective sample size (ESS).
result Our approach reduces mode collapse and converges marginal likelihood faster than MCMC methods.
We describe an adaptation of the simulated annealing algorithm to nonparametric clustering and related probabilistic models. This new algorithm learns nonparametric latent structure over a growing and constantly churning subsample of training data, where the portion of data subsampled can be interpreted as the inverse …
AGGAN uses genetic algorithm with simulated annealing to generate minority class data.
problem Overcoming class imbalance in minority class data.
method AGGAN combines genetic algorithm and simulated annealing to train GANs on scarce minority class data.
result AGGAN effectively generates minority class data distributions from limited samples.
Paper analyzes Annealed Langevin Dynamics for multimodal sampling stability.
problem Ensuring stability of Annealed Langevin Dynamics across dimensions.
method Uniform-in-dimension analysis of ALD for Gaussian-mixture targets.
result ALD achieves prescribed accuracy in KL divergence with spectral conditions.
Quantum machine learns faster by reverse annealing on AQCs.
problem Training RBMs on AQCs is hard due to low qubit connectivity.
method Embedding RBM nodes to virtual qubits, semantic quantum search, reverse annealing schedule.
result Reverse annealing accelerates RBM training and improves reconstruction scores.
Statistical Mechanics Calculations Using Variational Autoregressive Networks and Quantum Annealingcond-mat.dis-nn
New method uses quantum annealing and VAN for better statistical mechanics calculations.
problem Difficulty in computing partition function in statistical mechanics.
method Combines quantum annealing samples with variational autoregressive networks.
result Enhanced accuracy in finite-size Sherrington-Kirkpatrick model.