New algorithms improve MCMC efficiency for complex distributions.
problem High variance and low effective sample size in MCMC samplers.
method Antithetic Riemannian Manifold and Quantum-Inspired Hamiltonian Monte Carlo.
result Improved effective sample size and variance reduction.
This paper reviews various sampling methods from statistics and machine learning.
problem Addressing sampling methods in statistics and machine learning.
method Explains and reviews simple random sampling, bootstrapping, stratified sampling, cluster sampling, multistage sampling, network sampling, snowball sampling, and sampling from cumulative distribution function.
result Summarizes characteristics, pros, and cons of different sampling methods.
Combines normalizing flows and quasi-Monte Carlo for improved numerical integration.
problem Improving the efficiency of numerical integration methods.
method Uses normalizing flows to approximate distributions and quasi-Monte Carlo for sampling.
result Demonstrates an estimator with significantly lower variance.
SLMC improves sampling efficiency for high-dimensional distributions.
problem Sampling from high-dimensional distributions is computationally challenging.
method SLMC projects Langevin updates onto subsampled eigenblocks of a time-varying preconditioner.
result SLMC offers superior adaptability and computational efficiency compared to traditional methods.
PL-MCMC samples from normalizing flows' conditional distributions.
problem Sampling from complex conditional distributions learned by normalizing flows.
method Metropolis-Hastings implementation of PL-MCMC.
result PL-MCMC asymptotically samples from exact conditional distributions.
Generative models map simple samples to complex target samples.
problem Improving Monte-Carlo sampling techniques.
method Variational learning of dynamical maps between base and target measures.
result Improved sampling efficiency through feedback loops.
A new Monte Carlo sampling method derived from reverse diffusion.
problem Sampling from complex distributions, especially multi-modal ones.
method Transforming score matching into mean estimation; estimating means of regularized posterior distributions.
result rdMC can approximate sampling with any desired accuracy and is significantly faster than MCMC for complex distributions.
Introduces HMC method for sampling Gibbs densities.
problem Sampling from Gibbs densities efficiently.
method Hamiltonian Monte Carlo (HMC) method based on Hamiltonian dynamics.
result Idealized HMC preserves the target distribution and converges under certain conditions.
Deep learning enhances Hamiltonian Monte Carlo for sampling gauge field configurations.
problem Sampling from complex gauge field topologies efficiently.
method Stacked neural networks to generalize Hamiltonian Monte Carlo.
result Significantly reduces computational cost for generating gauge field configurations.
The paper improves Monte Carlo methods for optimization problems.
problem Efficiently solving optimization problems with biased Monte Carlo estimators.
method Introduces Multilevel Monte Carlo (MLMC) within Sample Average Approximation (SAA).
result Establishes uniform convergence and sample complexity for MLMC in SAA.
Study Langevin Monte Carlo for sampling non-log-concave distributions.
problem Sampling from non-log-concave distributions, especially Gaussian mixtures.
method Discretizations of overdamped Langevin diffusions.
result Numerical simulations compare Langevin Monte Carlo algorithms' performance.
Paper improves Monte Carlo sampling with new theoretical insights and methods.
problem Improving Monte Carlo sampling for variance reduction.
method Theoretical analysis of negatively dependent random variables and novel extensions using number theory and particle algorithms.
result Near-Orthogonal Monte Carlo (NOMC) consistently outperforms Orthogonal Monte Carlo (OMC) in various applications.
A new method combines AIS and SMCI for efficient evaluation of Ising models.
problem Efficiently evaluating expectations on Ising models under various temperatures.
method Combining Annealed Importance Sampling (AIS) and Spatial Monte Carlo Integration (SMCI).
result The proposed method performs efficiently in both high- and low-temperature regions.
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
Improved sampling for high-dimensional posteriors with underdamped Langevin.
problem Scalability issues in high-dimensional problems with approximate Thompson sampling.
method Underdamped Langevin Monte Carlo for accelerated posterior concentration.
result Logarithmic regret improvement from i l d e O ( d ) \mathcal{ ilde O}(d) i l d e O ( d ) to i l d e O ( d ) \mathcal{ ilde O}(\sqrt{d}) i l d e O ( d ) . New Hamiltonian Monte Carlo method for non-canonical dynamics.
problem Incompatibility of canonical symplectic structure with non-canonical dynamics.
method Developed a framework for Hamiltonian Monte Carlo using non-canonical symplectic structures with implicit integration.
result Non-canonical Hamiltonian Monte Carlo provides sampling advantages.
Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
RQMC improves optimization in variational Bayes problems.
problem Optimizing variational Bayes problems with noisy objective functions.
method Use of randomized quasi-Monte Carlo (RQMC) sampling with stochastic L-BFGS.
result RQMC can significantly speed up optimization and find better parameter values.
CHMC improves HMC efficiency for multimodal distributions.
problem Slow convergence of HMC in multimodal distributions.
method Integrates a counterdiabatic term to optimize Hamiltonian changes.
result CHMC achieves efficient sampling from challenging distributions.
We present a new method for conducting Monte Carlo inference in graphical models which combines explicit search with generalized importance sampling. The idea is to reduce the variance of importance sampling by searching for significant points in the target distribution. We prove that it is possible to introduce search…
AFT combines AIS, SMC, and NFs for better Monte Carlo estimates.
problem Estimating normalizing constants of complex probability distributions.
method Annealed Flow Transport (AFT) integrates AIS, SMC, and normalizing flows.
result AFT improves Monte Carlo estimates of normalizing constants and expectations.
Paper proposes a new method for sampling from complex distributions.
problem Sampling from unnormalised density functions in complex distributions.
method Combines amortised and particle-based methods with reinforcement learning.
result Improves sampling from complex distributions compared to existing methods.
Many machine learning problems involve Monte Carlo gradient estimators. As a prominent example, we focus on Monte Carlo variational inference (MCVI) in this paper. The performance of MCVI crucially depends on the variance of its stochastic gradients. We propose variance reduction by means of Quasi-Monte Carlo (QMC) sam…
New method reduces uncertainty in AI-driven Monte Carlo simulations.
problem Epistemic uncertainty in AI surrogate models affects Monte Carlo sampling outcomes.
method Penalty Ensemble Method (PEM) modifies Metropolis acceptance rule to increase rejection probability in uncertain regions.
result PEM enhances reliability of Monte Carlo simulations by reducing uncertainty propagation.
New method samples from piecewise smooth distributions using Hamiltonian Monte Carlo.
problem Sampling from distributions with discontinuous gradients.
method Generalized Randomized Hamiltonian Monte Carlo (GRHMC) for piecewise smooth targets.
result GRHMC processes sample from piecewise smooth target distributions with the desired distribution as the invariant distribution.
We consider the problem of simulating loss probabilities and conditional excesses for linear asset portfolios under the t-copula model. Although in the literature on market risk management there are papers proposing efficient variance reduction methods for Monte Carlo simulation of portfolio market risk, there is no pa…
A new method samples from a target density without initial samples using Monte Carlo estimation of the score.
problem Sampling from a target density without initial samples.
method Monte Carlo estimation of the score using oracle access to the log likelihood.
result Samples can be produced from the target density without needing initial samples.
New method reduces variance in estimating PL model expectations.
problem High variance in Monte Carlo estimates of PL model expectations.
method Combining Gumbel top-k trick with quasi-Monte Carlo sampling.
result More sample-efficient estimators of PL model expectations.
New Langevin algorithms improve sampling efficiency in high dimensions.
problem Sampling from log-concave and smooth distributions in high dimensions.
method Combining splitting and accurate integration methods for P P P -th order Langevin dynamics. result LMC algorithms converge faster with better dimension dependence as P P P increases. Monte Carlo (MC) techniques are often used to estimate integrals of a multivariate function using randomly generated samples of the function. In light of the increasing interest in uncertainty quantification and robust design applications in aerospace engineering, the calculation of expected values of such functions (e…
New insights into variational inference using Monte Carlo estimates.
problem Improving variational bounds in latent variable models.
method Analyzing properties of Monte Carlo estimates and their impact on variational gaps.
result Negative correlation reduces variational gaps, contrary to intuition.
MCNO learns PDE solution operators using Monte Carlo sampling.
problem Learning solution operators for PDEs efficiently and flexibly.
method Directly learns kernel function using Monte Carlo sampling of input-output pairs.
result Competitive accuracy with efficient computational cost on 1D PDE benchmarks.
LMC-TS uses MCMC for efficient posterior sampling in contextual bandits.
problem Efficiency of Thompson sampling for high-dimensional contextual bandits.
method Langevin Monte Carlo for direct posterior sampling.
result LMC-TS achieves sublinear regret bound for linear contextual bandits.
The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.
problem Sampling from complex target distributions efficiently.
method A neural network architecture inspired by Langevin Monte Carlo is proposed to map samples from a simple reference distribution to samples from the target.
result The proposed neural network architecture achieves approximation rates in the Wasserstein-2 distance for smooth, log-concave target distributions.
We propose a patch sampling strategy based on a sequential Monte-Carlo method for high resolution image classification in the context of Multiple Instance Learning. When compared with grid sampling and uniform sampling techniques, it achieves higher generalization performance. We validate the strategy on two artificial…
YOASOVI improves stochastic VI for large models with fast, self-correcting sampling.
problem Efficiently performing stochastic Variational Inference on large Bayesian models.
method YOASOVI uses acceptance sampling to draw only one sample per iteration, improving convergence speed and accuracy.
result YOASOVI converges faster and more accurately than regular Monte Carlo and Quasi-Monte Carlo methods.
Study improves sampling from complex distributions using annealed Langevin Monte Carlo.
problem Sampling from non-log-concave and multimodal distributions.
method Annealed Langevin Monte Carlo algorithm with theoretical guarantees.
result Oracle complexity of O(dβ²A²/ε⁶) for achieving ε² accuracy in Kullback-Leibler divergence.
Magnetic manifold HMC improves sampling on constrained manifolds.
problem Sampling from distributions restricted to embedded manifolds.
method Introduces magnetic manifold HMC, a generalization of HMC for constrained manifolds.
result Magnetic manifold HMC outperforms canonical manifold-constrained HMC.
Paper improves VAEs using Monte Carlo methods.
problem Improving the Evidence Lower Bound (ELBO) for VAEs.
method Uses Monte Carlo techniques to improve ELBO, specifically Sequential Importance Sampling (SIS) with carefully chosen kernels.
result Demonstrates improved performance on various applications.
The paper integrates multiple Gaussian process predictions using Monte Carlo sampling.
problem Accurate prediction of variables using multiple models.
method Log-linear pooling of Gaussian process predictions, combined with Monte Carlo sampling.
result The log-linear pooling method improves prediction accuracy compared to linear pooling.
Improved diffusion models using energy distillation and sequential Monte Carlo.
problem Training instability and inferior performance in energy parameterized diffusion models.
method Introduced a novel training regime for energy functions through distillation of pre-trained diffusion models, and cast the sampling procedure as a Feynman Kac model.
result Demonstrated improved performance and new sampling techniques.
A new algorithm improves sample complexity for thresholding in Monte Carlo Tree Search.
problem Determining if the root node value of a tree is at least a given threshold.
method Developed a δ-correct sequential sampling algorithm based on the Track-and-Stop strategy.
result Ratio-based modification of D-Tracking strategy reduces sample complexity and computational cost.
LMC improves sampling from complex distributions using quasi-random sequences.
problem Sampling from complex high-dimensional distributions with high accuracy.
method Using completely uniformly distributed (CUD) sequences in Langevin Monte Carlo (LMC) to generate Gaussian perturbations.
result LMC with low-discrepancy CUD sequences achieves smaller estimation error than standard LMC.
Bayesian framework for image inversion using regularization by denoising.
problem Image inversion and regularization in imaging tasks.
method Bayesian approach with Langevin-within-split Gibbs sampling.
result Demonstrates the effectiveness of the proposed method through numerical experiments.
This survey explores various optimality concepts in importance sampling.
problem Designing optimal proposal densities for Monte Carlo methods.
method Review of multiple frameworks and theoretical comparisons.
result Comprehensive understanding of optimality in importance sampling.
New method improves sampling from complex, multi-peaked distributions.
problem Sampling from high-dimensional, multimodal distributions using HMC.
method Combines tempered HMC with automatic tuning strategies.
result Demonstrates more effective scaling with dimension than adaptive methods.
Adaptive sampling method improves efficiency in complex target distributions.
problem Efficiency of importance sampling in complex target distributions, especially multimodal distributions in high-dimensional spaces.
method Proposes an adaptive scheme combining global sampling with delayed weighting to promote efficient exploration of target distributions.
result The proposed algorithm is geometrically convergent under mild assumptions and demonstrates improved efficiency in various numerical experiments.