The paper proves almost sure convergence of MCES algorithm for a specific class of MDPs.
problem Establishing convergence for the Monte Carlo Exploring Starts (MCES) algorithm in reinforcement learning.
method Introduced a novel inductive approach based on the strong law of large numbers.
result Almost sure convergence for Optimal Policy Feed-Forward MDPs.
Along with the recent advances in scalable Markov Chain Monte Carlo methods, sampling techniques that are based on Langevin diffusions have started receiving increasing attention. These so called Langevin Monte Carlo (LMC) methods are based on diffusions driven by a Brownian motion, which gives rise to Gaussian proposa…
Paper proposes an ensemble-based AIS for multimodal sampling.
problem Sampling from multimodal distributions is challenging.
method Combines AIS with population-based Monte Carlo methods.
result Improves efficiency through ensemble interaction.
VCSMC improves efficiency in Bayesian phylogenetic inference.
problem Inefficient exploration of phylogenetic state space.
method Variational Combinatorial Sequential Monte Carlo (VCSMC) and nested CSMC.
result VCSMC and VNCSMC explore higher probability spaces efficiently.
New algorithm speeds up HMC by generating a warm start in O(d^1/4) iterations.
problem Unclear how many iterations of HMC are needed for high-dimensional sampling.
method Developed a non-Metropolized HMC that generates a warm start in O(d^1/4) iterations, followed by Metropolized HMC.
result Final complexity of O(d^1/4) is the fastest algorithm for high-accuracy sampling under strong log-concavity assumptions.
Enhances SMC² with Hessian info for more efficient posterior approximation.
problem Improving accuracy and efficiency in Bayesian inference.
method Integrates second-order information (Hessian) into SMC²'s proposal distribution.
result Second-order proposals lead to more accurate posterior approximations and better step-size selection.
The paper improves SMC algorithm for multi-modal distributions by proving variance bounds.
problem Problems with SMC on multi-modal distributions, especially in terms of mixing time.
method Proves variance bounds for SMC on multi-modal distributions using soft decomposition.
result Bounds on SMC variance depend on local rather than global mixing times.
Gradient learning optimises MCMC proposal distributions.
problem Intractable targets in MCMC sampling.
method Gradient-based optimisation of proposal distributions using a maximum entropy regularised objective function.
result Our method can outperform traditional MCMC algorithms, including Hamiltonian Monte Carlo.
This work improves convergence guarantees for unadjusted HMC in KL and Rényi divergences.
problem Understanding convergence properties of unadjusted HMC in divergences like KL and Rényi.
method One-shot couplings to establish regularization and lift convergence bounds.
result Quantitative control of relative density mismatch and warm-start requirements.
Develops a neural surrogate for proton dose calculation using Monte Carlo dropout uncertainty.
problem Computational demand in proton therapy workflows requiring repeated evaluations.
method Integrates Monte Carlo dropout into a neural network surrogate for fast, differentiable dose predictions and uncertainty quantification.
result Shows significant speedups over MC while retaining uncertainty information.
The paper uses Fourier integral theorem for estimating multivariate distributions.
problem Estimating multivariate distributions and conditional distribution functions.
method Natural Monte Carlo and fully nonparametric estimators based on Fourier integral theorem.
result Explicit Monte Carlo estimators without estimated covariance matrix.
The paper uses LSM to solve complex monetary utility functions.
problem Computing dynamic monetary utility functions with high dimensions.
method Least Squares Monte Carlo (LSM) algorithm.
result LSM algorithm successfully applied to recursive Cost-of-Capital valuation.
Adaptive SAA solves large-scale stochastic linear programs efficiently.
problem Solving large-scale two-stage stochastic linear programs.
method Iterative algorithm with adaptive sample size and warm starts.
result The algorithm converges to the true solution set with a probabilistic guarantee.
Developed scalable Monte Carlo method for VIX option pricing.
problem VIX option pricing in stochastic Volterra rough volatility models with non-Markovian vol-of-vol.
method Infinite dimensional Markovian representation to devise scalable least squares Monte Carlo.
result Efficient VIX option pricing method for generalized models.
S-VBMC improves VBMC's exploration of complex posterior distributions.
problem Efficient inference for computationally expensive models with complex posterior distributions.
method Stacking multiple independent VBMC runs to create a robust global posterior approximation.
result Significant improvements in posterior approximation quality across various applications.
The paper introduces new estimators for multivariate functions using Fourier methods.
problem Estimating multivariate functions like densities and regression functions.
method Monte Carlo estimators based on the Fourier integral theorem.
result Established rates of convergence for new estimators, often superior to existing methods.
New method uses symmetric splitting for efficient HMC inference in large neural networks.
problem Efficient inference for Bayesian neural networks with large datasets.
method Introduces a symmetric integration scheme for Hamiltonian Monte Carlo (HMC) that does not rely on stochastic gradients.
result Symmetric splitting leads to more efficient HMC inference over large data sets.
A Monte Carlo method for pairs trading on mean-reverting spreads with Lévy processes.
problem Trading on mean-reverting spreads with flexible models.
method Monte Carlo simulation with variance gamma and alpha-gamma driving processes.
result Optimal trading strategies are affected by model parameters and correlation.
MCNO learns PDE solution operators using Monte Carlo sampling.
problem Learning solution operators for PDEs efficiently and flexibly.
method Directly learns kernel function using Monte Carlo sampling of input-output pairs.
result Competitive accuracy with efficient computational cost on 1D PDE benchmarks.
This thesis tackles non-convex Bayesian learning via scalable dynamic importance sampling algorithms.
problem Non-convex Bayesian learning problem in deep neural networks.
method Replica exchange Langevin Monte Carlo, control variates method, population-chain replica exchange, scalable dynamic importance sampling.
result Control variates method reduces variance and accelerates convergence in non-convex Bayesian learning.
ChEES-HMC improves SMC samplers' efficiency and speed.
problem Efficiently sampling from complex posterior distributions.
method Incorporating ChEES-HMC into SMC samplers.
result ChEES-HMC outperforms NUTS in speed and efficiency.
This paper is a broad and accessible survey of the methods we have at our disposal for Monte Carlo gradient estimation in machine learning and across the statistical sciences: the problem of computing the gradient of an expectation of a function with respect to parameters defining the distribution that is integrated; t…
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
Paper proposes MCTSPO for better reinforcement learning policy optimization.
problem Local optima and saddle points in gradient-based methods and poor initialization in gradient-free methods.
method Monte-Carlo tree search combined with gradient-free optimization.
result Improved performance on reinforcement learning tasks with deceptive or sparse reward functions.
Study improves sampling from complex distributions using annealed Langevin Monte Carlo.
problem Sampling from non-log-concave and multimodal distributions.
method Annealed Langevin Monte Carlo algorithm with theoretical guarantees.
result Oracle complexity of O(dβ²A²/ε⁶) for achieving ε² accuracy in Kullback-Leibler divergence.
We describe an embarrassingly parallel, anytime Monte Carlo method for likelihood-free models. The algorithm starts with the view that the stochasticity of the pseudo-samples generated by the simulator can be controlled externally by a vector of random numbers u, in such a way that the outcome, knowing u, is determinis…
Improved efficiency in HMC samplers reduces dissipative behavior.
problem Reducing dissipative behavior in HMC samplers.
method Variable integration time and partial velocity refreshment.
result Efficiency improved by a √κ factor in Wasserstein-2 distance.
Active Reinforcement Learning (ARL) is a twist on RL where the agent observes reward information only if it pays a cost. This subtle change makes exploration substantially more challenging. Powerful principles in RL like optimism, Thompson sampling, and random exploration do not help with ARL. We relate ARL in tabular …
Combines MCTS and neural networks for efficient multi-period financial planning.
problem Solving multi-period financial planning models with high transaction costs and regime switching.
method Integrates Monte Carlo Tree Search with deep neural networks, using UTC and lookup search.
result Combined approach outperforms individual methods, solving complex models.
Simple mode exploration methods do not improve performance in neural networks.
problem Improving predictive probabilities in neural networks.
method Exploring local regions around diverse solutions using simple methods.
result Simple mode exploration methods do not improve performance.
This paper improves Bayesian decision tree learning using HMC.
problem Bayesian decision tree learning is challenging due to a large parameter space.
method Develops and compares HMC-based algorithms for exploring Bayesian decision tree posteriors.
result HMC-based methods outperform existing methods in predictive accuracy and tree complexity.
The paper explores how control variates can reduce variance in Monte Carlo simulations, especially for Sobolev functions.
problem Efficiency of control variates in reducing variance for Monte Carlo simulations.
method Study of a specific quadrature rule using nonparametric regression-adjusted control variates.
result A specific quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under sufficient smoothness assumptions.
Computing the marginal likelihood (ML) of a model requires marginalizing out all of the parameters and latent variables, a difficult high-dimensional summation or integration problem. To make matters worse, it is often hard to measure the accuracy of one's ML estimates. We present bidirectional Monte Carlo, a technique…
New symplectic scheme speeds up RMHMC.
problem Reducing computational burden in RMHMC.
method Explicit symplectic integration for non-separable Hamiltonians.
result Significant reduction in higher-order derivative calculations.
Markov Chain Monte Carlo methods have revolutionised mathematical computation and enabled statistical inference within many previously intractable models. In this context, Hamiltonian dynamics have been proposed as an efficient way of building chains which can explore probability densities efficiently. The method emerg…
This survey explores various optimality concepts in importance sampling.
problem Designing optimal proposal densities for Monte Carlo methods.
method Review of multiple frameworks and theoretical comparisons.
result Comprehensive understanding of optimality in importance sampling.
We consider parallel asynchronous Markov Chain Monte Carlo (MCMC) sampling for problems where we can leverage (stochastic) gradients to define continuous dynamics which explore the target distribution. We outline a solution strategy for this setting based on stochastic gradient Hamiltonian Monte Carlo sampling (SGHMC) …
The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset at hand. Mosaic, a Monte-Carlo tree search (MCTS) based approach, is presented to handle the AutoML hybrid structural and parametric expensiv…
This paper explores alternative regression techniques in pricing American put options and compares to the least-squares method (LSM) in Monte Carlo implemented by Longstaff-Schwartz, 2001 which uses least squares to estimate the conditional expected payoff to the option holder from continuation. The pricing is done und…
Paper addresses theoretical risks in neural MCCFR, proposing Robust Deep MCCFR for improved performance.
problem Theoretical risks in neural MCCFR, especially in large games.
method Adaptive framework with selective component deployment, including target networks, exploration, and variance-aware training.
result Robust Deep MCCFR achieves significant exploitability improvements in both Kuhn and Leduc Poker.
A new metric tensor improves Riemann manifold Monte Carlo for Bayesian models.
problem Improving sampling efficiency in Bayesian hierarchical models.
method Metric tensor derived from log-density gradient covariance matrices.
result Metric tensors enhance sampling for complex Bayesian models.
Quantum RNG improves financial risk metrics estimation.
problem Estimating financial risk metrics with high precision.
method Quantum-Enhanced Monte Carlo using QRNG.
result Improved accuracy in VaR and CVaR estimation.
In this paper, we present an online reinforcement learning algorithm, called Renewal Monte Carlo (RMC), for infinite horizon Markov decision processes with a designated start state. RMC is a Monte Carlo algorithm and retains the advantages of Monte Carlo methods including low bias, simplicity, and ease of implementatio…
New method tunes SMC samplers efficiently without high costs.
problem Tuning SMC samplers with unadjusted kernels is challenging.
method Greedy Incremental Divergence Minimization (GIDM) for step size tuning.
result GIDM reduces KL divergence and tunes SMC samplers efficiently.
Recent advances in stochastic gradient techniques have made it possible to estimate posterior distributions from large datasets via Markov Chain Monte Carlo (MCMC). However, when the target posterior is multimodal, mixing performance is often poor. This results in inadequate exploration of the posterior distribution. A…
This work introduces efficient sampling methods for Gaussian processes by focusing on pathwise conditioning.
problem Intractable mathematical expressions in Gaussian process posteriors limit practical applications.
method Investigates a pathwise interpretation of conditioning to derive efficient sampling methods.
result Derives a general family of approximations that allow for efficient sampling of Gaussian process posteriors.
Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their ov…
Paper proposes a new method for sampling from complex distributions.
problem Sampling from unnormalised density functions in complex distributions.
method Combines amortised and particle-based methods with reinforcement learning.
result Improves sampling from complex distributions compared to existing methods.