This paper develops scalable control variates for Monte Carlo methods using stochastic optimization.
problem Reducing variance in Monte Carlo estimators for large-scale problems.
method Control variates based on Stein operators, optimized through stochastic optimization.
result Novel theoretical results and empirical validations show effective variance reduction.
The paper explores how control variates can reduce variance in Monte Carlo simulations, especially for Sobolev functions.
problem Efficiency of control variates in reducing variance for Monte Carlo simulations.
method Study of a specific quadrature rule using nonparametric regression-adjusted control variates.
result A specific quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under sufficient smoothness assumptions.
ControlSHAP stabilizes Shapley value approximations using control variates.
problem High computational cost of exact Shapley values in blackbox models.
method ControlSHAP uses Monte Carlo control variates to stabilize Shapley value approximations.
result Significant reduction in Monte Carlo variability of Shapley estimates.
New method reduces inference variance for faster optimization.
problem High variance in black-box variational inference.
method Joint control variate addressing both data subsampling and Monte Carlo noise.
result Significantly reduced gradient variance, leading to faster optimization.
We use neural networks as control variates with geometric integration techniques.
problem Analytic integration of neural network approximations for variance reduction.
method Integration domain subdivision using computational geometry for MLPs with continuous piecewise linear activation functions.
result Neural networks can be used as control variates with geometric integration methods.
A new weighted MLMC method improves efficiency in Monte Carlo simulations.
problem Improving efficiency in Monte Carlo simulations with correlated coarse level approximations.
method Generalization of MLMC to any number of levels with control variates and weights.
result Significant efficiency improvements possible, especially when coarse level approximations are poorly correlated.
PEMC uses ML to enhance Monte Carlo simulations, reducing variance and runtime.
problem Computational inefficiency in Monte Carlo simulations for complex tasks.
method Prediction-Enhanced Monte Carlo (PEMC) framework that uses ML surrogates as predictors.
result PEMC provides unbiased evaluations with reduced variance and runtime compared to standard Monte Carlo.
Combines control variates and adaptive importance sampling for Monte Carlo integration.
problem Improving Monte Carlo integration accuracy with control variates and adaptive sampling.
method A quadrature rule combining control variates and adaptive importance sampling.
result Non-asymptotic bound on the probabilistic error of the procedure.
Improved Least-Squares Monte Carlo with finite-difference ansatz.
problem Improving accuracy and stability in option pricing.
method Constructing an ansatz using finite-difference solution for conditional expected continuation payoffs.
result Reduces mean squared error and final pricing error.
A new method using spherical harmonics approximates the Sliced-Wasserstein distance.
problem Approximating the Sliced-Wasserstein distance between probability measures.
method Spherical Harmonics Control Variates (SHCV) method for Monte Carlo approximation of the SW distance.
result SHCV method provides an improved rate of convergence compared to Monte Carlo for general measures.
A Monte Carlo method for pairs trading on mean-reverting spreads with Lévy processes.
problem Trading on mean-reverting spreads with flexible models.
method Monte Carlo simulation with variance gamma and alpha-gamma driving processes.
result Optimal trading strategies are affected by model parameters and correlation.
NCV uses neural networks to improve Monte Carlo integration.
problem Improving variance reduction in parametric Monte Carlo integration.
method NCV combines a normalizing flow and a neural network to approximate the integrand and solve the integral equation, with a neural importance sampler to estimate the difference.
result NCV achieves state-of-the-art performance in light transport simulation with reduced noise and negligible bias.
CMCD sampler connects transport and variational inference for efficient sampling.
problem Efficient sampling and generative modeling in Bayesian computation.
method Developed a principled framework using divergences on path space, CMCD sampler with adaptive dynamics.
result CMCD sampler outperforms competing approaches across various experiments.
Neural SDEs reduce variance in stochastic simulations.
problem Efficiency of Monte Carlo simulations in finance.
method Use neural SDEs with control variates parameterized by neural networks.
result Prove optimality conditions for variance reduction in SDEs with infinite activity.
We introduce a stacking version of the Monte Carlo algorithm in the context of option pricing. Introduced recently for aeronautic computations, this simple technique, in the spirit of current machine learning ideas, learns control variates by approximating Monte Carlo draws with some specified function. We describe the…
Monte Carlo (MC) sampling algorithms are an extremely widely-used technique to estimate expectations of functions f(x), especially in high dimensions. Control variates are a very powerful technique to reduce the error of such estimates, but in their conventional form rely on having an accurate approximation of f, a pri…
Meta-CVs leverage task similarity to reduce variance with limited data.
problem Reducing variance in Monte Carlo estimators with few samples.
method Meta-learning control variates for related tasks.
result Meta-CVs lead to significant variance reduction in settings with limited data.
In statistics and machine learning, approximation of an intractable integration is often achieved by using the unbiased Monte Carlo estimator, but the variances of the estimation are generally high in many applications. Control variates approaches are well-known to reduce the variance of the estimation. These control v…
We show that deliberately introducing a nested simulation stage can lead to significant variance reductions when comparing two stopping times by Monte Carlo. We derive the optimal number of nested simulations and prove that the algorithm is remarkably robust to misspecifications of this number. The method is applied to…
Develops diffusion samplers for target distributions with efficient score and density estimates.
problem Estimating scores and densities for time-varying distributions.
method Sequential Monte Carlo with diffusion paths and control variates.
result Effective samplers for time-varying distributions with theoretical guarantees and practical applications.
Monte Carlo Tree Search improves financial derivative hedging efficiency.
problem Optimizing pricing and hedging of derivative contracts in incomplete markets.
method Integrates tree search techniques with Reinforcement Learning for optimal control problems.
result Monte Carlo Tree Search outperforms Q-learning in sample efficiency and learning speed. We introduce a method for non-uniform random number generation based on sampling a physical process in a controlled environment. We demonstrate one proof-of-concept implementation of the method that reduces the error of Monte Carlo integration of a univariate Gaussian by 1068 times while doubling the speed of the Monte…
We reduce variance in Bures-Wasserstein variational inference.
problem High variance in Monte Carlo approximations of Bures-Wasserstein gradients.
method Control variates to reduce variance in the forward step.
result Proposed estimator reduces variance by orders of magnitude.
Paper proposes a closed-form formula for geometric Istanbul call options.
problem Pricing geometric Istanbul call options under the Black-Scholes model.
method Second-order Taylor expansion to derive a closed-form approximation.
result The proposed formula accurately approximates GIC values compared to Monte-Carlo simulations.
Develops numerical methods for pricing exchange options in a market with limited liquidity.
problem Pricing European style exchange options in a market with finite liquidity.
method Integrates price impact into the dynamics of correlated assets using a controlled variate approach.
result Numerical pricing methods for exchange options are developed and validated.
New insights into variational inference using Monte Carlo estimates.
problem Improving variational bounds in latent variable models.
method Analyzing properties of Monte Carlo estimates and their impact on variational gaps.
result Negative correlation reduces variational gaps, contrary to intuition.
New method reduces variance in Bayesian inverse problems.
problem High variance in Monte Carlo estimates for inverse problems.
method Conditional neural control variates based on Stein's identity.
result Substantial variance reduction across different inverse problems.
Proposes variance reduction techniques for sliced Wasserstein distance estimation.
problem Intractability of estimating sliced Wasserstein distances.
method Uses control variates based on Gaussian approximations of projected measures.
result Significant reduction in variance of SW distance estimators.
New distributions on manifolds for better sampling.
problem Creating flexible distributions on Riemannian manifolds.
method Area-preserving maps and isometries for constructing distributions.
result Flexibility and straightforward sampling of distributions.
Proposes a method to stabilize Black Box Variational Inference using the James-Stein estimator.
problem Stability issues and fine-tuning required in basic Black Box Variational Inference.
method Reframe stochastic gradient ascent as multivariate estimation problem using James-Stein estimator.
result Provides a simpler method with consistent performance in terms of model fit and convergence time.
RQMC improves optimization in variational Bayes problems.
problem Optimizing variational Bayes problems with noisy objective functions.
method Use of randomized quasi-Monte Carlo (RQMC) sampling with stochastic L-BFGS.
result RQMC can significantly speed up optimization and find better parameter values.
AFT combines AIS, SMC, and NFs for better Monte Carlo estimates.
problem Estimating normalizing constants of complex probability distributions.
method Annealed Flow Transport (AFT) integrates AIS, SMC, and normalizing flows.
result AFT improves Monte Carlo estimates of normalizing constants and expectations.
A new variational method for SSMs improves inference efficiency.
problem Hard variational inference for state space models.
method Proposes variational marginal particle filter (VMPF) based on Rao-Blackwellization.
result VMPF provides tighter variational bounds and sometimes benefits from unbiased reparameterization.
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
There are no known exact formulas for the valuation of a number of exotic options, and this is particularly true for options under discrete monitoring and for American style options. Therefore, one usually recourses to a Monte Carlo Simulation approach, amongst other numerical methods, to estimate the value of these op…
VCSMC improves efficiency in Bayesian phylogenetic inference.
problem Inefficient exploration of phylogenetic state space.
method Variational Combinatorial Sequential Monte Carlo (VCSMC) and nested CSMC.
result VCSMC and VNCSMC explore higher probability spaces efficiently.
In this short note we provide an unbiased multilevel Monte Carlo estimator of the log marginal likelihood and discuss its application to variational Bayes.
Paper advances black-box VI using flows and Monte-Carlo methods.
problem Improving automatic posterior inference in black-box VI.
method Combines normalizing flows, Monte-Carlo methods, and optimization considerations.
result Significant improvement in state-of-the-art variational inference.
In this paper a novel modification of the multilevel Monte Carlo approach, allowing for further significant complexity reduction, is proposed. The idea of the modification is to use the method of control variates to reduce variance at level zero. We show that, under a proper choice of control variates, one can reduce t…
Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on (i) formulating a flexible parametric family of distributions, and (ii) optimizing the parameters to find the member of this family that most closely approximates the exact posterior…
Variational Inference shows promise for Bayesian GARCH model estimation.
problem Bayesian estimation of GARCH-family models using Monte Carlo sampling.
method Variational Inference as an alternative to Monte Carlo sampling.
result Variational Inference is a reliable and competitive method for Bayesian learning in GARCH-like models.
Unified framework for output analysis using Monte Carlo sampling.
problem Accurately assess the quality of estimated values in predictive models.
method Unified output analysis framework through Monte Carlo sampling, leveraging fast iterative bootstrap sampling and higher-order influence functions.
result Clear advantage in building more robust confidence intervals with higher coverage probability.
New sampling methods improve statistical efficiency for intractable targets.
problem Sampling from complex, intractable probability distributions.
method Gaussian invariant versions of RWM, MALA, and Hessian MALA.
result Gaussian invariant sampling leads to improved statistical efficiency.
Efficiently reduces computational burden of rollout acquisition functions in Bayesian optimization.
problem Expensive computation of rollout acquisition functions in Bayesian optimization.
method Combines quasi-Monte Carlo, common random numbers, and control variates to reduce computational burden. Formulates a policy-search approach to eliminate the need to optimize the rollout acquisition function.
result Significant reduction in computational burden of rollout acquisition functions.
Improved diffusion models using energy distillation and sequential Monte Carlo.
problem Training instability and inferior performance in energy parameterized diffusion models.
method Introduced a novel training regime for energy functions through distillation of pre-trained diffusion models, and cast the sampling procedure as a Feynman Kac model.
result Demonstrated improved performance and new sampling techniques.
Paper improves VAEs using Monte Carlo methods.
problem Improving the Evidence Lower Bound (ELBO) for VAEs.
method Uses Monte Carlo techniques to improve ELBO, specifically Sequential Importance Sampling (SIS) with carefully chosen kernels.
result Demonstrates improved performance on various applications.
This thesis tackles non-convex Bayesian learning via scalable dynamic importance sampling algorithms.
problem Non-convex Bayesian learning problem in deep neural networks.
method Replica exchange Langevin Monte Carlo, control variates method, population-chain replica exchange, scalable dynamic importance sampling.
result Control variates method reduces variance and accelerates convergence in non-convex Bayesian learning.
Many machine learning problems involve Monte Carlo gradient estimators. As a prominent example, we focus on Monte Carlo variational inference (MCVI) in this paper. The performance of MCVI crucially depends on the variance of its stochastic gradients. We propose variance reduction by means of Quasi-Monte Carlo (QMC) sam…