Adaptive importance sampling is a class of techniques for finding good proposal distributions for importance sampling. Often the proposal distributions are standard probability distributions whose parameters are adapted based on the mismatch between the current proposal and a target distribution. In this work, we prese…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “adaptive importance-sampling”
This paper balances bias and variance in adaptive importance sampling using mirror descent.
problem Large variance in adaptive importance sampling weights.
method Regularization strategy with power raised importance weights connected to mirror descent.
result The regularization parameter balances bias and variance.
HAIS improves importance sampling in high dimensions using HMC.
problem Improving importance sampling in high-dimensional problems.
method Two-step adaptive process with parallel HMC chains.
result Significant performance improvement in high-dimensional problems.
Combines control variates and adaptive importance sampling for Monte Carlo integration.
problem Improving Monte Carlo integration accuracy with control variates and adaptive sampling.
method A quadrature rule combining control variates and adaptive importance sampling.
result Non-asymptotic bound on the probabilistic error of the procedure.
TP-AIS improves sampling efficiency over existing methods.
problem Efficient sampling from complex probability distributions.
method Iterative sampling using a tree pyramid structure.
result TP-AIS outperforms DM-PMC, M-PMC, and LAIS.
New adaptive importance samplers improve stability and accuracy.
problem Improving the stability and accuracy of importance sampling estimators.
method Introducing AdaOAIS, a new adaptive importance sampler using adaptive optimisers to address the instability of OAIS.
result AdaOAIS leads to stable importance sampling estimators in practice.
Framework improves gradient estimation for faster training convergence.
problem Efficiently estimating noisy gradients in stochastic optimization.
method Dynamic adaptive importance sampling combining multiple distributions.
result Adaptively weighted multiple importance sampling yields superior gradient estimates.
We propose a novel adaptive importance sampling algorithm which incorporates Stein variational gradient decent algorithm (SVGD) with importance sampling (IS). Our algorithm leverages the nonparametric transforms in SVGD to iteratively decrease the KL divergence between our importance proposal and the target distributio…
Gradient Importance Samplingstat.ML
Adaptive Monte Carlo schemes developed over the last years usually seek to ensure ergodicity of the sampling process in line with MCMC tradition. This poses constraints on what is possible in terms of adaptation. In the general case ergodicity can only be guaranteed if adaptation is diminished at a certain rate. Import…
Adaptive importance sampling for estimating point process statistics.
problem Estimating the expected value of a statistic of a locally stable point process.
method Adaptive importance sampling with Poisson point processes and cross-entropy minimization.
result The proposed estimator converges to the target value almost surely and is asymptotically normal.
New algorithm improves Bayesian neural networks using adaptive importance sampling.
problem High computational cost in training Bayesian neural networks.
method Adaptive Importance Sampling (AIS) integrated into a novel algorithm (PMCnet).
result Improved performance and exploration capabilities for both shallow and deep neural networks.
Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizescs.LG
Avare improves optimization and sampling with adaptive importance sampling.
problem Improving convergence rate of stochastic gradient-based algorithms.
method Adaptive importance sampling with decreasing step-sizes.
result Achieves dynamic regret bounds of and .
Entropic Mirror Monte Carlostat.ME
Adaptive sampling method improves efficiency in complex target distributions.
problem Efficiency of importance sampling in complex target distributions, especially multimodal distributions in high-dimensional spaces.
method Proposes an adaptive scheme combining global sampling with delayed weighting to promote efficient exploration of target distributions.
result The proposed algorithm is geometrically convergent under mild assumptions and demonstrates improved efficiency in various numerical experiments.
Adaptive Monte Carlo methods are recent variance reduction techniques. In this work, we propose a mathematical setting which greatly relaxes the assumptions needed by for the adaptive importance sampling techniques presented by Vazquez-Abad and Dufresne, Fu and Su, and Arouna. We establish the convergence and asymptoti…
Proposal-Guided Greedy Surrogate Refinement for PDE-Driven High-Dimensional Rare-Event Estimationmath.NA
Proposes a method to refine PDE-driven high-dimensional rare-event simulation.
problem Challenges in constructing accurate surrogates for rare-event simulation.
method Adaptive importance sampling framework that refines a locally constructed surrogate.
result Achieves accuracy comparable to true-model adaptive importance sampling with fewer high-fidelity evaluations.
We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation. Borrowing ideas from the bandits literature, we propose Daisee, a partition-based AIS algorithm. We further introduce a notion of regret for AI…
Efficiently estimates online variational learning using importance sampling.
problem Online variational estimation in state-space models.
method Variational approach with Monte Carlo importance sampling.
result Proposed efficient algorithm for streaming data.
AIS algorithm improves heavy-tailed distribution estimation.
problem Inconsistent estimators and slow convergence in AIS for heavy-tailed distributions.
method Adapts Student-t proposal distributions by matching escort moments and minimizing α-divergence.
result Improves estimation accuracy for heavy-tailed distributions.
The paper proposes AIS for Bayesian inversion of multioutput signals with covariance estimation.
problem Performing uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals.
method Adaptive Importance Sampling (AIS) scheme, split variables, frequentist approach for noise covariance, prior density over covariance matrix.
result Estimation of model parameters and covariance matrix of noise.
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
Generative Adversarial Networks (GAN) training process, in most cases, apply Uniform or Gaussian sampling methods in the latent space, which probably spends most of the computation on examples that can be properly handled and easy to generate. Theoretically, importance sampling speeds up stochastic optimization in supe…
We develop a new method to estimate failure probabilities in complex systems.
problem Estimating failure probabilities in safety-critical autonomous systems is challenging due to the rarity of failures and large state spaces.
method We propose an adaptive importance sampling algorithm that minimizes forward Kullback-Leibler divergence and uses Markov score ascent methods.
result Our method provides more accurate failure probability estimates than existing techniques.
BIS uses bandits to efficiently sample from expensive-to-evaluate densities.
problem Sampling from computationally expensive target densities.
method Sequential selection through multi-armed bandits, optimizing sample set directly.
result BIS achieves accurate sampling with fewer evaluations than adaptive methods.
This survey explores various optimality concepts in importance sampling.
problem Designing optimal proposal densities for Monte Carlo methods.
method Review of multiple frameworks and theoretical comparisons.
result Comprehensive understanding of optimality in importance sampling.
RADIS uses deep regression to create efficient importance sampling for model inversion and emulation.
problem Efficiently sampling from posterior distributions for model inversion and emulation.
method RADIS uses a deep architecture of nested importance sampling schemes to construct a non-parametric emulator that mimics the posterior distribution.
result RADIS asymptotically converges to an exact sampler under mild conditions and can be used as a surrogate model.
Improves transferability of representations from source to target domains with weights and invariant representations.
problem Label shift between source and target domains in unsupervised domain adaptation.
method Integrates weights and invariant representations to bound the target risk, highlighting the role of inductive bias.
result Empirical evidence shows that weak inductive bias makes adaptation more robust.
This paper tackles label-efficient evaluation in extreme class imbalance.
problem Challenges in obtaining a sufficient sample for accurate evaluation in tasks with extreme class imbalance.
method Develops a framework for online evaluation based on adaptive importance sampling.
result Establishes strong consistency and a central limit theorem for performance estimates.
Adaptive importance sampling techniques are widely known for the Gaussian setting of Brownian driven diffusions. In this work, we want to extend them to jump processes. Our approach relies on a change of the jump intensity combined with the standard exponential tilting for the Brownian motion. The free parameters of ou…
The estimation of normalizing constants is a fundamental step in probabilistic model comparison. Sequential Monte Carlo methods may be used for this task and have the advantage of being inherently parallelizable. However, the standard choice of using a fixed number of particles at each iteration is suboptimal because s…
New algorithm optimizes stochastic optimization with circular dependency.
problem Circular dependency between decision variable and importance sampling.
method Single-loop stochastic approximation algorithm based on Nesterov's dual averaging.
result Achieves minimal asymptotic variance and resolves circular optimization challenge.
Entropy-based GP adaptive design improves failure probability estimation.
problem Limited accuracy in failure probability estimation due to model evaluation costs.
method Entropy-based Gaussian process (GP) adaptive design combined with multifidelity importance sampling (MFIS).
result More accurate failure probability estimates and higher confidence.
BLISS: Bandit Layer Importance Sampling Strategy for Efficient Training of Graph Neural Networkscs.LG
BLISS optimizes GNN training by adaptively sampling nodes.
problem High computational costs in training GNNs on large graphs.
method Uses Bandit Layer Importance Sampling to dynamically select nodes.
result Improves GNN performance with reduced computational cost.
Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learningstat.ML
New guarantees for ERM with adaptively collected data.
problem Failure of ERM guarantees with adaptively collected data.
method Importance sampling weighted ERM algorithm with maximal inequality.
result First generalization guarantees and fast convergence rates for adaptively collected data.
Efficient momentum-based methods for reinforcement learning with improved sample complexity.
problem Efficient reinforcement learning algorithms for non-concave performance functions.
method Adaptive momentum-based policy gradient methods using variance reduction and importance sampling techniques.
result Both IS-MBPG and HA-MBPG reach the best known sample complexity of for finding an -stationary point.
Optimized AIS scheme reduces bias and MSE for general proposals.
problem Performing Monte Carlo integration with general proposals.
method Global optimization of χ²-divergence using stochastic gradient Langevin dynamics.
result Explicit theoretical guarantees for uniform-in-time MSE reduction.
CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
Sampling is an important tool for estimating large, complex sums and integrals over high dimensional spaces. For instance, important sampling has been used as an alternative to exact methods for inference in belief networks. Ideally, we want to have a sampling distribution that provides optimal-variance estimators. In …
This paper introduces AdaSDCA: an adaptive variant of stochastic dual coordinate ascent (SDCA) for solving the regularized empirical risk minimization problems. Our modification consists in allowing the method adaptively change the probability distribution over the dual variables throughout the iterative process. AdaSD…
The importance weighted autoencoder (IWAE) (Burda et al., 2016) is a popular variational-inference method which achieves a tighter evidence bound (and hence a lower bias) than standard variational autoencoders by optimising a multi-sample objective, i.e. an objective that is expressible as an integral over Mont…
Adaptive importance sampling (AIS) uses past samples to update the \textit{sampling policy} at each stage . Each stage is formed with two steps : (i) to explore the space with points according to and (ii) to exploit the current amount of information to update the sampling policy. The very funda…
Active learning method optimizes seismic fragility curve estimation.
problem Optimizing calls to complex numerical models for fragility curve estimation.
method Importance sampling based active learning for parametric seismic fragility curve estimation.
result The method optimizes the estimation of fragility curves with mathematical rigor.
Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimationstat.ML
We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a sequential factorization of the marginal likelihood in such settings by using stochastic gradient Markov Chain Monte Carlo techniques. This appro…
Monte Carlo methods represent the "de facto" standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use simpler proposal probability densities to draw candidate samples. The performan…
Computing partition functions, the normalizing constants of probability distributions, is often hard. Variants of importance sampling give unbiased estimates of a normalizer Z, however, unbiased estimates of the reciprocal 1/Z are harder to obtain. Unbiased estimates of 1/Z allow Markov chain Monte Carlo sampling of "d…
AIS corrects rollout-training mismatch in quantized RL, improving speed and stability.
problem Rollout-training mismatch in quantized RL causes bias and training collapse.
method Adaptive Importance Sampling (AIS) adjusts gradient correction per batch.
result AIS matches BF16 baseline on most tasks while improving speed.
Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction. In this work, we propose a new framework for variance reduction that enables the use of mixtures over predefined sampling distributions, which can naturally encode prior knowledg…
Importance sampling is often used in machine learning when training and testing data come from different distributions. In this paper we propose a new variant of importance sampling that can reduce the variance of importance sampling-based estimates by orders of magnitude when the supports of the training and testing d…
Review of MLMC in financial engineering, focusing on option pricing and risk management.
problem Efficient estimation of financial risks and option prices using Monte Carlo methods.
method Incorporation of importance sampling and adaptive sampling algorithms in MLMC framework.
result Hybrid algorithms reduce overall variance in estimating financial risks and option prices.