Adaptive importance sampling is a class of techniques for finding good proposal distributions for importance sampling. Often the proposal distributions are standard probability distributions whose parameters are adapted based on the mismatch between the current proposal and a target distribution. In this work, we prese…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Adaptive Importance Sampling”
New adaptive importance samplers improve stability and accuracy.
problem Improving the stability and accuracy of importance sampling estimators.
method Introducing AdaOAIS, a new adaptive importance sampler using adaptive optimisers to address the instability of OAIS.
result AdaOAIS leads to stable importance sampling estimators in practice.
This paper balances bias and variance in adaptive importance sampling using mirror descent.
problem Large variance in adaptive importance sampling weights.
method Regularization strategy with power raised importance weights connected to mirror descent.
result The regularization parameter balances bias and variance.
Framework improves gradient estimation for faster training convergence.
problem Efficiently estimating noisy gradients in stochastic optimization.
method Dynamic adaptive importance sampling combining multiple distributions.
result Adaptively weighted multiple importance sampling yields superior gradient estimates.
We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation. Borrowing ideas from the bandits literature, we propose Daisee, a partition-based AIS algorithm. We further introduce a notion of regret for AI…
Adaptive importance sampling for estimating point process statistics.
problem Estimating the expected value of a statistic of a locally stable point process.
method Adaptive importance sampling with Poisson point processes and cross-entropy minimization.
result The proposed estimator converges to the target value almost surely and is asymptotically normal.
Gradient Importance Samplingstat.ML
Adaptive Monte Carlo schemes developed over the last years usually seek to ensure ergodicity of the sampling process in line with MCMC tradition. This poses constraints on what is possible in terms of adaptation. In the general case ergodicity can only be guaranteed if adaptation is diminished at a certain rate. Import…
HAIS improves importance sampling in high dimensions using HMC.
problem Improving importance sampling in high-dimensional problems.
method Two-step adaptive process with parallel HMC chains.
result Significant performance improvement in high-dimensional problems.
Entropic Mirror Monte Carlostat.ME
Adaptive sampling method improves efficiency in complex target distributions.
problem Efficiency of importance sampling in complex target distributions, especially multimodal distributions in high-dimensional spaces.
method Proposes an adaptive scheme combining global sampling with delayed weighting to promote efficient exploration of target distributions.
result The proposed algorithm is geometrically convergent under mild assumptions and demonstrates improved efficiency in various numerical experiments.
We propose a novel adaptive importance sampling algorithm which incorporates Stein variational gradient decent algorithm (SVGD) with importance sampling (IS). Our algorithm leverages the nonparametric transforms in SVGD to iteratively decrease the KL divergence between our importance proposal and the target distributio…
This paper introduces Tree-Pyramidal Adaptive Importance Sampling (TP-AIS), a novel iterated sampling method that outperforms state-of-the-art approaches like deterministic mixture population Monte Carlo (DM-PMC), mixture population Monte Carlo (M-PMC) and layered adaptive importance sampling (LAIS). TP-AIS iteratively…
Combines control variates and adaptive importance sampling for Monte Carlo integration.
problem Improving Monte Carlo integration accuracy with control variates and adaptive sampling.
method A quadrature rule combining control variates and adaptive importance sampling.
result Non-asymptotic bound on the probabilistic error of the procedure.
Proposal-Guided Greedy Surrogate Refinement for PDE-Driven High-Dimensional Rare-Event Estimationmath.NA
Proposes a method to refine PDE-driven high-dimensional rare-event simulation.
problem Challenges in constructing accurate surrogates for rare-event simulation.
method Adaptive importance sampling framework that refines a locally constructed surrogate.
result Achieves accuracy comparable to true-model adaptive importance sampling with fewer high-fidelity evaluations.
Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizescs.LG
Avare improves optimization and sampling with adaptive importance sampling.
problem Improving convergence rate of stochastic gradient-based algorithms.
method Adaptive importance sampling with decreasing step-sizes.
result Achieves dynamic regret bounds of and .
New algorithm improves Bayesian neural networks using adaptive importance sampling.
problem High computational cost in training Bayesian neural networks.
method Adaptive Importance Sampling (AIS) integrated into a novel algorithm (PMCnet).
result Improved performance and exploration capabilities for both shallow and deep neural networks.
Proposes a new method for handling domain shift in samples with biases in both covariates and labels.
problem Domain shift in samples with biases in both covariates and labels.
method Factorizable Joint Shift (FJS) and Joint Importance Aligning (JIA).
result Our method can handle co-existence of sampling bias in covariates and labels.
Adaptive Monte Carlo methods are recent variance reduction techniques. In this work, we propose a mathematical setting which greatly relaxes the assumptions needed by for the adaptive importance sampling techniques presented by Vazquez-Abad and Dufresne, Fu and Su, and Arouna. We establish the convergence and asymptoti…
AIS algorithm improves heavy-tailed distribution estimation.
problem Inconsistent estimators and slow convergence in AIS for heavy-tailed distributions.
method Adapts Student-t proposal distributions by matching escort moments and minimizing α-divergence.
result Improves estimation accuracy for heavy-tailed distributions.
BIS uses bandits to efficiently sample from expensive-to-evaluate densities.
problem Sampling from computationally expensive target densities.
method Sequential selection through multi-armed bandits, optimizing sample set directly.
result BIS achieves accurate sampling with fewer evaluations than adaptive methods.
Sampling is an important tool for estimating large, complex sums and integrals over high dimensional spaces. For instance, important sampling has been used as an alternative to exact methods for inference in belief networks. Ideally, we want to have a sampling distribution that provides optimal-variance estimators. In …
BLISS: Bandit Layer Importance Sampling Strategy for Efficient Training of Graph Neural Networkscs.LG
BLISS optimizes GNN training by adaptively sampling nodes.
problem High computational costs in training GNNs on large graphs.
method Uses Bandit Layer Importance Sampling to dynamically select nodes.
result Improves GNN performance with reduced computational cost.
Adaptive importance samplers are adaptive Monte Carlo algorithms to estimate expectations with respect to some target distribution which \textit{adapt} themselves to obtain better estimators over a sequence of iterations. Although it is straightforward to show that they have the same convergen…
The paper proposes AIS for Bayesian inversion of multioutput signals with covariance estimation.
problem Performing uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals.
method Adaptive Importance Sampling (AIS) scheme, split variables, frequentist approach for noise covariance, prior density over covariance matrix.
result Estimation of model parameters and covariance matrix of noise.
Paper tackles non-Markovian control problems with new learning methods.
problem Non-Markovian stochastic control problems with unknown parameters.
method Off-model training and importance sampling for deep neural network approximation.
result Quantitative error bounds for adaptive learning under model uncertainty.
This paper tackles label-efficient evaluation in extreme class imbalance.
problem Challenges in obtaining a sufficient sample for accurate evaluation in tasks with extreme class imbalance.
method Develops a framework for online evaluation based on adaptive importance sampling.
result Establishes strong consistency and a central limit theorem for performance estimates.
Generative Adversarial Networks (GAN) training process, in most cases, apply Uniform or Gaussian sampling methods in the latent space, which probably spends most of the computation on examples that can be properly handled and easy to generate. Theoretically, importance sampling speeds up stochastic optimization in supe…
Improves transferability of representations from source to target domains with weights and invariant representations.
problem Label shift between source and target domains in unsupervised domain adaptation.
method Integrates weights and invariant representations to bound the target risk, highlighting the role of inductive bias.
result Empirical evidence shows that weak inductive bias makes adaptation more robust.
The importance weighted autoencoder (IWAE) (Burda et al., 2016) is a popular variational-inference method which achieves a tighter evidence bound (and hence a lower bias) than standard variational autoencoders by optimising a multi-sample objective, i.e. an objective that is expressible as an integral over Mont…
Time Series Classification (TSC) has been an important and challenging task in data mining, especially on multivariate time series and multi-view time series data sets. Meanwhile, transfer learning has been widely applied in computer vision and natural language processing applications to improve deep neural network's g…
GUESS improves surrogate model accuracy with adaptive sampling.
problem Creating accurate surrogate models with limited data.
method Gradient and Uncertainty Enhanced Sequential Sampling (GUESS) using predictive uncertainty and Taylor expansion.
result GUESS achieved highest sample efficiency compared to other strategies.
Optimized AIS scheme reduces bias and MSE for general proposals.
problem Performing Monte Carlo integration with general proposals.
method Global optimization of χ²-divergence using stochastic gradient Langevin dynamics.
result Explicit theoretical guarantees for uniform-in-time MSE reduction.
Efficiently estimates online variational learning using importance sampling.
problem Online variational estimation in state-space models.
method Variational approach with Monte Carlo importance sampling.
result Proposed efficient algorithm for streaming data.
Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction. In this work, we propose a new framework for variance reduction that enables the use of mixtures over predefined sampling distributions, which can naturally encode prior knowledg…
RADIS uses deep regression to create efficient importance sampling for model inversion and emulation.
problem Efficiently sampling from posterior distributions for model inversion and emulation.
method RADIS uses a deep architecture of nested importance sampling schemes to construct a non-parametric emulator that mimics the posterior distribution.
result RADIS asymptotically converges to an exact sampler under mild conditions and can be used as a surrogate model.
Monte Carlo methods represent the "de facto" standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use simpler proposal probability densities to draw candidate samples. The performan…
Adambs adapts Adam to prioritize important training examples.
problem Training deep learning models efficiently with varying example importance.
method Adapt Adam using a bandit sampling approach to prioritize examples based on their importance.
result Adambs improves convergence rate compared to Adam in some cases.
This survey explores various optimality concepts in importance sampling.
problem Designing optimal proposal densities for Monte Carlo methods.
method Review of multiple frameworks and theoretical comparisons.
result Comprehensive understanding of optimality in importance sampling.
Adaptive importance sampling (AIS) uses past samples to update the \textit{sampling policy} at each stage . Each stage is formed with two steps : (i) to explore the space with points according to and (ii) to exploit the current amount of information to update the sampling policy. The very funda…
Adaptive Nucleus Truncation Improves Long-Form Reasoning
problem Improving long-form reasoning in language models
method Adaptive Nucleus Truncation Sampling (ANTS)
result Significant performance gains across various benchmarks
We develop a new method to estimate failure probabilities in complex systems.
problem Estimating failure probabilities in safety-critical autonomous systems is challenging due to the rarity of failures and large state spaces.
method We propose an adaptive importance sampling algorithm that minimizes forward Kullback-Leibler divergence and uses Markov score ascent methods.
result Our method provides more accurate failure probability estimates than existing techniques.
Efficient momentum-based methods for reinforcement learning with improved sample complexity.
problem Efficient reinforcement learning algorithms for non-concave performance functions.
method Adaptive momentum-based policy gradient methods using variance reduction and importance sampling techniques.
result Both IS-MBPG and HA-MBPG reach the best known sample complexity of for finding an -stationary point.
Adaptive importance sampling techniques are widely known for the Gaussian setting of Brownian driven diffusions. In this work, we want to extend them to jump processes. Our approach relies on a change of the jump intensity combined with the standard exponential tilting for the Brownian motion. The free parameters of ou…
This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
BAMS uses Bayesian sampling to discover AV failures more efficiently and accurately.
problem Discovering potential failure cases in autonomous vehicles efficiently and accurately.
method Bayesian adaptive multifidelity sampling (BAMS) prioritizes exploration of low performance regions.
result BAMS discovers 10 times more issues than traditional methods with narrower rate estimates.
Domain adaptation has become a prominent problem setting in machine learning and related fields. This review asks the question: how can a classifier learn from a source domain and generalize to a target domain? We present a categorization of approaches, divided into, what we refer to as, sample-based, feature-based and…
In modern data analysis, random sampling is an efficient and widely-used strategy to overcome the computational difficulties brought by large sample size. In previous studies, researchers conducted random sampling which is according to the input data but independent on the response variable, however the response variab…
The estimation of normalizing constants is a fundamental step in probabilistic model comparison. Sequential Monte Carlo methods may be used for this task and have the advantage of being inherently parallelizable. However, the standard choice of using a fixed number of particles at each iteration is suboptimal because s…