Framework improves gradient estimation for faster training convergence.
problem Efficiently estimating noisy gradients in stochastic optimization.
method Dynamic adaptive importance sampling combining multiple distributions.
result Adaptively weighted multiple importance sampling yields superior gradient estimates.
Paper finds efficient OPE estimator for multiple logging policies with minimum variance.
problem Finding optimal importance sampling weights for multiple logging policies with varying variances.
method Established efficiency bound under stratified sampling and proposed an estimator achieving this bound.
result Proposed estimator achieves minimum variance for any instance.
Study reduces NAS search cost by generating multiple complex architectures in one shot.
problem Finding multiple neural architectures with varying complexities efficiently.
method Uses importance sampling to generate and update multiple distributions of architectures.
result Reduces search cost by finding multiple architectures with different complexities in a single search.
New MCMC method corrects bias without extra cost.
problem Correcting bias in MCMC algorithms without additional computational cost.
method Generalized Markov Chain Importance Sampling methods.
result Proposed methods are more efficient than Metropolis-Hastings versions.
This survey explores various optimality concepts in importance sampling.
problem Designing optimal proposal densities for Monte Carlo methods.
method Review of multiple frameworks and theoretical comparisons.
result Comprehensive understanding of optimality in importance sampling.
Combines control variates and adaptive importance sampling for Monte Carlo integration.
problem Improving Monte Carlo integration accuracy with control variates and adaptive sampling.
method A quadrature rule combining control variates and adaptive importance sampling.
result Non-asymptotic bound on the probabilistic error of the procedure.
We provide a comparative study of several widely used off-policy estimators (Empirical Average, Basic Importance Sampling and Normalized Importance Sampling), detailing the different regimes where they are individually suboptimal. We then exhibit properties optimal estimators should possess. In the case where examples …
Importance weighted variational inference (Burda et al., 2015) uses multiple i.i.d. samples to have a tighter variational lower bound. We believe a joint proposal has the potential of reducing the number of redundant samples, and introduce a hierarchical structure to induce correlation. The hope is that the proposals w…
A new method combines AIS and SMCI for efficient evaluation of Ising models.
problem Efficiently evaluating expectations on Ising models under various temperatures.
method Combining Annealed Importance Sampling (AIS) and Spatial Monte Carlo Integration (SMCI).
result The proposed method performs efficiently in both high- and low-temperature regions.
Variational inference (VI) and Markov chain Monte Carlo (MCMC) are two main approximate approaches for learning deep generative models by maximizing marginal likelihood. In this paper, we propose using annealed importance sampling for learning deep generative models. Our proposed approach bridges VI with MCMC. It gener…
Enhances anomaly detection using multiple reference datasets.
problem Lack of performance with multiple reference datasets.
method Generalizes existing techniques CWoLa and SALAD for multiple reference datasets.
result Improved performance across various settings.
Unified framework for estimating density ratios across multiple distributions.
problem Binary density ratio estimation for multiple distributions.
method Unified framework based on Bregman divergence minimization.
result Generalization of binary DRE methods to multiple distributions.
Unified framework connects NCE, MIS, RLR, and bridge sampling for EBMs.
problem Challenges in parameter estimation for intractable likelihood EBMs.
method Unified framework connecting NCE, RLR, MIS, and bridge sampling.
result Unified perspective clarifies relationships among existing methods.
Improves decision-making in models fit with AEVB by using distinct approximate posteriors.
problem Bias in expected risk estimates due to variational distribution use.
method Use multiple approximate posteriors, including those distinct from variational, for decision-making.
result Proposed approach outperforms state-of-the-art methods in single-cell RNA sequencing.
This paper improves SNN training by using multiple sample compartments.
problem Training SNNs with single-sample estimators leads to inaccurate log-likelihood estimates.
method Proposes a GEM-based online learning algorithm that uses multiple independent spiking signals.
result Significant improvements in log-likelihood, accuracy, and calibration with multiple compartments.
Improved SVMs learn from few samples with composition and multiple scales.
problem Learning with small sample sizes.
method Transformation-invariant SVMs with composition and locality at multiple scales.
result Kernels based on maximum similarity are positive definite and yield superior accuracy.
This study improves uncertainty quantification in seismic inversion.
problem Uncertainty in seismic inversion due to limited data and model diversity.
method Integrates ensemble methods with importance sampling.
result More accurate uncertainty quantification in velocity models.
Given samples from a distribution, how many new elements should we expect to find if we continue sampling this distribution? This is an important and actively studied problem, with many applications ranging from unseen species estimation to genomics. We generalize this extrapolation and related unseen estimation proble…
Develops a new inference method for split-sample estimators using multiple splits.
problem Statistical dependence and variability in split-sample estimators.
method Averaging across multiple splits, proving a central limit theorem, and developing new inference approaches.
result Valid confidence intervals and improved power in comparing model performance.
RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.
problem Policy Optimization in continuous control tasks.
method RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for regret minimization in PO.
result Achieving constant regret under certain circumstances in PO with mediator feedback.
Multi-sample, importance-weighted variational autoencoders (IWAE) give tighter bounds and more accurate uncertainty estimates than variational autoencoders (VAE) trained with a standard single-sample objective. However, IWAEs scale poorly: as the latent dimensionality grows, they require exponentially many samples to r…
A method to improve clustering explainability using bagging and feature dropout.
problem Lack of explainability in clustering methods.
method Bagging and feature dropout to generate feature importance scores.
result Improved stability and robustness of cluster definition, especially in small-sample or noisy settings.
VISA improves inference efficiency for complex models.
problem Efficient approximate inference in computationally intensive models.
method Sequential sample-average approximations within a trust region.
result VISA achieves comparable accuracy with computational savings.
New bounds for estimating partition functions under bounded f-divergence.
problem Estimating partition functions with limited sample access.
method Information-theoretic characterization using integrated coverage profile and f-divergences. result Sharp phase transitions in sample complexity under f-divergences. CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
New model improves prediction accuracy and interpretability for network-connected data.
problem Reduced effective sample size and poor prediction performance in network-connected data.
method Network-assisted Random Forest+ (RF+) with interpretable feature importance measures.
result Highly competitive prediction accuracy and interpretable feature importance.
CRAFT improves on existing methods for sampling complex distributions.
problem Sampling from complex probability distributions.
method Combines SMC with variational inference using normalizing flows.
result Improves on Annealed Flow Transport Monte Carlo and MCMC-based Stochastic Normalizing Flows.
MultiImport infers node importance from multiple KG signals.
problem Inferring node importance in a knowledge graph from multiple input signals.
method End-to-end latent variable model using attentive graph neural networks.
result MultiImport consistently outperforms existing methods, achieving up to 23.7% higher NDCG@100.
Monte Carlo methods represent the "de facto" standard for approximating complicated integrals involving multidimensional target distributions. In order to generate random realizations from the target distribution, Monte Carlo techniques use simpler proposal probability densities to draw candidate samples. The performan…
Bayesian methods and their implementations by means of sophisticated Monte Carlo techniques have become very popular in signal processing over the last years. Importance Sampling (IS) is a well-known Monte Carlo technique that approximates integrals involving a posterior distribution by means of weighted samples. In th…
Low bit-width integer weights and activations are very important for efficient inference, especially with respect to lower power consumption. We propose Monte Carlo methods to quantize the weights and activations of pre-trained neural networks without any re-training. By performing importance sampling we obtain quantiz…
The paper predicts and explains the decay of stock anomaly performance over time.
problem Predicting and explaining the drop in risk-adjusted performance of stock anomalies.
method The authors propose ex-ante characteristics based on hypotheses of out-of-sample decay and in-sample overfitting.
result The year of publication explains 30% of the variance in Sharpe decay across factors.
We introduce a novel approach for estimating Latent Dirichlet Allocation (LDA) parameters from collapsed Gibbs samples (CGS), by leveraging the full conditional distributions over the latent variable assignments to efficiently average over multiple samples, for little more computational cost than drawing a single addit…
A new framework for controllable generation of discrete masked models.
problem Efficient controllable generation of discrete data models.
method Plug-and-play framework based on importance sampling.
result Demonstrates versatility across multiple domains, including protein design.
MT-HAL learns features and task associations for multiple tasks with a shared sparse structure.
problem Learning features and task associations for multiple tasks with shared structure.
method Fully nonparametric approach that learns features, samples, and task associations with a shared sparse structure.
result MT-HAL achieves a powerful convergence rate and outperforms other methods across various simulation settings.
Gaussian graphical model is a graphical representation of the dependence structure for a Gaussian random vector. It is recognized as a powerful tool in different applied fields such as bioinformatics, error-control codes, speech language, information retrieval and others. Gaussian graphical model selection is a statist…
G-SHAP generates multiple types of explanations for machine learning models.
problem Understanding model predictions and their differences across groups.
method Generalization of SHAP method to produce additional types of explanations.
result G-SHAP produces explanations for classification, intergroup differences, and model failure.
We propose a general statistical framework for clustering multiple time series that exhibit nonlinear dynamics into an a-priori-unknown number of sub-groups. Our motivation comes from neuroscience, where an important problem is to identify, within a large assembly of neurons, subsets that respond similarly to a stimulu…
NEO combines orbits to sample and estimate complex distributions.
problem Sampling and normalizing complex distributions.
method NEO-IS and NEO-MCMC using invertible maps and orbits.
result NEO provides unbiased estimators and explores multimodal targets.
The impressive performance of neural networks on natural language processing tasks attributes to their ability to model complicated word and phrase compositions. To explain how the model handles semantic compositions, we study hierarchical explanation of neural network predictions. We identify non-additivity and contex…
DAGgr aggregates multiple DAGs to stabilize causal structure learning.
problem Stability in learning causal structure from data.
method Model averaging of candidate DAGs weighted by predictive likelihood, with acyclicity enforced.
result DAGgr consistently outperforms individual DAGs and bootstrap-aggregation baselines.
NERS improves RL by sampling diverse transitions considering local and global contexts.
problem Sampling biases in experience replay lead to redundant transitions.
method Neural Experience Replay Sampler (NERS) that considers both local and global contexts.
result NERS significantly improves RL performance by sampling diverse and meaningful transitions.
Mixture components improve VAE performance by increasing latent flexibility.
problem Improving variational autoencoder (VAE) performance through more flexible latent representations.
method Modeling mixture components with separate encoder networks and analyzing their impact on ELBO.
result Increasing the number of mixture components improves VAE performance on various datasets.
Improved preterm prediction using synthetic EHG signals.
problem Prediction bias towards term labor in preterm EHG data.
method Quantifying synthetic samples' effect, optimizing feature weights, and combining activation functions.
result Substantial improvement in prediction precision.
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
New method improves self-supervised representation learning using probabilistic modeling and Monte Carlo integration.
problem Improving self-supervised representation learning for multimodal data.
method Discriminative probabilistic modeling with multiple importance sampling (MIS) for robust Monte Carlo integration.
result Proposes a novel non-parametric method for approximating conditional probability densities through convex optimization.
New method detects and analyzes correlation in multiple network data.
problem Detecting and analyzing correlation in multiple network data.
method Generalized omnibus embedding methodology.
result Induced correlation can significantly extend the reach of spectral inference procedures.
Importance sampling is often used in machine learning when training and testing data come from different distributions. In this paper we propose a new variant of importance sampling that can reduce the variance of importance sampling-based estimates by orders of magnitude when the supports of the training and testing d…