Improves variational inference for deep latent models using multi-sample importance sampling.
problem Improving variational inference for deep latent variable models.
method Developed a new unbiased gradient estimator for multi-sample importance-sampled objectives.
result The new estimator yields models that use more of their capacity and achieve higher likelihoods.
AISLE framework improves on IWAE by directly optimising proposal distribution.
problem IWAE's multi-sample objective leads to inference-network gradients that break down with increasing samples.
method Introduces AISLE framework, which optimises proposal distribution directly.
result AISLE admits IWAE-STL and IWAE-DREG as special cases, avoiding breakdown.
Improved privacy-preserving methods for estimating multiple samples from distributions.
problem Estimating multiple samples from distributions while maintaining privacy.
method Developed new multi-sampling techniques for differentially private data estimation.
result Achieved significant reduction in sample complexity for multi-sampling from finite domains and Gaussian distributions.
Enhanced dropout technique improves training speed and generalization.
problem Improving generalization and training speed of deep neural networks.
method Multi-sample dropout technique, creating multiple dropout samples and averaging their losses.
result Multi-sample dropout accelerates training and achieves lower error rates.
ARMS improves gradient estimation for binary variables using antithetic samples.
problem Estimating gradients for binary variables in discrete latent variable models.
method ARMS uses antithetic samples generated by a copula to estimate gradients more efficiently and unbiasedly.
result ARMS outperforms competing methods in training generative models and optimizing variational bounds.
New MI bounds improve estimation in deep generative models.
problem Estimating mutual information without density information is intractable.
method Importance sampling, Annealed Importance Sampling, Generalized IWAE, MINE-AIS.
result Improved bounds for estimating mutual information in deep models.
AgrLearn improves deep learning by optimizing multi-sample quantization.
problem Improving deep learning performance with fewer training samples.
method Develops AgrLearn, a vector IB quantization-based framework.
result Significant improvements in image recognition and text classification.
Tensor Monte Carlo improves variational autoencoders for high-dimensional latent spaces.
problem Scalability issues in IWAEs for high-dimensional latent spaces.
method Tensor Monte Carlo (TMC) draws exponentially many samples separately for each latent variable and averages them.
result TMC outperforms IWAE on a generative model with multiple stochastic layers.
New models learn from samplers to approximate EBMs.
problem Intractable sampling and density evaluation in EBMs.
method Maximize likelihood of sampler-induced distribution.
result EIMs provide exact samples and tractable log-likelihood.
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
This paper improves SNN training by using multiple sample compartments.
problem Training SNNs with single-sample estimators leads to inaccurate log-likelihood estimates.
method Proposes a GEM-based online learning algorithm that uses multiple independent spiking signals.
result Significant improvements in log-likelihood, accuracy, and calibration with multiple compartments.
A novel rejection sampling step improves variational inference for latent variable models.
problem High variance in gradient estimates for approximate posterior in stochastic variational inference.
method Rejection sampling to discard low-likelihood samples and a new gradient estimator.
result Improves marginal log-likelihood estimation by 3.71 nats and 0.21 nats.
GumBolt extends Gumbel trick for Boltzmann priors in VAEs.
problem Non-differentiability of discrete units in Boltzmann machines prevents using the reparameterization trick.
method Proposes GumBolt, extending Gumbel trick to Boltzmann priors in VAEs.
result Significantly simpler than recent methods and outperforms them.
Improves Bayesian predictive performance in misspecified models.
problem Misspecification gap between inferential and predictive risks.
method Develops a multi-sample loss (PACm) to bridge the gap. result Empirical study shows improved predictive distribution.
Two new estimators improve VAE training for hierarchical and prior parameters.
problem Efficient gradient estimation for VAEs with hierarchical and prior parameters.
method Developed two generalizations of Doubly-Reparameterized Gradient Estimators (DReGs) for VAEs.
result Improved training of conditional and hierarchical VAEs on image modeling tasks.
DisARM improves gradient estimation for binary latent variables.
problem Challenges in training models with discrete latent variables.
method Uses antithetic sampling over continuous augmentation.
result DisARM consistently outperforms ARM and baseline methods in log-likelihood and variance.
Bayesian neural networks with data augmentation show a persistent cold posterior effect.
problem Understanding the cold posterior effect in Bayesian neural networks with data augmentation.
method Developed principled Bayesian neural networks using data augmentation, providing exact likelihoods and tight bounds.
result The cold posterior effect persists even in models incorporating data augmentation, suggesting it's not an artifact.
New method designs joint initial noises for diffusion models to improve diversity and alignment.
problem Independent initial noises limit diversity in generated images.
method Coupling of initial noises, maintaining Gaussian distribution while allowing dependence.
result Repulsive Gaussian coupling improves diversity without increasing sampling cost.
Bayesian model for cancer drug studies maps dose-response curves.
problem Mapping dose-response curves in cancer drug studies.
method Bayesian Tensor Filtering (BTF) with low-dimensional embeddings and structured shrinkage priors.
result BTF outperforms state-of-the-art methods in cancer drug studies.
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
New techniques model related samples using kernel mixtures, addressing shared and varying components with misalignments.
problem Modeling related samples with shared and varying components, accounting for misalignments.
method Introduces ψ-stick breaking for mixing weights and kernel perturbation for misalignment. result Efficient Bayesian inference for models incorporating these techniques.
Improved gradient estimator boosts performance in latent variable models.
problem Poor gradient estimation in multi-sample variational bounds.
method Doubly reparameterized gradient (DReG) estimator.
result DReG estimator reduces variance and improves model performance.
ECLIPSE detects AI hallucinations in finance with high accuracy.
problem Hallucinations in AI-generated answers limit safe deployment in finance.
method Combines entropy estimation and perplexity decomposition to measure model evidence use.
result ECLIPSE achieves ROC AUC of 0.89 and average precision of 0.90 on financial QA dataset.
Survey on statistical inference methods for random dot product graphs.
problem Statistical inference on random dot product graphs.
method Spectral embeddings of adjacency and Laplacian matrices.
result Consistency and asymptotic normality of spectral embeddings.
This paper introduces a new learning rule for probabilistic SNNs that improves log-likelihood, accuracy, and calibration.
problem Training and inference of deterministic SNNs are constrained by their inability to generate multiple independent outputs.
method Introduces a generalized expectation-maximization (GEM) learning rule for probabilistic SNNs.
result The GEM-SNN learning rule leads to significant improvements in log-likelihood, accuracy, and calibration.
Paper proposes inference method for high-dimensional censored quantile regression.
problem Identifying heterogeneous effects of high-dimensional genetic biomarkers on survival outcomes.
method Combines low-dimensional model estimates based on multi-sample splittings and variable selection.
result Proposed estimator is consistent and asymptotically follows a Gaussian process.
UCPO improves diversity in reinforcement learning models, maintaining high accuracy.
problem RLVR objectives often lead to diversity collapse, reducing coverage of correct solutions.
method UCPO adds a conditional uniformity penalty to GRPO, redistributing probability mass.
result UCPO improves Pass@K and diversity while maintaining competitive Pass@1 accuracy.
New method improves importance sampling for complex proposals.
problem Limited power of traditional importance sampling.
method Black-box importance sampling for any unknown proposal.
result Better and richer proposals improve estimation accuracy.
Paper proposes a new importance sampling method for reducing variance.
problem Reducing variance in importance sampling when training and testing data come from different distributions.
method A new variant of importance sampling that reduces variance by orders of magnitude.
result The new estimator can improve estimates of treatment effectiveness using limited data.
Paper proposes a method to estimate variance reduction in DNN training using importance sampling.
problem Challenges in assessing variance reduction during DNN training using importance sampling.
method Proposes a method for estimating variance reduction using minibatches sampled under importance sampling.
result Demonstrates consistent reduction in variance, improved training efficiency, and enhanced model accuracy.
FIS-GAN uses importance sampling in GANs to speed up training.
problem Efficiency in GAN training by focusing on hard-to-generate examples.
method Adapting importance sampling into GANs using normalizing flows.
result Significant acceleration in GAN optimization with improved fidelity.
New method improves Bayesian cross-validation.
problem Finding good proposal distributions for importance sampling.
method Implicitly adaptive importance sampling that iteratively matches moments.
result Better than many existing parametric adaptive importance sampling methods.
The paper analyzes and improves privacy in machine learning through importance sampling.
problem Ensuring privacy in machine learning while maintaining utility and efficiency.
method Individualized privacy analysis of importance sampling, proposing two approaches for constructing sampling distributions.
result Proposed approaches optimize privacy-efficiency trade-off and outperform uniform sampling.
The importance-weighted risk estimator can be skewed, leading to suboptimal regularization parameters.
problem Skewed sampling distribution of the importance-weighted risk estimator affects model selection.
method Empirical study of the sampling distribution of the importance-weighted risk estimator.
result The importance-weighted risk estimator produces overestimates for the majority of cases and underestimates for tail cases, leading to suboptimal regularization parameters.
The paper explains how importance sampling can be used for optimization of rare events.
problem Minimizing tail risks in stochastic optimization formulations.
method Importance sampling for reducing sample requirements in estimating rare events.
result Effective importance sampling techniques for optimization of rare events.
Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA). Although uniform sampling can guarantee that the sampled stochastic quantity is an unbiased estim…
Paper introduces a new sampling method combining Consistency Models with importance sampling.
problem Inherent errors in samples and high NFEs for high-quality samples in Boltzmann distributions.
method Combines Consistency Models with importance sampling to produce unbiased samples with minimal NFEs.
result Produces unbiased samples using only 6-25 NFEs, comparable to 100 NFEs for DDPMs.
Proposes a new method combining importance sampling with minibatching.
problem Training efficiency and variance reduction in supervised learning.
method Importance sampling for minibatches.
result Significant improvement in training time for certain data properties.
Low-rank MPPCA improves importance sampling in high dimensions.
problem Estimating full-rank GMM covariance matrices in high dimensions is numerically unstable.
method Use MPPCA mixtures as low-rank proposals for importance sampling in high-dimensional spaces.
result Consistent gains in sample efficiency and quality of failure distribution characterization.
Framework improves gradient estimation for faster training convergence.
problem Efficiently estimating noisy gradients in stochastic optimization.
method Dynamic adaptive importance sampling combining multiple distributions.
result Adaptively weighted multiple importance sampling yields superior gradient estimates.
TP-AIS improves sampling efficiency over existing methods.
problem Efficient sampling from complex probability distributions.
method Iterative sampling using a tree pyramid structure.
result TP-AIS outperforms DM-PMC, M-PMC, and LAIS.
New IS methods fail to reduce variance in long-horizon MDPs.
problem High variance in off-policy evaluation for long-horizon domains.
method Conditional Monte Carlo analysis of IS methods.
result No strict variance reduction for per-decision or stationary IS methods in finite horizon MDPs.
Enhances statistical mechanics solving using VANs with MCMC or importance sampling.
problem Sampling error in solving statistical mechanics using VANs.
method Integrates MCMC or importance sampling to correct sampling error in VANs.
result Asymptotically unbiased estimators for physical quantities are achieved.
Stein Variational Adaptive Importance Sampling improves IS with SVGD, reducing KL divergence.
problem Improving the efficiency and interpretability of importance sampling.
method Combines Stein variational gradient descent with importance sampling.
result Significantly reduces KL divergence between proposal and target distributions.
Paper improves off-policy evaluation by estimating behavior policy.
problem Evaluating policies with data from a different behavior policy.
method Importance sampling with an estimated behavior policy.
result Estimating behavior policy reduces mean squared error.
BR-SNIS reduces bias in self-normalized IS without increasing variance.
problem Bias in self-normalized IS.
method Iterated sampling-importance resampling (ISIR) to form a bias-reduced estimator.
result Significant reduction in bias without increasing variance.
We develop a new method to estimate failure probabilities in complex systems.
problem Estimating failure probabilities in safety-critical autonomous systems is challenging due to the rarity of failures and large state spaces.
method We propose an adaptive importance sampling algorithm that minimizes forward Kullback-Leibler divergence and uses Markov score ascent methods.
result Our method provides more accurate failure probability estimates than existing techniques.
We present a new method for conducting Monte Carlo inference in graphical models which combines explicit search with generalized importance sampling. The idea is to reduce the variance of importance sampling by searching for significant points in the target distribution. We prove that it is possible to introduce search…