Method uses normalizing flows to efficiently sample from complex target densities.
problem Sampling from complex target densities with zero values in regions of transformation.
method Normalizing flows to address exploding reverse Kullback-Leibler divergence.
result Demonstrated efficient sampling from multi-mode complex density function.
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
Detects adversarial samples using density ratio estimation.
problem Adversarial samples lead to incorrect classifications in machine learning models.
method Direct density ratio estimation for model agnostic detection of adversarial samples.
result Effective detection of adversarial samples with various methods and constraints.
A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…
Generative models learn smoother densities to sample from unknown distributions.
problem Sampling from unknown distributions in high-dimensional spaces.
method Formalizes sampling problem, introduces multimeasurement noise model, derives Bayes estimator, and uses underdamped Langevin MCMC.
result Formulation leads to efficient sampling methods and theoretical connections with denoising autoencoders.
Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
A new method samples from a target density without initial samples using Monte Carlo estimation of the score.
problem Sampling from a target density without initial samples.
method Monte Carlo estimation of the score using oracle access to the log likelihood.
result Samples can be produced from the target density without needing initial samples.
New model for density estimation using tensor trains.
problem Estimation of high-dimensional probability density functions.
method Tensor train-based density estimation (TTDE) with Riemannian optimization.
result TTDE outperforms competitors in training speed and performance.
The paper analyzes how the one-dimensional Wasserstein distance captures pointwise density differences in finite samples.
problem Uncertainty in identifying density differences when supports overlap and densities have substantial pointwise differences.
method Analysis using the Poisson process and neural spike train decoding.
result The one-dimensional Wasserstein distance highlights meaningful density differences related to both rate and support.
Density sketches summarize data distributions for accurate sampling and estimation.
problem Accurately estimating and sampling from complex data distributions.
method Online algorithm generating additive density sketches.
result Density sketches provide statistically sound estimators and sampling capabilities.
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of m−1/2. Estimates Gaussian mixtures from weighted samples efficiently.
problem Estimating Gaussian mixtures from weighted samples with correct weight treatment.
method Density interpretation and expectation-maximization method considering weights.
result Correctly estimates Gaussian mixtures with weighted samples.
The Riemannian Langevin Algorithm samples from manifolds efficiently.
problem Sampling from distributions on manifolds with log-Sobolev inequality.
method Riemannian Langevin Algorithm, log-Sobolev inequality, self-concordance extension, stochastic smoothness bounding.
result The Riemannian Langevin Algorithm converges rapidly to the target density.
Cubic-Spline Flows improve autoregressive flow performance in density estimation.
problem Improving the performance of flow-based models in density estimation.
method Stacking a new coupling transform based on monotonic cubic splines with LU-decomposed linear layers.
result Cubic-Spline Flows close the gap with autoregressive flows on density-estimation tasks.
Graph Laplace operators uniquely identify metrics and densities on manifolds.
problem Identifying Riemannian metrics and sampling densities from graph Laplace operators.
method Analyzing intrinsic and extrinsic graph Laplace operators on compact Riemannian manifolds.
result Graph Laplace operators uniquely determine metrics and densities under certain conditions.
JEPAs learn data density by predicting perturbed samples, enabling density estimation.
problem Representation collapse in latent spaces.
method Combines latent-space prediction and anti-collapse terms to estimate data density.
result JEPAs can estimate sample probabilities efficiently and in closed-form.
Kernel Density Machines learn probability densities without structural assumptions.
problem Learning probability densities under minimal assumptions.
method Kernel-based framework, agnostic to structural requirements.
result Consistency and functional central limit theorem for sample estimator.
New method learns models from data density and generates samples.
problem Learning models that estimate data density and generate samples.
method Denoising density estimators (DDEs) trained to minimize KL-divergence.
result Our method converges to correct solution without specific network architecture.
A new method for estimating density ratios using geodesics on statistical manifolds.
problem Stability of density ratio estimation when distributions are distant.
method Iterative sampling along generalized geodesics on the Riemannian manifold.
result The proposed method outperforms existing incremental mixture methods.
A new model for estimating multivariate densities efficiently.
problem Estimating complex multivariate densities efficiently.
method CDO model based on kernel mean embeddings and RKHS.
result Competitive performance with neural models and Gaussian processes.
New sampling methods for log-concave densities using implicit integrators.
problem Sampling from log-concave densities efficiently.
method θ-method discretization of the overdamped Langevin diffusion.
result Geometric ergodicity and stability for θ≥1/2. This survey explores various optimality concepts in importance sampling.
problem Designing optimal proposal densities for Monte Carlo methods.
method Review of multiple frameworks and theoretical comparisons.
result Comprehensive understanding of optimality in importance sampling.
Optimizes noisy IS with better proposal densities.
problem Improving IS estimators with noisy data.
method Derives optimal proposal densities considering noise variance.
result Optimal proposals enhance IS estimators by focusing on noisy regions.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
This paper improves machine learning density modeling with missing data.
problem Missing data in multidimensional data samples.
method Hierarchical correlation reconstruction using orthonormal functions and L2 optimization.
result Improved imputation and prediction of missing coordinates.
BIS uses bandits to efficiently sample from expensive-to-evaluate densities.
problem Sampling from computationally expensive target densities.
method Sequential selection through multi-armed bandits, optimizing sample set directly.
result BIS achieves accurate sampling with fewer evaluations than adaptive methods.
New method improves sampling from high-dimensional target densities.
problem Sampling from high-dimensional target densities using Monte Carlo algorithms.
method Extends Metropolis-Adjusted Langevin Diffusion algorithm with random precondition matrix modeling.
result Significantly improves performance and computational efficiency over standard MCMC methods.
Bayesian inference engines improve density estimation accuracy and scalability.
problem Constructing accurate and scalable probability density functions.
method Bayesian inference engines (no-U-turn sampling and expectation propagation) with binning strategy.
result Density estimates have excellent comparative performance and scale well to large sample sizes.
Two neural samplers improve high-quality sample generation from un-normalized densities.
problem Generating high-quality samples from un-normalized probability densities.
method Developed two neural samplers using deep neural networks to transform a reference distribution to a target distribution. Training schemes minimize Stein discrepancy variations.
result The proposed samplers generate samples instantaneously and perform better than traditional methods.
A new online method estimates mode from samples using ODE analysis.
problem Estimating mode from samples when analytical density is unknown.
method Online iterative algorithm using ODE analysis for asymptotic convergence and stability.
result Asymptotic convergence and stability of estimates demonstrated.
The Riemann-Theta Boltzmann machine's visible sector is sampled using a discrete multi-variate Gaussian.
problem Sampling the visible sector of the Riemann-Theta Boltzmann machine.
method Discrete multi-variate Gaussian over the hidden state space.
result The visible sector probability density function is an infinite mixture of multi-variate Gaussians.
Introduces HMC method for sampling Gibbs densities.
problem Sampling from Gibbs densities efficiently.
method Hamiltonian Monte Carlo (HMC) method based on Hamiltonian dynamics.
result Idealized HMC preserves the target distribution and converges under certain conditions.
REGS samples from unnormalized distributions using gradient flow and neural networks.
problem Sampling from unnormalized distributions with high accuracy and efficiency.
method REGS is a particle method that iteratively transforms samples from a reference distribution to match an unnormalized target distribution using Wasserstein gradient flow and neural networks.
result REGS outperforms state-of-the-art methods in sampling from challenging multimodal distributions and real datasets.
Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
SPH-ParVI uses fluid dynamics to sample unknown densities efficiently.
problem Sampling partially known densities or using gradients in probabilistic models.
method Smoothed Particle Hydrodynamics (SPH) for modeling fluid dynamics to approximate target densities.
result SPH-ParVI provides fast, flexible, scalable, and deterministic sampling for Bayesian inference and generative models.
MESSY estimation recovers symbolic density functions from samples using maximum entropy.
problem Estimating probability density functions from limited samples.
method Maximum-Entropy approach with gradient flow and symbolic regression.
result Efficiently finds optimal symbolic expressions for unknown distributions.
A new copula estimation method using classification.
problem Estimating copula density from joint and marginal distributions.
method Train a classifier to distinguish joint density from product of marginals.
result Empirically outperforms existing copula estimators.
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
EBM life cycle project improves MCMC for image generation, defense, and density modeling.
problem Improving MCMC for diverse EBM applications.
method Three novel MCMC initialization methods for negative samples.
result State-of-the-art performance gains across three applications.
Estimates nonparametric densities from mixed samples.
problem Unmixing convex combinations of nonparametric densities from observed groups.
method Proposes an estimator using topic modeling and U-statistics.
result Rate-optimal estimator for nonparametric density estimation.
A new sampling method using log-concave Markov chains.
problem Sampling from unnormalized densities efficiently.
method Decomposes sampling into log-concave Markov chains with noisy measurements.
result Shows remarkable capacity to 'tunnel' between modes of a distribution.
We propose a method for nonparametric density estimation that exhibits robustness to contamination of the training sample. This method achieves robustness by combining a traditional kernel density estimator (KDE) with ideas from classical M-estimation. We interpret the KDE based on a radial, positive semi-definite ke…
This work improves density estimation by characterizing pdf complexity using NL-spectrum.
problem Improving density estimation rates for general probability densities.
method Introducing NL-spectrum to characterize pdf complexity and deriving dimension-independent rates of convergence.
result Dimension-independent rates of convergence for fast density estimation.
Unified framework for sampling from complex densities using PDEs and neural networks.
problem Sampling from complicated probability densities.
method Dynamical measure transport via PDEs and physics-informed neural networks (PINNs).
result Significantly better mode coverage and high accuracy in sampling.
DBSCAN estimates density level sets on manifolds with i.i.d. samples.
problem Estimating connected components of density level sets on manifolds.
method DBSCAN algorithm applied to i.i.d. samples.
result Rates of estimation error for different data settings.
We provide finite-sample analysis of a general framework for using k-nearest neighbor statistics to estimate functionals of a nonparametric continuous probability density, including entropies and divergences. Rather than plugging a consistent density estimate (which requires k→∞ as the sample size $n \to \in…
SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
Improved OOD detection using label smoothing and k-NN density estimates.
problem Detecting out-of-distribution examples in classification models.
method Label smoothing and k-NN density estimate on intermediate activations.
result Label smoothing improves OOD detection performance, both theoretically and empirically.