Estimates unnormalized models with missing data using imputation and noise contrastive estimation.
problem Statistical models with intractable normalization constants and missing data.
method Combines imputation techniques with estimators for unnormalized models like noise contrastive estimation and score matching.
result Effective statistical inference with unnormalized models from missing data.
Unified framework for efficient estimation of unnormalized models.
problem Estimation of unnormalized models with statistical efficiency.
method Unified estimation framework combining density-ratio matching and nonparametric estimators.
result Asymptotic variance of proposed estimators is the same as MLE.
New method estimates model discrepancy without sampling for unnormalized models.
problem Evaluating and training unnormalized density models efficiently.
method Estimate Stein discrepancy using neural network parameterized vector function.
result Method outperforms existing goodness-of-fit tests and training methods.
Paper presents a new method to learn unnormalized models efficiently.
problem Scalability issues in score matching for flexible unnormalized models.
method Connects learning objectives to Wasserstein gradient flows for scalability.
result Demonstrates improved learning of unnormalized models on manifolds.
Unified view on learning unnormalized distributions using NCE.
problem Learning unnormalized distributions across different communities.
method Noise-Contrastive Estimation (NCE) for a unified perspective.
result Established finite-sample convergence rates for exponential families.
Adversarial learning tackles unnormalized densities without samples.
problem Learning from unnormalized densities without direct samples.
method Adversarial learning extended to unnormalized densities, with new GAN regularization concepts.
result Encouraging results across applications, including deep soft Q-learning.
New CUSUM algorithm detects changes in unnormalized models.
problem Change detection in models with unknown normalization constants.
method Score-based CUSUM (SCUSUM) algorithm based on Fisher divergence and Hyvärinen score.
result Asymptotic optimality of the SCUSUM algorithm demonstrated.
Unified empirical and variational Bayes for unnormalized densities.
problem Approximating unnormalized densities using latent variable models.
method Formulate a latent variable model for Y = X + N ( 0 , σ 2 I d ) Y=X+N(0,σ^2 I_d) Y = X + N ( 0 , σ 2 I d ) , use ELBO as parametrization of Y Y Y 's energy function, and estimate X X X with empirical Bayes least-squares. result UVB has higher capacity to approximate energy functions than MLPs in DEEN.
REGS samples from unnormalized distributions using gradient flow and neural networks.
problem Sampling from unnormalized distributions with high accuracy and efficiency.
method REGS is a particle method that iteratively transforms samples from a reference distribution to match an unnormalized target distribution using Wasserstein gradient flow and neural networks.
result REGS outperforms state-of-the-art methods in sampling from challenging multimodal distributions and real datasets.
Paper analyzes NCE method for unnormalized models, reducing asymptotic variance.
problem Estimating parameters of unnormalized models with high asymptotic variance.
method Proposes a method to reduce asymptotic variance by estimating auxiliary distribution parameters and analyzing objective function forms.
result NCE estimator is consistent and asymptotically normal, with reduced variance.
CD algorithm achieves near-optimal convergence rate for unnormalized models.
problem Training unnormalized models with high efficiency.
method Non-asymptotic analysis of contrastive divergence algorithm.
result CD can achieve O ( n − 1 / 2 ) O(n^{-1 / 2}) O ( n − 1/2 ) convergence rate under regularity assumptions. PDDS samples from unnormalized densities using iterative particle scheme.
problem Sampling from unnormalized probability densities.
method Iterative particle scheme with novel score matching loss.
result Asymptotically consistent estimates for multimodal and high-dimensional tasks.
We show that the Bregman divergence provides a rich framework to estimate unnormalized statistical models for continuous or discrete random variables, that is, models which do not integrate or sum to one, respectively. We prove that recent estimation methods such as noise-contrastive estimation, ratio matching, and sco…
Paper optimizes change detection in unnormalized distributions.
problem Detecting changes in unnormalized pre- and post-change distributions.
method Log-Partition Approximation Cumulative Sum (LPA-CUSUM) algorithm based on thermodynamic integration.
result Asymptotically optimal performance achieved through unbiased estimation of CUSUM statistics.
We develop underdamped diffusion bridges for sampling from unnormalized densities.
problem Sampling from unnormalized densities without direct access to samples.
method Underdamped diffusion bridges with rigorous score matching equivalence.
result State-of-the-art performance in sampling across various problems.
iEFM trains CNF models from unnormalized densities efficiently.
problem Training generators from energy functions or unnormalized densities.
method Iterated energy-based flow matching (iEFM) with simulation-free objective.
result iEFM outperforms existing methods in probabilistic modeling.
Single-step samplers generate high-quality samples efficiently.
problem Sampling from unnormalized distributions is computationally expensive.
method Developed consistent diffusion samplers that generate samples in a single step.
result Single-step samplers produce high-fidelity samples with less than 1% of traditional samplers' evaluations.
Proposes a method combining CNFs and rejection-resampling for sampling from unnormalized densities.
problem Sampling from unnormalized probability densities, especially multimodal ones.
method Combines continuous normalizing flows with rejection-resampling steps based on importance weights.
result The method improves sampling accuracy and performance compared to state-of-the-art methods.
A new approach to distill unnormalized EBM for energy-based seq2seq models.
problem Training unnormalized EBM for energy-based seq2seq models is challenging.
method Relating the problem to distributional RL, proposing a general distillation approach.
result General approach applicable to any sequential EBM, illustrated on GAM experiments.
A new sequential test for unnormalized densities.
problem Testing unnormalized densities with adaptive stopping.
method Sequential kernelized Stein discrepancy test, using non-uniform Stein kernels.
result Valid test with asymptotic lower bound for growth.
A new model estimates complex densities without explicit normalizing constants.
problem Accurately estimating the normalizing constant of high-dimensional energy functions.
method Autoregressive Energy Machine (AEM) learns an unnormalized density and an importance-sampling estimate of the normalizing constant.
result Achieves state-of-the-art performance on density-estimation tasks.
A problem about Khovanov homology and 3-manifolds is discussed in this paper.
A new method optimizes a generalized Kullback-Leibler divergence for better simulation-based inference.
problem Optimizing likelihood functions when they are only known implicitly.
method Optimizes a generalized Kullback-Leibler divergence that accounts for normalization constants in unnormalized distributions.
result Unified approach that combines Neural Posterior Estimation and Neural Ratio Estimation.
A new method called TemperFlow tackles multimodality in sampling from unnormalized distributions.
problem Sampling from unnormalized distributions with isolated modes.
method TemperFlow learns a sequence of tempered distributions to progressively approach the target distribution.
result TemperFlow overcomes the limitations of existing methods and achieves superior performance.
LFIS uses a time-dependent velocity field to sample from complex distributions.
problem Sampling from unnormalized density functions.
method LFIS learns a time-dependent velocity field to transport samples from a simple initial distribution to a complex target distribution.
result LFIS achieves state-of-the-art performance on various benchmark problems.
New method minimizes robust density power-based divergences for general parametric densities.
problem Computational complexity of minimizing DPD for general parametric densities.
method Stochastic approach to minimize DPD for general parametric density models.
result Proposed method can be applied to minimize other density power-based γ-divergences.
We introduce a new family of estimators for unnormalized statistical models. Our family of estimators is parameterized by two nonlinear functions and uses a single sample from an auxiliary distribution, generalizing Maximum Likelihood Monte Carlo estimation of Geyer and Thompson (1992). The family is such that we can e…
Given any diagram of a link, we define on the cube of Kauffman's states a "2-complex" whose homology is an invariant of the associated framed links, and such that the graded Euler characteristic reproduces the unnormalized Kauffman bracket. This includes a categorification of brackets skein relation. Then we incorporat…
DDS samples from noisy data by reversing diffusion, providing theoretical guarantees.
problem Sampling from unnormalized densities.
method Denoising diffusion process, score matching, optimal control, Schrödinger bridges.
result DDS provides theoretical guarantees for sampling.
SoftCVI uses contrastive estimation to infer complex posteriors.
problem Estimating complex posteriors in Bayesian inference.
method Contrastive variational inference with self-generated soft labels.
result SoftCVI outperforms other variational approaches in stability and coverage.
Morse neural networks improve uncertainty quantification and detection.
problem Uncertainty quantification and out-of-distribution detection.
method Generalizes unnormalized Gaussian densities to high-dimensional submanifolds using KL-divergence loss.
result Unified approach for OOD detection, anomaly detection, and continuous learning.
We construct the universal sl(2)-tangle cohomology using an approach with webs and dotted foams. This theory depends on two parameters, and for the case of links it is a categorification of the unnormalized Jones polynomial of the link.
We establish a parabolic version of Tian's C 2 , α C^{2,α} C 2 , α -estimate for conical complex Monge-Ampere equations, which includes conical Kähler-Einstein metrics. Our estimate will complete the proof of the existence of unnormalized conical Kähler-Ricci flow in arXiv:1411.7284.
New methods improve memory efficiency for sampling from complex distributions.
problem Sampling from complex unnormalized distributions over discrete domains.
method Two novel training methods for discrete diffusion samplers.
result Achieve state-of-the-art results in unsupervised combinatorial optimization.
A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
New method for sampling from complex distributions using stochastic localization.
problem Sampling from unnormalized target densities in multi-modal distributions.
method Stochastic Localization via Iterative Posterior Sampling (SLIPS) framework.
result Approximate samples from target distribution and denoiser learned iteratively.
Graph Laplacians adapt to different manifold dimensions, while Dirichlet energies converge to a tensorized Dirichlet energy.
problem Understanding machine learning methods for data with varying intrinsic dimensions.
method Γ-convergence of graph Dirichlet energies and spectral convergence of graph Laplacians on intersecting manifolds of varying dimensions.
result Normalized Dirichlet energy converges to a tensorized Dirichlet energy that adapts to all dimensions simultaneously.
We generalize the maximal time existence of Kähler-Ricci flow in Tian-Zhang and Song-Tian to conical case. Furthermore, if the twisted canonical bundle K M + ( 1 − β ) [ D ] K_{M}+(1-β)[D] K M + ( 1 − β ) [ D ] is big or big and nef, we can expect more on the limit behaviors of such conical Kähler-Ricci flow. Moreover, the results still hold for simple normal …
Ancient solutions found on flag manifolds from invariant Einstein metrics.
problem Understanding the behavior of Ricci flow on flag manifolds.
method Global study of the dynamical system induced by the Ricci flow, using invariant Einstein metrics and Poincaré compactification.
result Non-collapsed ancient solutions emerge from invariant Einstein metrics, with a Type I singularity in finite time.
A new variational inference method using sliced Wasserstein distance is proposed.
problem The inefficiency and unreasonable properties of Kullback-Leibler divergence.
method Minimizing sliced Wasserstein distance, a valid metric from optimal transport.
result The proposed method approximates the unnormalized distribution efficiently and without requiring a tractable density function.
Improved rank aggregation via spectral method reduces sample complexity.
problem Ranking items from pairwise comparisons with corrupted data.
method Spectral ranking algorithms based on unnormalized and normalized data matrices.
result Sharper ℓ ∞ \ell_{\infty} ℓ ∞ -norm perturbation bound and error bound on maximum displacement for each item. MT-SGD samples from multiple target distributions using gradient descent.
problem Sampling from multiple unnormalized target distributions.
method Proposes MT-SGD, a flow of intermediate distributions to sample from multiple target distributions.
result Asymptotic analysis shows MT-SGD reduces to multiple-gradient descent for multi-objective optimization.
CR-AIS improves AIS efficiency by constant rate annealing.
problem Efficiently sample from intractable distributions.
method Constant rate annealing schedule for AIS.
result CR-AIS outperforms existing Adaptive AIS methods.
SMRL uses score matching for efficient RL with exponential family models.
problem Efficient reinforcement learning with unknown transition models.
method Score matching for parameter estimation of exponential family models.
result Achieves online regret of O ( d H 3 T ) O(d\sqrt{H^3T}) O ( d H 3 T ) . New sampling method uses gradient-free IPS with RKHS velocity field.
problem Efficient sampling from unnormalized target densities.
method Gradient-free interacting particle systems (IPS) with RKHS velocity field.
result IPS produce high-quality samples from various target distributions.
Improved diffusion bridge sampling with rKL-LD loss.
problem Improving sampling from unnormalized distributions using diffusion bridges.
method Employing the rKL-LD loss instead of the Log Variance (LV) loss for diffusion bridges.
result rKL-LD consistently outperforms LV loss in diffusion bridges.
We investigate the limiting behavior of the unnormalized Kahler-Ricci flow on a Kahler manifold with a polarized initial Kahler metric. We prove that the Kahler-Ricci flow becomes extinct in finite time if and only if the manifold has positive first Chern class and the initial Kahler class is proportional to the first …
Learning latent variable models with stochastic variational inference is challenging when the approximate posterior is far from the true posterior, due to high variance in the gradient estimates. We propose a novel rejection sampling step that discards samples from the variational posterior which are assigned low likel…