The paper establishes bounds for score-matching in causal discovery and generative modeling.
problem Estimating causal relationships from data.
method Training a deep neural network to estimate the score function and applying it to causal discovery.
result Bounds on the error rate of causal discovery methods using score-matching.
Large learning rates prevent memorization in denoising score matching.
problem Memorization of training data in diffusion-based generative models.
method Investigating the role of large learning rates in the small-noise regime, proving that they prevent convergence to the empirical optimal score.
result Large learning rates prevent memorization by making it impossible for the learned score to be arbitrarily close to the empirical optimal score.
SDE Matching eliminates simulation for training Latent SDEs, achieving similar performance.
problem Training Latent SDEs with adjoint sensitivity methods is computationally expensive and limited.
method SDE Matching, inspired by Score- and Flow Matching, eliminates simulation for training Latent SDEs.
result SDE Matching achieves performance comparable to adjoint sensitivity methods while reducing computational complexity.
A new method for stochastic optimal control improves accuracy over existing techniques.
problem Improving the accuracy of stochastic optimal control for noisy systems.
method Stochastic Optimal Control Matching (SOCM) using Iterative Diffusion Optimization (IDO) with path-wise reparameterization trick.
result SOCM achieves lower error than existing techniques for three out of four control problems, sometimes by an order of magnitude.
A new method for training diffusion models using likelihood matching.
problem Training efficient and accurate diffusion models.
method Likelihood Matching approach, quasi-likelihood approximation, score and Hessian estimation.
result Consistent matching of first two transitional moments between diffusion steps.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
WS diffusion models handle anisotropic Gaussian noise better than conventional methods.
problem Handling anisotropic Gaussian noise in imaging inverse problems.
method Whitened Score (WS) diffusion models based on stochastic differential equations.
result WS DMs outperform conventional DMs on anisotropic Gaussian noise.
This paper conditions non-linear infinite-dimensional diffusion processes.
problem Conditioning non-linear and infinite-dimensional diffusion processes.
method Infinite-dimensional Girsanov's theorem to condition function-valued stochastic processes.
result Conditioning of non-linear infinite-dimensional diffusion processes is achieved.
New inequalities for unbounded functions improve denoising score matching.
problem Statistical error bounds for denoising score matching with unbounded objective functions.
method Derive new concentration inequalities using McDiarmid's inequality and Rademacher complexity bounds.
result Improved statistical error bounds for denoising score matching.
SSDMs generate quantum states directly, outperforming classical methods.
problem Generating pure-state quantum representations efficiently.
method Score-based generative model on complex projective manifold.
result SSDMs match target pure-state ensembles by orders of magnitude.
New analysis improves convergence guarantees for diffusion-based samplers in Wasserstein distance.
problem Improving convergence guarantees for diffusion-based generative models.
method Simple framework to analyze discretization, initialization, and score estimation errors.
result First Wasserstein convergence bound for the Heun sampler and improved results for Euler sampler.
Grad-TTS models speech from text using diffusion probabilistic techniques.
problem Creating high-quality speech from text input.
method Score-based decoder with stochastic differential equations for noise-to-speech transformation.
result Grad-TTS produces mel-spectrograms from text input with competitive quality.
iDEM generates samples from Boltzmann densities without data.
problem Generating statistically independent samples from unnormalized distributions.
method Iterative algorithm using energy and gradient for diffusion-based sampler training.
result iDEM achieves state-of-the-art performance and trains faster than existing methods.
Unified framework for training diffusion and flow models to sample from target distributions.
problem Training diffusion and flow models to sample from target distributions defined by exponential tilting.
method Unified framework combining stochastic optimal control and non-equilibrium thermodynamics perspectives.
result Unified bias-variance decompositions and theoretical support for adjoint-based methods.
When maximum likelihood estimation is infeasible, one often turns to score matching, contrastive divergence, or minimum probability flow to obtain tractable parameter estimates. We provide a unifying perspective of these techniques as minimum Stein discrepancy estimators, and use this lens to design new diffusion kerne…
Improves BBVI for high-dimensional Gaussian approximations by using low-rank approximations.
problem Scalability issues with BBVI for high-dimensional multivariate Gaussian approximations.
method Extends BaM framework to handle full covariance matrices by integrating patch step for low-rank parameterization.
result Shows improved efficiency and scalability on synthetic and real-world high-dimensional inference problems.
This work analyzes SGD for SGMs, providing convergence rates and error bounds.
problem Optimization dynamics of SGMs trained with stochastic gradients.
method Non-convex convergence rate analysis and Neural Tangent Kernel analysis.
result Theoretical insights into SGD convergence and error bounds for SGMs.
Paper introduces FDM for efficient training of Neural SDEs.
problem Training Neural SDEs using existing methods is computationally expensive and unstable.
method Developed a novel scoring rule called Finite Dimensional Matching (FDM) to bypass signature kernels and reduce training complexity.
result FDM achieves superior performance in terms of computational efficiency and generative quality.
Proximal Diffusion Models improve generative model efficiency.
problem Improving generative model efficiency and accuracy.
method Developed Proximal Diffusion Models using proximal maps instead of scores.
result Proximal Diffusion Models achieve faster convergence and higher accuracy.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
New risk bound for drift estimator in stochastic models.
problem Theoretical guarantees for drift estimation in stochastic differential equations.
method Derives an explicit risk bound using diffusion model theory.
result Explicit decomposition of risk into multiple sources of error.
Density estimation is a fundamental problem in statistical learning. This problem is especially challenging for complex high-dimensional data due to the curse of dimensionality. A promising solution to this problem is given here in an inference-free hierarchical framework that is built on score matching. We revisit the…
CatFlow uses variational flow matching for efficient graph generation.
problem Graph generation tasks, especially for categorical data.
method Variational flow matching for categorical data, computationally efficient.
result CatFlow achieves strong results on graph generation tasks.
New method modifies diffusions for singular rewards.
problem Handling singular rewards in diffusions.
method Malliavin calculus for non-differentiable rewards.
result Stable and reliable training of diffusions.
Neural score matching improves high-dimensional causal inference by using neural networks for balancing scores.
problem Impracticality of traditional matching methods in high-dimensional datasets due to the curse of dimensionality.
method Develops neural networks to create non-trivial, multivariate balancing scores for high-dimensional causal inference.
result Neural score matching outperforms other methods in treatment effect estimation and reducing imbalance on high-dimensional datasets.
Improved score matching methods for estimating score functions and Hessians without high dimensionality.
problem Estimating score functions and Hessians efficiently in high-dimensional data.
method Implicit score matching and denoising score matching, leveraging Gagliardo-Nirenberg inequalities.
result Achieves convergence rates similar to denoising score matching and estimates Hessians without dimensionality issues.
New error bounds for flow matching methods using deterministic sampling.
problem Improving the accuracy of flow matching methods for generating probability distributions.
method Derived error bounds for flow matching methods under deterministic sampling conditions.
result Presented error bounds for flow matching methods using L2 loss and regularity conditions. Score matching is a recently developed parameter learning method that is particularly effective to complicated high dimensional density models with intractable partition functions. In this paper, we study two issues that have not been completely resolved for score matching. First, we provide a formal link between maxim…
New approach to score function in diffusion models using Malliavin calculus.
problem Estimating score function for complex data distributions.
method Combines Malliavin calculus with Bismut-type formula to derive exact score function expression.
result Derives exact, closed-form expression for score function in diffusion models.
Score matching is a popular method for estimating unnormalized statistical models. However, it has been so far limited to simple, shallow models or low-dimensional data, due to the difficulty of computing the Hessian of log-density functions. We show this difficulty can be mitigated by projecting the scores onto random…
New algorithms learn graph structures privately, matching best results.
problem Private learning of graph structures with multiple blocks.
method Sum-of-squares relaxation and exponential mechanism for score function.
result Matches statistical utility of previous best non-private methods.
New method for sampling diffusion bridges on sub-Riemannian manifolds.
problem Sampling conditioned diffusion processes on sub-Riemannian manifolds is challenging.
method Score matching for machine learning, adapted to non-holonomic frames.
result Demonstrated method works on Heisenberg group and other sub-Riemannian manifolds.
New method calibrates DPMs to improve likelihood bounds.
problem Improving the likelihood bounds of DPMs.
method Deriving concentration bounds and using the optional stopping theorem for data scores to calibrate DPMs.
result Calibrated DPMs can increase likelihood bounds and improve sampling quality.
This work extends Tweedie's formulae to non-Gaussian processes for better diffusion model generation.
problem Limited exploration of non-Gaussian diffusion models and corresponding Tweedie's formulae.
method Extended Tweedie's formulae to geometric Brownian motion, squared Bessel, and Cox-Ingersoll-Ross processes.
result Demonstrated potential of non-Gaussian models in image and financial time series generation.
MARS meta-learns function scores for improved predictive accuracy and uncertainty.
problem Difficulty in specifying expressive priors for Bayesian meta-learning.
method Meta-learning the score function of data-generating process marginals in the function space.
result State-of-the-art predictive accuracy and improved uncertainty estimates.
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
New approach to control diffusion processes with soft constraints.
problem Finding an optimal diffusion process with a target terminal distribution.
method Generalized Schrödinger bridge problem with soft constraints, solving for a geometric mixture of target and other distributions.
result The terminal distribution of the optimally controlled process is a geometric mixture of the target and another distribution.
DDSME outperforms SME in estimating multimodal distributions.
problem Efficiency of score matching in multimodal distributions.
method Diffusion-based denoising score matching (DDSME) compared to vanilla score matching (SME).
result DDSME avoids the error bound deterioration of SME with increasing mode separation.
This work interprets diffusion score matching using normalizing flows for better model training and evaluations.
problem Limitations of diffusion score matching when dealing with certain types of distributions.
method The approach involves interpreting the diffusion matrix using normalizing flows to provide better interpretation and usage of diffusion score matching.
result Diffusion score matching is equivalent to the original score matching evaluated in the transformed space defined by the normalizing flow.
TRACE improves conformal prediction for multi-dimensional outputs.
problem Challenges in constructing valid and informative conformal prediction regions for multi-dimensional outputs.
method TRACE uses transport alignment in diffusion and flow matching models to define nonconformity scores.
result TRACE yields valid and adaptive conformal prediction regions for multimodal and non-convex distributions.
Develops Hamiltonian Score Matching and Generative Flows for machine learning.
problem Estimating score functions and designing generative models.
method Introduces Hamiltonian velocity predictors (HVPs) for score matching and generative flows.
result Hamiltonian Generative Flows (HGFs) rival leading generative modeling techniques.
New bounds close the score matching gap for diffusion models.
problem The difference between sample quality and score matching loss in diffusion models.
method Theoretical analysis of score matching gap, developing tighter bounds for KL divergence, reverse KL divergence, and Wasserstein distance.
result The quality of score approximation impacts closing the score matching gap for low noise scales.
Robust score matching improves parameter estimation in contaminated data.
problem Parameter estimation in data contaminated by outliers.
method Geometric median of means to develop a robust score matching procedure.
result Consistent parameter estimates in contaminated data settings.
Score matching fails for general point processes, a new estimator improves accuracy.
problem Score matching estimators fail for general point processes.
method Introduced a weighted score matching estimator for point processes.
result The weighted score matching estimator accurately estimates model parameters on synthetic and real data.
Score matching method improves image generation quality.
problem Score matching method underperforms GANs in metrics like Fréchet Inception Distance.
method DSM-ALS with denoising and improved sampling techniques.
result Score matching methods can match GANs in image generation quality.
Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.
problem Intractable densities in variational distributions hinder SIVI training.
method SIVI-SM uses score matching to handle intractable densities in a minimax formulation.
result SIVI-SM outperforms ELBO-based SIVI methods in Bayesian inference tasks.
ScoreMatchingRiesz improves debiased machine learning and policy effects estimation.
problem Improving debiased machine learning and policy effects estimation.
method Score matching and Riesz representer estimation.
result Estimates policy path for continuous treatments, improving interpretability.
Score matching method improves density estimation for truncated data on manifolds.
problem Density estimation for truncated data on manifolds with intractable normalising constant.
method Truncated score matching extended to Riemannian manifolds with boundary.
result Score matching estimator approximates true parameter values with low error.