Improved score matching methods for estimating score functions and Hessians without high dimensionality.
problem Estimating score functions and Hessians efficiently in high-dimensional data.
method Implicit score matching and denoising score matching, leveraging Gagliardo-Nirenberg inequalities.
result Achieves convergence rates similar to denoising score matching and estimates Hessians without dimensionality issues.
Heuristic weighting improves denoising score matching without requiring noise distribution assumptions.
problem Improving denoising score matching without assuming noise distribution.
method Demonstrated heteroskedasticity, derived optimal weighting functions, and provided theoretical and empirical comparisons.
result Heuristical weighting function can achieve lower variance than optimal weighting, facilitating more stable and efficient training.
Study improves denoising score matching under relaxed manifold assumptions.
problem Improving denoising score matching under relaxed manifold assumptions.
method Model density with nonparametric Gaussian mixtures, relax manifold assumption, derive non-asymptotic bounds.
result Non-asymptotic bounds on approximation and generalization errors, rates of convergence determined by intrinsic dimension.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
This paper improves change-point detection for complex data streams using denoising score matching.
problem Timely identification of distributional shifts in high-dimensional, complex data streams.
method Score-based CUSUM change-point detection with denoising score matching.
result Denoising score matching enhances detection power by effectively controlling noise scale.
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
Score matching method improves image generation quality.
problem Score matching method underperforms GANs in metrics like Fréchet Inception Distance.
method DSM-ALS with denoising and improved sampling techniques.
result Score matching methods can match GANs in image generation quality.
Large learning rates prevent memorization in denoising score matching.
problem Memorization of training data in diffusion-based generative models.
method Investigating the role of large learning rates in the small-noise regime, proving that they prevent convergence to the empirical optimal score.
result Large learning rates prevent memorization by making it impossible for the learned score to be arbitrarily close to the empirical optimal score.
DDSME outperforms SME in estimating multimodal distributions.
problem Efficiency of score matching in multimodal distributions.
method Diffusion-based denoising score matching (DDSME) compared to vanilla score matching (SME).
result DDSME avoids the error bound deterioration of SME with increasing mode separation.
Unified framework for denoising models across various spaces.
problem Improving generative models and approximate posterior simulation.
method Generalizing denoising diffusions to a broader class of spaces using a new extension of score matching.
result Unified models for denoising and posterior simulation.
New score matching method estimates local intrinsic dimension efficiently.
problem Quantifying the local intrinsic dimension of complex data.
method Denoising score matching loss and equivalent implicit score matching loss.
result Denoising score matching loss is a highly competitive and scalable LID estimator.
PDDS samples from unnormalized densities using iterative particle scheme.
problem Sampling from unnormalized probability densities.
method Iterative particle scheme with novel score matching loss.
result Asymptotically consistent estimates for multimodal and high-dimensional tasks.
Energy-Based Models (EBMs) assign unnormalized log-probability to data samples. This functionality has a variety of applications, such as sample synthesis, data denoising, sample restoration, outlier detection, Bayesian reasoning, and many more. But training of EBMs using standard maximum likelihood is extremely slow b…
Simplifies denoising score matching for manifold learning.
problem Learning distributions on manifolds is computationally intensive.
method Modifies denoising score matching to implicitly account for the manifold.
result Reduces computational burden while maintaining efficiency.
A method to estimate high order derivatives of data distributions from samples.
problem Estimating high order derivatives of data distributions efficiently and accurately.
method Generalizing denoising score matching via Tweedie's formula to estimate higher order derivatives.
result Models trained with the proposed method can approximate second order derivatives more efficiently and accurately than via automatic differentiation.
A faster method for density estimation using denoising score matching with random Fourier features.
problem Intractability of normalizing constant in density estimation.
method Denoising Score Matching with Random Fourier Features.
result The method is computationally faster and scalable to complex high-dimensional data.
Enhances learning of structured distributions using nonlinear denoising score matching.
problem Learning structured distributions from noisy data.
method Latent Nonlinear Denoising Score Matching (LNDSM) integrating nonlinear dynamics with VAE-based latent score matching.
result LNDSM achieves superior sample quality and variability compared to structure-agnostic methods.
Automated denoising score matching handles nonlinear diffusion processes.
problem Nonlinear diffusion processes limit generative modeling and property estimation.
method Local-DSM using local increments and Taylor expansions.
result Tractable training and score estimation for nonlinear diffusion processes.
Improves generative models by using heavy-tailed noise in score matching.
problem High-dimensional limitations of Gaussian noise in generative models.
method Extended DSM to generalised normal distribution, relaxed key assumptions, developed iterative noise scaling algorithm.
result Heavy-tailed DSM leads to improved generative performance.
New method improves sampling from noisy energy models.
problem Training and sampling challenges in Energy-Based Models.
method Pseudo-Gibbs sampling with moment matching.
result Effective sampling from clean model using noisy DSM-trained model.
DDS samples from noisy data by reversing diffusion, providing theoretical guarantees.
problem Sampling from unnormalized densities.
method Denoising diffusion process, score matching, optimal control, Schrödinger bridges.
result DDS provides theoretical guarantees for sampling.
New method denoises images without clean reference using Tweedie distributions.
problem Image denoising without clean reference images.
method Combining Tweedie distributions, Noise2Score, and saddle point approximation.
result General closed-form denoising formula for various noise distributions.
High-quality image synthesis with diffusion models, achieving state-of-the-art FID score.
problem Generating high-quality images from latent variables.
method Training diffusion probabilistic models with a weighted variational bound, inspired by denoising score matching and Langevin dynamics.
result State-of-the-art FID score of 3.17 on CIFAR10 dataset.
New inequalities for unbounded functions improve denoising score matching.
problem Statistical error bounds for denoising score matching with unbounded objective functions.
method Derive new concentration inequalities using McDiarmid's inequality and Rademacher complexity bounds.
result Improved statistical error bounds for denoising score matching.
AR-CSM models use derivatives of univariate log-conditionals to estimate joint distributions efficiently.
problem Scalability and stability issues in training autoregressive models.
method Parameterize joint distribution using derivatives of univariate log-conditionals and introduce Composite Score Matching (CSM) for efficient training.
result AR-CSM models are more scalable and stable compared to previous score matching algorithms.
A new method improves data generation quality by correcting score mismatches.
problem Score mismatch issue in conditional score-based data generation methods.
method Denoising Likelihood Score Matching (DLSM) loss for classifier training.
result The proposed method outperforms previous methods on Cifar-10 and Cifar-100 benchmarks.
Exact posterior score estimation for solving linear inverse problems
problem Solving linear inverse problems
method Derive the exact posterior score and use it as a denoising training objective
result EPS outperforms training-free and training-based baselines on various metrics
New denoisers improve signal recovery from noisy data without knowing noise distribution.
problem Denoising signals when only noise level is known, not distribution.
method Universal denoisers that shrink PY toward PX with higher-order accuracy. result Achieves O(σ4) and O(σ6) accuracy in matching generalized moments and densities. New method uses SURE to denoise signals, outperforming NPMLE.
problem Learning to optimally denoise signals corrupted by Gaussian noise.
method Hyvärinen's score matching (SM) is shown equivalent to SURE minimization.
result SURE achieves nearly parametric rates of convergence in empirical Bayes settings.
This work uncovers algorithm-dependent regularisation in diffusion models.
problem Understanding and improving generalisation in high-dimensional diffusion models.
method Algorithmic stability and score stability analysis.
result Identifies multiple sources of implicit regularisation unique to diffusion models.
We study convergence of a generative modeling method that first estimates the score function of the distribution using Denoising Auto-Encoders (DAE) or Denoising Score Matching (DSM) and then employs Langevin diffusion for sampling. We show that both DAE and DSM provide estimates of the score of the Gaussian smoothed p…
Density estimation is a fundamental problem in statistical learning. This problem is especially challenging for complex high-dimensional data due to the curse of dimensionality. A promising solution to this problem is given here in an inference-free hierarchical framework that is built on score matching. We revisit the…
Paper improves molecular property prediction using denoising autoencoders.
problem Limited data for molecular property prediction from 3D structures.
method Pre-training via denoising for learning molecular force fields.
result Achieves new state-of-the-art performance on QM9 dataset.
New method DDVI improves posterior inference for deep Gaussian processes.
problem Inference of inducing points in DGPs is challenging and biased.
method DDVI uses denoising diffusion SDE and score matching for posterior approximation.
result Empirically shows DDVI outperforms baseline methods in inducing point inference.
DSM on manifolds removes singularities and computes small-noise expansions.
problem DSM on manifolds with singular noise.
method Rao-Blackwellized score matching, nearest-point projection, intrinsic Riemannian score.
result Canonical target equals intrinsic Riemannian score up to a small correction.
ScoreMatchingRiesz improves debiased machine learning and policy effects estimation.
problem Improving debiased machine learning and policy effects estimation.
method Score matching and Riesz representer estimation.
result Estimates policy path for continuous treatments, improving interpretability.
Semi-Implicit Variational Inference (SIVI) is improved with SIVI-SM using score matching.
problem Intractable densities in variational distributions hinder SIVI training.
method SIVI-SM uses score matching to handle intractable densities in a minimax formulation.
result SIVI-SM outperforms ELBO-based SIVI methods in Bayesian inference tasks.
New findings show score matching's accuracy doesn't ensure numerical stability in diffusion sampling.
problem Numerical stability issues in diffusion sampling despite small forward-marginal error.
method Constructing a smooth score field with arbitrarily small forward-marginal L2 error, showing nonexplosive behavior and moments of every order. result Euler--Maruyama discretizations can converge in probability even when moments diverge, demonstrating failure of weak convergence.
This work introduces a new data-driven estimator for the Bayesian Cramér-Rao bound using score matching.
problem Benchmarking the performance of statistical estimators and providing a principled metric for system design and optimization.
method Score matching technique to estimate the Bayesian Cramér-Rao bound from training data.
result Developed novel non-asymptotic bounds on the score matching error and the Bayesian CRB estimator.
This work extends Tweedie's formulae to non-Gaussian processes for better diffusion model generation.
problem Limited exploration of non-Gaussian diffusion models and corresponding Tweedie's formulae.
method Extended Tweedie's formulae to geometric Brownian motion, squared Bessel, and Cox-Ingersoll-Ross processes.
result Demonstrated potential of non-Gaussian models in image and financial time series generation.
Paper bridges score estimation to parameter and density estimation in DDPMs.
problem Efficiently estimating scores for generative models.
method Introduces a framework linking score estimation to parameter and density estimation.
result Denoising score-matching in DDPMs is asymptotically efficient for parameter estimation.
SJDs unify masked, continuous, and hybrid diffusion models.
problem Unified modeling of diffusion processes.
method Continuous-time Markov processes with token embeddings and hazard rates.
result Unified model recovers masked, continuous, and hybrid diffusion as limits.
Unified approach to denoising Markov models for efficient sampling.
problem Designing efficient sampling algorithms for complex distributions.
method Mathematical foundation using measure transport and nonequilibrium statistical mechanics.
result Unified variational objective and backward generator construction.
A new method for generating samples without training, using smoothed score matching.
problem Generating samples efficiently and without training.
method Moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD).
result The method enables fast, robust, training-free sampling with competitive sample fidelity and diversity.
The study examines how the number of noise samples affects diffusion models' performance.
problem Understanding the balance between generalization and memorization in diffusion models.
method Theoretical analysis and empirical experiments with Denoising Score Matching (DSM) using random features.
result Precise expressions for test and train errors under specific conditions reveal the mechanisms of generalization and memorization.
The paper constructs denoisers that recover the Brenier map from higher-order score functions.
problem Estimating the Brenier map from noisy data.
method Constructs a hierarchy of denoisers using higher-order score functions.
result The T∞ denoiser recovers the Brenier map from the additive Gaussian model. This work extends diffusion models to function space for better generative modeling.
problem Limited applicability of diffusion models to functional data domains.
method Introduces Denoising Diffusion Operators (DDOs) for training diffusion models in function space.
result Demonstrates accurate function-valued generation at fixed cost.
Improved generative models using critically-damped Langevin diffusion.
problem Current score-based generative models (SGMs) use overly simplistic diffusion processes, leading to complex denoising tasks and suboptimal performance.
method Proposed a novel critically-damped Langevin diffusion (CLD) and derived a score matching objective and sampling scheme.
result CLD-based SGMs achieve superior performance in synthesis quality compared to previous methods.