SGMs can generate samples from low-dimensional data manifolds.
problem Understanding the conditions under which SGMs can produce samples from a low-dimensional data manifold.
method Analyzing the conditions for SGMs to approximate the scores and generate samples from a manifold.
result Precise conditions for SGMs to generate samples from a low-dimensional data manifold.
RSGMs extend SGMs to Riemannian manifolds for better data modeling.
problem Current SGMs are limited to Euclidean spaces; RSGMs handle Riemannian manifolds.
method RSGMs use a noising stage with a diffusion process and a denoising model approximating the time-reversal of the diffusion on Riemannian manifolds.
result RSGMs improve generative modeling for data on Riemannian manifolds.
SGMs fail to generate samples from complex distributions even when the score function is learned well.
problem Score-based Generative Models fail to produce high-quality samples from complex distributions.
method Score-based Generative Models (SGMs) are evaluated under conditions where the score function is learned well.
result SGMs can only generate Gaussian blurring of training data points, not complex distributions.
New analysis improves understanding of SGMs' generalization.
problem Improving understanding of SGMs' generalization behavior.
method Algorithm- and data-dependent generalization analysis.
result Established bounds accounting for optimization dynamics.
The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadr…
LSGM trains SGMs in latent space for faster sampling.
problem Efficiently generating high-quality samples from complex distributions.
method LSGM trains SGMs in latent space using a variational autoencoder framework, introducing new score-matching objectives and parameterizations.
result LSGM achieves state-of-the-art FID score of 2.10 on CIFAR-10 and outperforms previous SGMs in sampling time.
The paper establishes convergence guarantees for SGMs in 2-Wasserstein distance.
problem Establishing convergence guarantees for SGMs in 2-Wasserstein distance.
method Assuming accurate score estimates and smooth log-concave data distribution, the paper specializes its result to several concrete SGMs with specific forward processes modeled by stochastic differential equations.
result Obtained an upper bound on the iteration complexity for each model and a lower bound for Gaussian data distribution.
This work analyzes SGD for SGMs, providing convergence rates and error bounds.
problem Optimization dynamics of SGMs trained with stochastic gradients.
method Non-convex convergence rate analysis and Neural Tangent Kernel analysis.
result Theoretical insights into SGD convergence and error bounds for SGMs.
We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes over the data. We show that optimal generalization error bounds c…
Proposes SGM for modeling complex dependencies in high-dimensional systems.
problem Limited pairwise interactions in PGMs for high-dimensional systems.
method Simplicial Gaussian model (SGM) using discrete Hodge theory and independent random components.
result Maximum-likelihood inference algorithm for parameter recovery and conditional dependence structure.
Extends SGM to functional spaces for multimodal data.
problem Modeling densities in functional spaces.
method Represent data in spectral space, dissociate stochastic and space-time components, use SGM for sampling.
result Demonstrates effectiveness on multimodal datasets.
Improved generative models using critically-damped Langevin diffusion.
problem Current score-based generative models (SGMs) use overly simplistic diffusion processes, leading to complex denoising tasks and suboptimal performance.
method Proposed a novel critically-damped Langevin diffusion (CLD) and derived a score matching objective and sampling scheme.
result CLD-based SGMs achieve superior performance in synthesis quality compared to previous methods.
A new method for diffusion generative models improves sample quality and speed.
problem Improving the efficiency and quality of diffusion generative models.
method Developed a complete recipe for forward diffusion processes in SGMs, introducing PSLD.
result PSLD achieves superior sample quality and speed compared to existing methods.
New method generates novel samples from closed-form diffusion models.
problem Closed-form SGMs memorize training data and cannot generate novel samples.
method Explicitly smooth closed-form score, use nearest-neighbor estimator.
result Efficient method generates novel samples without training.
SGM combines deep learning and planning for robust long-horizon tasks.
problem Combining deep learning and planning for robust long-horizon tasks.
method Sparse Graphical Memory (SGM) that stores states and feasible transitions in a sparse memory, aggregating states according to a two-way consistency objective.
result SGM significantly outperforms current state of the art methods on long horizon, sparse-reward visual navigation tasks.
Mathematical analysis improves SGMs, resolving memorization issues.
problem Improving performance and avoiding memorization in SGMs.
method Formulated SGMs using Wasserstein proximal operators and mean-field games.
result Improved SGM performance in terms of training samples and time.
Enhances learning of structured distributions using nonlinear denoising score matching.
problem Learning structured distributions from noisy data.
method Latent Nonlinear Denoising Score Matching (LNDSM) integrating nonlinear dynamics with VAE-based latent score matching.
result LNDSM achieves superior sample quality and variability compared to structure-agnostic methods.
Improves SGM convergence bounds in W2-distance without strict assumptions.
problem Convergence bounds for SGMs in W2-distance require stringent assumptions.
method Novel framework using the OU process and PDE analysis.
result Log-concavity evolves from weak to strong over time.
This work improves SGMs' convergence guarantees for semiconvex distributions with discontinuous gradients.
problem Establishing convergence guarantees for SGMs under weak regularity conditions.
method Developed non-asymptotic Wasserstein-2 convergence analysis for SGMs targeting semiconvex distributions with discontinuous gradients.
result Achieved optimal dependence of O ( d ) O(\sqrt{d}) O ( d ) on data dimension d d d and convergence rate of order one. Wavelet SGM accelerates generative modeling with linear time complexity.
problem High computational cost in SGMs.
method Factorizing data distribution into wavelet coefficients.
result WSGM synthesizes wavelet coefficients with linear time complexity.
Abstract: Generalizes SGMs to infinite-dimensional Hilbertian setting.
problem Difficulties in extending SGMs to infinite-dimensional settings.
method Uses Gamma and Malliavin Calculus, Dirichlet forms, Wiener chaoses, and time-reversal formula.
result Generalized SGMs to Hilbertian setting with finite-dimensional entropic convergence bounds.
MCGDiff uses SGM to guide SMC for solving ill-posed linear inverse problems.
problem Solving ill-posed linear inverse problems in Bayesian settings.
method Exploiting SGM structure, defining a sequence of intermediate problems, and using SMC methods.
result MCGDiff outperforms competing methods in Bayesian ill-posed inverse problems.
SGMs are robust to practical errors via uncertainty quantification.
problem Robustness of SGMs to practical implementation errors.
method Wasserstein uncertainty propagation (WUP) theorem and Bernstein estimates.
result SGMs are provably robust to multiple sources of error.
MFGs explain and enhance generative models, revealing new model types.
problem Understanding and improving generative models.
method Mean-field games (MFGs) as a framework to explain and enhance generative models.
result Established connections between MFGs and generative flows, diffusions, and gradient flows.
Paper analyzes SGMs for learning sub-Gaussian distributions without dimensionality constraints.
problem Learning sub-Gaussian distributions in high dimensions with SGMs.
method Introduced complexity notion and proved approximation and generalization rates.
result SGMs can approximate target sub-Gaussian distributions in total variation with dimension-independent rate.
New polynomial convergence guarantees for SGM on general data distributions.
problem Efficient guarantees for multimodal and non-smooth distributions in SGM.
method Polynomial convergence guarantees for denoising diffusion models on general data distributions, with no assumptions on functional inequalities or smoothness.
result Wasserstein distance guarantees for distributions of bounded support or decaying tails, and TV guarantees for further smoothness assumptions.
Researchers establish bounds for SGMs' KL and Wasserstein divergences under various noise schedules.
problem Estimating the error between target and estimated distributions in SGMs.
method Established upper bounds for KL divergence and Wasserstein distance, incorporating target distribution properties and SGM hyperparameters.
result Optimal noise schedules identified for SGMs, improving generative quality.
This paper analyzes stability and generalization of Markov chain stochastic gradient methods.
problem Analyzing stability and generalization of Markov chain stochastic gradient methods.
method Algorithmic stability in statistical learning theory.
result Established optimal generalization bounds for both smooth and non-smooth cases.
Polynomial convergence proved for SGM, improving over previous methods.
problem Learning probability distributions from data and generating samples efficiently.
method Proved polynomial convergence for SGM using accurate score estimates.
result First polynomial convergence guarantees for SGM, independent of dimensionality.
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization error in terms of the absolute difference between the test and the training error. Our problem is to…
We show that parametric models trained by a stochastic gradient method (SGM) with few iterations have vanishing generalization error. We prove our results by arguing that SGM is algorithmically stable in the sense of Bousquet and Elisseeff. Our analysis only employs elementary tools from convex and continuous optimizat…
Skip connections make adversarial examples more transferable in ResNets.
problem Transferability of adversarial examples in ResNets with skip connections.
method Skip Gradient Method (SGM) to craft more transferable adversarial examples.
result SGM improves transferability of adversarial examples in various DNNs.
Paper analyzes neural network models for sub-Gaussian distributions, proving approximation and generalization abilities.
problem Estimating unknown distributions from i.i.d. observations using neural network models.
method Score-based neural network generative models (SGMs) with specific network architectures and stopping strategies.
result SGMs can approximate scores with high accuracy and achieve nearly optimal convergence rates under mild assumptions.
Paper extends information theory for efficient probabilistic modeling.
problem Efficient and data-efficient non-parametric density estimation.
method Structured generative model (SGM) using Rényi's information.
result SGM improves mutual information estimation and generative adversarial networks.
AIS uses a suboptimal extended target distribution, which this paper improves using SGM.
problem Improving the efficiency of Annealed Importance Sampling for marginal likelihood estimation.
method Leveraging score-based generative modeling to approximate the optimal extended target distribution.
result Demonstrated novel, differentiable AIS procedures on synthetic and real-world data.
Paper establishes a density formula for diffusion models, linking target density to score function.
problem Lack of theoretical foundation for optimizing DDPMs using ELBO.
method Developed a density formula for continuous-time diffusion processes, revealing the connection between target density and score function.
result The minimizer of the ELBO objective for DDPMs nearly coincides with the true objective, providing a theoretical foundation.
SURGIN uses generative models to infer subsurface flow data efficiently.
problem Inefficient and task-specific inversion methods for subsurface multiphase flow.
method SURGIN integrates U-FNO surrogate with SGM for zero-shot conditional generation.
result Decent inference of heterogeneous geological fields and flow dynamics with uncertainty quantification.
We study the generalization properties of stochastic gradient methods for learning with convex loss functions and linearly parameterized functions. We show that, in the absence of penalizations or constraints, the stability and approximation properties of the algorithm can be controlled by tuning either the step-size o…
New framework trains Schrödinger Bridge models using SDEs for generative tasks.
problem Unclear relation between SB optimization and modern generative model training.
method Forward-Backward SDEs theory for likelihood training of SB models.
result Training algorithm achieves comparable results on image generation datasets.
Paper analyzes Langevin dynamics for solving infinite-dimensional Bayesian inverse problems.
problem Solving high-dimensional Bayesian inverse problems in infinite-dimensional function spaces.
method Preconditioned Langevin dynamics with score-based generative models (SGMs).
result Derives error estimates and sufficient conditions for global convergence in Kullback-Leibler divergence.
Unified framework for generating heavy-tailed distributions.
problem Extending SGMs to heavy-tailed targets.
method Combining early stopping with initialization for diffusion, and normalizing flows for generation.
result Unified generative framework with theoretical guarantees for heavy-tailed distributions.
This paper improves SGMs by using a predictor-corrector scheme to converge faster.
problem Theoretical and practical limitations of existing SGMs when T 1 o ∞ T_1 o \infty T 1 o ∞ . method Integrates a predictor-corrector scheme after the forward process to converge in finite time.
result Convergence guarantees for SGMs require only a fixed finite time T 1 T_1 T 1 . This work analyzes SGGMs, offering convergence insights and practical design tips.
problem Theoretical convergence analysis for SGGMs with a system of coupled SDEs.
method Non-asymptotic convergence analysis for three graph generation paradigms.
result Unique factors affecting convergence in SGGMs and practical hyperparameter selection.
Dynamic topic model improves mental health note analysis for children.
problem Lack of longitudinal topic models for psychiatric clinical notes.
method Developed a dynamic topic model with consistent topics and individualized temporal dependencies.
result Achieved a 38% increase in topic coherence.
Paper bridges score estimation to parameter and density estimation in DDPMs.
problem Efficiently estimating scores for generative models.
method Introduces a framework linking score estimation to parameter and density estimation.
result Denoising score-matching in DDPMs is asymptotically efficient for parameter estimation.
The paper tackles singularities in diffusion models on submanifolds.
problem Analyzing singularities in diffusion models on lower-dimensional submanifolds.
method Small-time approximations of the Green's function and derivation of a new target function.
result The new target function remains bounded for singular data distributions.
Improved sampling in generative models using CLDs with a hyperparameter.
problem Improving sampling performance in generative models.
method Extending Critically-damped Langevin Diffusions with a hyperparameter to control noise.
result Derivation of a novel upper bound on Wasserstein sampling error.
DualVDT improves time-series forecasting with a novel dual reparametrized structure.
problem Time-series forecasting with improved performance and analytical rigor.
method Dual reparametrized variational mechanisms on VAE, latent score based generative model, reverse time stochastic differential equation, variational ancestral sampling, KL divergence reduction.
result Advanced performance in time-series forecasting with reduced KL divergence.