GADD accelerates uniform-rate discrete diffusion models by 2 orders of magnitude.
problem Slow sampling in uniform-rate discrete diffusion models.
method Gibbs-based corrector (GADD) that constructs Gibbs posterior likelihoods directly from the concrete score function.
result Achieves an overall sampling complexity of O(polylog(ε−1)). UDM reparameterization improves language model generation.
problem Mismatch between UDM training objective and denoising posterior.
method Leave-one-out denoising and absorbing state reformulation.
result Improved UDM generation through leave-one-out parameterization.
Discrete diffusion models improve data generation for discrete data like language and graphs.
problem Adapting diffusion models to discrete state spaces for better data generation.
method Formulated as CTMCs, used uniformization of continuous Markov chains for sampling.
result Derive guarantees for sampling from any distribution on a hypercube, aligning with state-of-the-art achievements.
Uniform diffusion approximation for SGD in non-convex settings.
problem Finite-time diffusion approximation for SGD.
method Establishing uniform-in-time diffusion approximation with strong convexity and mild conditions.
result Uniform-in-time diffusion approximation of SGD without convexity of each loss function.
Adaptive sampling improves graph diffusion models by maintaining uniform information speed.
problem Standard diffusion models overlook non-homogeneous dynamics on complex manifolds.
method Information-geometric framework using Fisher-Rao metric and Drift Variation Score (DVS).
result DVS solver ensures uniform rate of distributional change, improving structural fidelity and efficiency.
Unified framework for discrete diffusion modeling with flexible noising processes.
problem Efficient modeling of large discrete state spaces with arbitrary corruption dynamics.
method Generalized Discrete Diffusion from Snapshots (GDDS) framework that supports uniformization for fast noising and snapshot-based ELBO for reverse process.
result GDDS outperforms existing discrete diffusion methods in training efficiency and generation quality.
Simulates financial market orders using anomalous diffusion models.
problem Anomalous diffusion in financial market order dynamics.
method Discrete Time Random Walk with Sibuya waiting times, non-uniform sampling, and cubic spline interpolation.
result Demonstrates price impact for different forcing functions and model parameters.
Paper tackles estimating initial conditions of spatio-temporal processes from sparse data.
problem Estimating initial conditions of spatio-temporal advection-diffusion processes from sparse data.
method Regularized convex optimization problem with Alternating Direction Method of Multipliers.
result Efficient solutions for non-uniform and shifted uniform sampling schemes.
This letter presents an improved version of diffusion least mean ppower (LMP) algorithm for distributed estimation. Instead of sum of mean square errors, a weighted sum of mean square error is defined as the cost function for global and local cost functions of a network of sensors. The weight coefficients are updated b…
Starting from a sequence of independent Wright-Fisher diffusion processes on [0,1], we construct a class of reversible infinite dimensional diffusion processes on $\DD_\infty:= \{{\bf x}\in Let $MbeacompleteRiemnnianmanifoldandμthedistributionofthediffusionprocessgeneratedby\ff 1 2\DD+ZwhereZ$…
Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the …
The paper analyzes sampling efficiency of discrete diffusion models, providing sharp and adaptive guarantees.
problem Theoretical foundations of discrete diffusion models, especially sampling efficiency.
method Continuous-time Markov chain (CTMC) formulation, τ-leaping-based samplers, effective total correlation. result The τ-leaping algorithm achieves an iteration complexity of order ildeO(d/ε) for uniform discrete diffusion, improving existing bounds by a factor of d. Remasking improves the quality of discrete diffusion models for natural language and image generation.
problem Limited iterative refinement in masked discrete diffusion models.
method Introducing ReMDM sampler that allows remasking during inference.
result Remasking enables better quality outputs with increased sampling steps.
FSD-CAP improves graph feature imputation under high missing rates.
problem Challenges in imputing missing node features in graphs, especially under high missing rates.
method Two-stage framework: subgraph expansion, fractional diffusion, class-aware propagation.
result Significantly improved imputation quality compared to existing methods, achieving high accuracy on benchmark datasets.
Improved error estimate for SGLD sampling algorithm.
problem Establishing a precise error bound for SGLD.
method Sharp uniform-in-time error estimate for SGLD under mild assumptions.
result Uniform-in-time O(η2) bound for KL-divergence between SGLD and Langevin diffusion. Diffusion models tackle noisy inverse problems with posterior sampling.
problem Efficiently solving general noisy inverse problems.
method Approximation of posterior sampling for diffusion models.
result Diffusion models can handle various noise statistics and nonlinear problems.
Improved sampling for diffusion models and log-concave distributions.
problem Efficient sampling for diffusion models and log-concave distributions.
method Algorithms for sampling with δ-error in polylog(1/δ) steps using accurate score estimates. result Exponential improvement in complexity over previous results.
Develops a framework for distilling flow models from few steps.
problem Improving few-step sampling in diffusion models for better performance.
method Local approximation errors and dynamical amplification controlled through analytical tractability.
result Deep residual compositions efficiently approximate long-horizon transport with controlled global error.
Uniform heat kernel and diffusion bridge asymptotics for sub-Riemannian geometry.
problem Analyzing sub-Riemannian heat kernels and their derivatives on incomplete manifolds.
method Localized asymptotic analysis, focusing on minimizing geodesics and the non-abnormal cut locus.
result Uniform bounds and expansions for heat kernels and their derivatives on compacts, including the diffusion bridge measure.
Few-step distillation improves T2I models without real images or CFG trade-offs.
problem Challenges in accelerating T2I models with high-resolution and CFG.
method Score identity distillation (SiD) for few-step generation, with adversarial loss and new guidance strategies.
result State-of-the-art performance on SDXL at 1024x1024 resolution, robust to real images absence.
A discrete diffusion model learns denoising, scoring, and bridging in different coordinates.
problem Understanding what a discrete diffusion model learns in different coordinate systems.
method Rigorous derivation of continuous-time Markov chain ELBO, Oracle Distance theorem, and exact coordinates for optimizer.
result The negative ELBO is exactly equal to the data entropy plus the path KL from the oracle reverse process to the learned one.
Unified framework for convergence of discrete diffusion models without state space size dependence.
problem Fundamental limitations in existing convergence theory for discrete diffusion models, especially under singular priors and large vocabularies.
method Unified adjoint-equation-based framework that establishes dimension-free convergence guarantees in any integral probability metric (IPM).
result First dimension-free convergence bounds applicable to both masked and uniform priors, free of state space size S. Diffusion models improve sample quality with guidance, proving it works for general data distributions.
problem Theoretical understanding of guidance effect in diffusion models for general data distributions.
method Analyzing diffusion guidance under general data distributions, proving improvement in sample quality.
result Guidance improves the average reciprocal of the classifier probability, aligning with its motivation.
This primer explains diffusion models in general state spaces.
problem Diffusion models in general state spaces are not well-introduced.
method Develops discrete-time and continuous-time views of diffusion models, deriving Fokker-Planck and master equations.
result Unified understanding of diffusion models across continuous and discrete domains.
Study uniform convergence of random walk Laplacians to diffusion Laplacian on smooth manifolds.
problem Uniform convergence of random walk Laplacians to diffusion Laplacian on smooth manifolds.
method Analysis of random walks on geometric and directed kNN graphs, using concentration tools and differential geometry.
result Uniform convergence of kNN Laplacians to diffusion Laplacian, without continuity of transition kernel. This work uncovers algorithm-dependent regularisation in diffusion models.
problem Understanding and improving generalisation in high-dimensional diffusion models.
method Algorithmic stability and score stability analysis.
result Identifies multiple sources of implicit regularisation unique to diffusion models.
A coupling by reflection of a time-inhomogeneous diffusion process on a manifold are studied. The condition we assume is a natural time-inhomogeneous extension of lower Ricci curvature bounds. In particular, it includes the case of backward Ricci flow. As in time-homogeneous cases, our coupling provides a gradient esti…
We analyze the probability of ruin for the {\it scaled} classical Cramér-Lundberg (CL) risk process and the corresponding diffusion approximation. The scaling, introduced by Iglehart \cite{I1969} to the actuarial literature, amounts to multiplying the Poisson rate $\la$ by n, dividing the claim severity by $\sqrtn$, …
We propose to use Gaussian process regression to accurately estimate the diffusion MRI signal at arbitrary locations in q-space. By estimating the signal on a grid, we can do synthetic diffusion spectrum imaging: reconstructing the ensemble averaged propagator (EAP) by an inverse Fourier transform. We also propose an a…
A fundamental result in differential privacy states that the privacy guarantees of a mechanism are preserved by any post-processing of its output. In this paper we investigate under what conditions stochastic post-processing can amplify the privacy of a mechanism. By interpreting post-processing as the application of a…
QTD integrates quantization with diffusion for efficient data generation.
problem Challenges in continuous diffusion models, especially long-range transitions and biases.
method Quantized Transition Diffusion (QTD) integrates data quantization with discrete diffusion dynamics.
result QTD achieves efficient data generation with minimal score evaluations.
The validity of an approximation formula for European option prices under a general stochastic volatility model is proved in the light of the Edgeworth expansion for ergodic diffusions. The asymptotic expansion is around the Black-Scholes price and is uniform in bounded payoff func- tions. The result provides a validat…
New algorithm uniformly samples high-dimensional convex bodies efficiently.
problem Uniform sampling of high-dimensional convex bodies.
method Stochastic diffusion perspective to show contraction to the target distribution.
result Achieves state-of-the-art runtime complexity with strong guarantees on output.
We develop a new model for VIX derivatives with closed-form solutions.
problem VIX derivatives pricing and risk management.
method Data-driven Legendre polynomial model for VIX volatility, deriving analytical series solutions.
result Equal or superior accuracy compared to existing models, offering an efficient alternative.
We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion. We analyze these methods under a uniform set of assumptions on the log-posterior distribution, assuming it to be smo…
We investigate the extension of the multilevel Monte Carlo path simulation method to jump-diffusion SDEs. We consider models with finite rate activity, using a jump-adapted discretisation in which the jump times are computed and added to the standard uniform dis- cretisation times. The key component in multilevel analy…
We study the problem of identifying the source of a diffusion spreading over a regular tree. When the degree of each node is at least three, we show that it is possible to construct confidence sets for the diffusion source with size independent of the number of infected nodes. Our estimators are motivated by analogous …
Improves sampling, rounding, and integration of logconcave functions.
problem Sampling, rounding, and integration of logconcave functions.
method Algorithmic diffusion approach.
result First complexity improvements in nearly two decades for general logconcave functions.
Unified error analysis for discrete flow models.
problem Error analysis of discrete flow models.
method Stochastic calculus theory, Girsanov theorem, generator matching, uniformization.
result First error analysis for discrete flow models.
We introduce a novel paradigm for learning non-parametric drift and diffusion functions for stochastic differential equation (SDE). The proposed model learns to simulate path distributions that match observations with non-uniform time increments and arbitrary sparseness, which is in contrast with gradient matching that…
EntroPath learns manifold geometry from diffusion paths.
problem Learning geodesic geometry from data graphs with spurious shortcuts.
method Maximum Entropy Path Ensemble Embedding (MERW) with k-step diffusion paths.
result EntroPath converges to squared geodesic distance in the short-time limit.
In this paper, we propose a novel approach for manifold learning that combines the Earthmover's distance (EMD) with the diffusion maps method for dimensionality reduction. We demonstrate the potential benefits of this approach for learning shape spaces of proteins and other flexible macromolecules using a simulated dat…
Spectral methods that are based on eigenvectors and eigenvalues of discrete graph Laplacians, such as Diffusion Maps and Laplacian Eigenmaps are often used for manifold learning and non-linear dimensionality reduction. It was previously shown by Belkin and Niyogi \cite{belkin_niyogi:2007} that the eigenvectors and eige…
New analysis shows scores learn data manifolds better than distributions.
problem Learning the full distribution vs. just the data manifold.
method Novel analysis of scores in the small-σ regime.
result Scores learn data manifold information Θ(σ−2) stronger than distribution information. Unified kernel framework extends to stochastic systems, improving numerical stability.
problem Extending kernel methods to stochastic dynamical systems with diffusion.
method Unified kernel framework, Feynman-Kac path-integral representations, collocation-based computational framework.
result Kernel equivalence under uniform ellipticity assumptions and improved numerical stability with moderate diffusion.
Unified framework for robust, stable, and efficient density ratio estimation.
problem Density-chasm and support-chasm problems in density ratio estimation.
method Dequantified diffusion-Schrödinger bridge (D3RE) framework with DDBI and DSBI.
result Offers uniform approximation and bounded time scores in theory and empirical performance.
Extends Langevin dynamics for constrained domains.
problem Optimization of constrained probability measures.
method Mirror mean-field Langevin dynamics (MMFLD).
result Linear convergence guarantees and propagation of chaos results.
The use of machine learning algorithms to address classification problems is on the rise in many research areas. The current study is aimed at testing the potential of using such algorithms to auto-select the best solvers for transport problems in uniform slabs. Three solvers are used in this work: Richardson, diffusio…