This work extends Tweedie's formulae to non-Gaussian processes for better diffusion model generation.
problem Limited exploration of non-Gaussian diffusion models and corresponding Tweedie's formulae.
method Extended Tweedie's formulae to geometric Brownian motion, squared Bessel, and Cox-Ingersoll-Ross processes.
result Demonstrated potential of non-Gaussian models in image and financial time series generation.
New blurring diffusion models bridge heat dissipation and denoising.
problem Developing a new generative modeling approach.
method Connecting blurring to Gaussian diffusion with non-isotropic noise.
result Proposed Blurring Diffusion Models offer the best of both Gaussian denoising and inverse heat dissipation.
WS diffusion models handle anisotropic Gaussian noise better than conventional methods.
problem Handling anisotropic Gaussian noise in imaging inverse problems.
method Whitened Score (WS) diffusion models based on stochastic differential equations.
result WS DMs outperform conventional DMs on anisotropic Gaussian noise.
Diffusion models generate data with Gaussian Universality, matching linear model test errors.
problem Analyzing the performance of models trained on synthetic data generated by diffusion models.
method Investigates Gaussian Universality for data distributions generated via diffusion models, matching test errors of linear models trained on synthetic data to Gaussian Mixture models.
result The test error of a linear model trained on diffusion-generated data matches the test error of a linear model trained on Gaussian Mixture data with matching means and covariances per class.
Proposes diffusion models using mixed Gaussian priors for better data representation.
problem Improving data representation in diffusion models.
method Structured diffusion models with a mixture of Gaussians as prior.
result Improved model performance compared to classical diffusion models.
Conditional diffusion models can approximate target distributions well with Gaussian-mixture reverse kernels.
problem Approximating target distributions in conditional diffusion models.
method Using finite Gaussian mixtures with ReLU-network logits as reverse kernels, reducing the problem to static conditional density approximation.
result The resulting neural reverse-kernel class is dense in conditional KL divergence under exact terminal matching.
New method uses diffusion models for Bayesian inverse problems.
problem Solving Bayesian inverse problems with linear-Gaussian models.
method Decoupled Diffusion Sequential Monte Carlo (DDSMC) method.
result Asymptotically exact solution demonstrated on various data types.
New method for fast inference in diffusion models.
problem Intractable probabilistic inference in diffusion models.
method Variational Gaussian Process, exponential family description, convex optimization.
result Improved fast algorithm for learning model parameters.
Algorithm learns mixtures of Gaussians efficiently using diffusion models.
problem Learning mixtures of Gaussians with identity covariance.
method Analytic approach using diffusion models to learn score functions.
result Quasi-polynomial time and sample complexity for learning mixtures.
Introduces Star-Shaped DDPMs for non-Gaussian distributions.
problem Difficulties in defining DDPMs for non-Gaussian distributions.
method Star-shaped diffusion process, duality with specific Markovian diffusions, efficient algorithms.
result SS-DDPMs can model distributions like Beta, von Mises-Fisher, Dirichlet, Wishart.
Diffusion models achieve high-quality samples from complex high-dimensional Gaussian mixtures without scaling with dimension.
problem Achieving accurate sampling from high-dimensional distributions using diffusion models.
method Investigates the effectiveness of diffusion models in sampling from Gaussian Mixture Models (GMMs) without scaling with dimension.
result DDPM requires at most O(1/ε) iterations to attain an ε-accurate distribution in total variation distance, independent of dimension and number of components. Diffusion models optimize objectives similar to ELBO with Gaussian noise augmentation.
problem Optimizing diffusion models for high perceptual quality.
method Showed diffusion objectives are weighted ELBOs over noise levels, with Gaussian noise augmentation.
result Diffusion objectives equate to ELBO with Gaussian noise augmentation under monotonic weighting.
Improves generative models by adding jump-diffusion noise.
problem Limited performance of diffusion models in generating samples from unknown distributions.
method Generalizes diffusion processes to include jump-diffusion noise, deriving closed-form generalized score functions.
result Jump-diffusion models outperform Gaussian models in specific parameter regimes.
Proposes sampling from reverse diffusion posteriors for contextual bandits.
problem Complex distributions in contextual bandits.
method Approximate posterior sampling with a diffusion model prior using Laplace approximation.
result Empirically consistent and efficient approximations for contextual bandits.
Automated denoising score matching handles nonlinear diffusion processes.
problem Nonlinear diffusion processes limit generative modeling and property estimation.
method Local-DSM using local increments and Taylor expansions.
result Tractable training and score estimation for nonlinear diffusion processes.
The paper analyzes how guidance affects diffusion models using Gaussian mixture models.
problem Understanding how guidance influences diffusion models in specific contexts.
method Theoretical study using Gaussian mixture models and comparison inequalities for differential equations.
result Guidance boosts classification confidence but reduces distribution diversity, leading to lower differential entropy.
DDS samples from noisy data by reversing diffusion, providing theoretical guarantees.
problem Sampling from unnormalized densities.
method Denoising diffusion process, score matching, optimal control, Schrödinger bridges.
result DDS provides theoretical guarantees for sampling.
We propose to use Gaussian process regression to accurately estimate the diffusion MRI signal at arbitrary locations in q-space. By estimating the signal on a grid, we can do synthetic diffusion spectrum imaging: reconstructing the ensemble averaged propagator (EAP) by an inverse Fourier transform. We also propose an a…
This paper extends neural network approximation results to denoising diffusion models.
problem Improving the efficiency and accuracy of generative models.
method Leveraging connections to stochastic control and neural network approximation.
result Established neural network approximation results for the Föllmer drift are extended to denoising diffusion models.
We analyze the Standard & Poor's 500 stock market index from the last 22 years. The probability density function of price returns exhibits two well-distinguished regimes with self-similar structure: the first one displays strong super-diffusion together with short-time correlations, and the second one corresponds to we…
End-to-end learnable Gaussian mixture priors improve diffusion models' exploration and expressiveness.
problem Challenges in diffusion models when priors differ from target distributions.
method End-to-end learnable Gaussian mixture priors (GMPs) with iterative refinement.
result Significant performance improvements across various benchmark problems.
Diffusion Transformer captures spatial-temporal dependencies in sequential data.
problem Capturing rich spatial and temporal dependencies in sequential data.
method Established theoretical guarantees for diffusion transformers learning Gaussian process data.
result Spatial-temporal dependencies are captured within attention layers of diffusion transformers.
PoGDiff improves diffusion models on imbalanced datasets.
problem Diffusion models struggle with imbalanced text-image pairs.
method PoGDiff replaces ground-truth with a Product of Gaussians.
result PoGDiff enhances generation accuracy and quality on imbalanced datasets.
NDPs learn to sample from complex function distributions using neural networks and diffusion models.
problem Learning rich distributions over functions with neural networks.
method NDPs use denoising diffusion models and custom attention blocks to incorporate stochastic process properties.
result NDPs can capture functional distributions close to true Bayesian posteriors and outperform neural processes.
DiffC compresses images by diffusing Gaussian noise, outperforming state-of-the-art methods.
problem Efficient lossy image compression without transform coding.
method Unconditional diffusion generative models to encode and denoise corrupted pixels.
result DiffC outperforms HiFiC on ImageNet 64x64, achieving a 3 dB gain at high bitrates.
New method converts and optimizes sampling schedules for generative models.
problem Optimizing sampling schedules for generative models like flows and diffusions.
method Unified framework for stochastic interpolants, including point mass schedules.
result Demonstrated efficient generation of images with fewer steps.
Combines deep state space models with diffusion models for better forecasting and capturing latent dynamics
problem Forecasting and capturing latent dynamics in time series
method DDSSM: Diffusion-driven state space model
result Empirically outperforms state-of-the-art deep SSM
The paper models stock returns using q-Gaussians and negative binomials.
problem Modeling stock return distributions and pricing options.
method Proposes a generalized jump-diffusion model and uses q-Gaussians and negative binomial distributions. result An explicit option pricing formula is derived.
The paper connects Riemannian Gaussian distributions to random matrix theory and diffusion kernels.
problem Analyzing Riemannian Gaussian distributions on symmetric spaces.
method Analytical computation of marginals using orthogonal and skew orthogonal polynomials, and diffusion kernels.
result Riemannian Gaussian distributions are random matrix types, and their probability density functions can be computed analytically.
New method DDVI improves posterior inference for deep Gaussian processes.
problem Inference of inducing points in DGPs is challenging and biased.
method DDVI uses denoising diffusion SDE and score matching for posterior approximation.
result Empirically shows DDVI outperforms baseline methods in inducing point inference.
CCDF reduces diffusion sampling steps for inverse problems.
problem Slow sampling from diffusion models in inverse problems.
method Starting from a single forward diffusion step with better initialization, followed by stochastic contraction.
result Significantly reduced sampling steps for state-of-the-art reconstruction.
Study non-Gaussian measures' concentration properties in metric spaces.
problem Concentration properties for non-linear Gaussian functionals with non-Gaussian tails.
method Prove generalised Transportation-Cost Inequalities (TCIs) for specific functionals.
result Extended TCIs for rough volatility and Parabolic Anderson Model.
We consider small-time asymptotics for diffusion processes conditioned by their initial and final positions, under the assumption that the diffusivity has a sub-Riemannian structure, not necessarily of constant rank. We show that, if the endpoints are joined by a unique path of minimal energy, and lie outside the sub-R…
Enhances diffusion models by preprocessing data to improve reconstruction quality.
problem Slow sampling and poor reconstruction quality in diffusion models, especially for small-scale networks.
method Applying Gaussianization preprocessing to the training data to make the target distribution more Gaussian-like.
result Improves generation quality, especially in the early stages of reconstruction with small networks.
Self-regulating annealing improves sampling from heavy-tailed datasets.
problem Sampling from heavy-tailed distributions using diffusion models.
method Proposed an SDE-based sampler with a state-dependent diffusion coefficient.
result State dependence induces a self-regulating annealing mechanism.
This work extends diffusion models to function space for better generative modeling.
problem Limited applicability of diffusion models to functional data domains.
method Introduces Denoising Diffusion Operators (DDOs) for training diffusion models in function space.
result Demonstrates accurate function-valued generation at fixed cost.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
The Perona-Malik model has been very successful at restoring images from noisy input. In this paper, we reinterpret the Perona-Malik model in the language of Gaussian scale mixtures and derive some extensions of the model. Specifically, we show that the expectation-maximization (EM) algorithm applied to Gaussian scale …
Score-based diffusion models achieve optimal error bounds under non-parametric assumptions.
problem Improving the minimax optimality of score-based diffusion models.
method Kernel-based score estimation and early stopping strategy.
result Achieves minimax optimal error bounds under sub-Gaussian and Sobolev space assumptions.
Bi-Lipschitz flows approximate a wide range of distributions.
problem Characterizing the expressivity of bi-Lipschitz normalizing flows.
method Linking score regularity to transport map bi-Lipschitzness via probability flow ODE.
result Gaussian pullbacks induced by bi-Lipschitz variance-preserving transport maps are L1-dense among all probability densities. This paper is a further extension of the method proposed in Itkin, 2014 as applied to another set of jump-diffusion models: Inverse Normal Gaussian, Hyperbolic and Meixner. To solve the corresponding PIDEs we accomplish few steps. First, a second-order operator splitting on financial processes (diffusion and jumps) is …
Gaussian prior and likelihood improve bandit learning performance.
problem Improving bandit learning with misspecified Gaussian distributions.
method An agent with a bounded information ratio interacts with a Bernoulli bandit based on a Gaussian prior and likelihood.
result The regret increase is at most linear in the square-root of the time horizon for diffuse distributions.
Solves memorization in diffusion models for manifold data.
problem Memorization effect in diffusion models for manifold data.
method Inertia update at the end of empirical diffusion simulation.
result Approximates true data distribution on a C2 manifold. Paper adapts diffusion sampler training for faster convergence and better sampling.
problem Training limitations in diffusion samplers.
method Decouples generation and destruction variances, learns both as unconstrained Gaussians.
result Training both processes leads to faster convergence and improved sampling quality.
The paper shows how to construct non-Gaussian Martingales using hyperbolic diffusion.
problem The challenge of modeling extreme financial events.
method Constructing Martingale processes with Cauchy distribution in the large volatility limit.
result Financial justification for using non-Gaussian distributions in modeling extreme events.
New diffusion models capture heavy-tailed distributions better.
problem Diffusion models struggle with rare or extreme events in heavy-tailed distributions.
method Repurposed diffusion framework using multivariate Student-t distributions, tailored perturbation kernel, and γ-divergence. result Our models generate rare and extreme events more effectively than standard diffusion models.
In deep latent Gaussian models, the latent variable is generated by a time-inhomogeneous Markov chain, where at each time step we pass the current state through a parametric nonlinear map, such as a feedforward neural net, and add a small independent Gaussian perturbation. This work considers the diffusion limit of suc…
Robust diffusion adaptive estimation algorithms based on the maximum correntropy criterion (MCC), including adaptation to combination MCC and combination to adaptation MCC, are developed to deal with the distributed estimation over network in impulsive (long-tailed) noise environments. The cost functions used in distri…