Study on kinetic Langevin diffusions and their couplings, showing subtle TV bounds and new non-Markovian couplings.
problem Understanding and quantifying the TV distance between solutions of kinetic Langevin diffusions with different initial values.
method Established new non-Markovian couplings for kinetic Langevin diffusions, derived from optimal coalescence trajectories, and analyzed their TV bounds.
result No Markovian coupling can capture the asymptotic decay rate of the TV distance between solutions of kinetic Langevin diffusions with different initial values.
Proposes a new method for constrained generative modeling using Langevin dynamics.
problem Challenges in satisfying underlying constraints with score-based generative models.
method Uses kinetic Langevin dynamics with specular reflection to model constraints.
result Demonstrates efficient numerical samplers with optimal convergence rates.
KIPLMC methods improve statistical inference in latent variable models.
problem Statistical inference in latent variable models.
method Joint diffusion process in parameter and latent variable spaces, with two explicit discretizations.
result KIPLMC methods achieve accelerated convergence rates in Wasserstein-2 distance.
The paper develops methods for sampling from log-concave distributions with constraints.
problem Sampling from log-concave distributions with constraints.
method Randomized midpoint discretization of Langevin diffusions with various projections.
result New convergence guarantees for constrained Langevin algorithms.
Improved KLMC for sampling under various conditions.
problem Stable simulation of kinetic Langevin dynamics under different parameters.
method Revisited synchronous Wasserstein coupling analysis with stochastic exponential Euler discretization.
result Exponential integrator can simulate kinetic Langevin dynamics in the overdamped regime with proper time acceleration.
New algorithms improve sampling from complex distributions.
problem Sampling from complex probability distributions efficiently.
method Regime-switching Langevin dynamics and Monte Carlo algorithms.
result Convergence guarantees and iteration complexities provided.
Paper proposes new Langevin samplers for sampling from log-concave distributions with superlinear gradient growth.
problem Sampling from log-concave distributions with superlinear gradient growth.
method Proposes two novel discretizations of kinetic Langevin SDEs, showing contractivity and log-Sobolev inequality.
result Establishes non-asymptotic bounds in 2-Wasserstein distance between sampled distributions and target measures.
New sampling method on Lie groups converges quickly.
problem Sampling on non-Euclidean Lie groups.
method Kinetic Langevin dynamics with noise added.
result Exponential convergence rate proved under W2 distance. This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in W2 distance for log-strongly-concave targets. New algorithms improve sampling from constrained distributions.
problem Generating samples from distributions under constraints.
method Kinetic Langevin dynamics and splitting schemes.
result Improved complexity bounds over existing methods.
New methods improve efficiency of sampling algorithms for complex systems.
problem Efficiently sampling from complex, high-dimensional probability distributions.
method Randomized Runge-Kutta-Nyström methods tailored for Hamiltonian flows.
result Quantitative 5/2-order L2-accuracy in approximating Hamiltonian flows. Linear stochastic models and discretized kinetic theory are two complementary analytical techniques used for the investigation of complex systems of economic interactions. The former employ Langevin equations, with an emphasis on stock trade; the latter is based on systems of ordinary differential equations and is bett…
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
Error estimates found between SGD with momentum and Langevin diffusion.
problem Quantifying the difference between SGD with momentum and Langevin diffusion.
method Established error estimates using 1-Wasserstein and total variation distances.
result Quantitative error estimates between SGD with momentum and underdamped Langevin diffusion.
New method improves sampling efficiency in complex stochastic systems.
problem Sampling efficiency in nonconvex stochastic gradient cases.
method Reflection coupling for unadjusted generalized Hamiltonian Monte Carlo.
result Quantitative Gaussian concentration bounds and convergence rates established.
Study of diffusion annealed Langevin dynamics for generative models.
problem Theoretical efficiency of score-based diffusion processes.
method Rigorous construction and analysis of diffusion processes with Poincaré and logarithmic Sobolev inequalities.
result Improvement in efficiency of diffusion processes through Poincaré and logarithmic Sobolev inequalities.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
We derive a diffusion approximation for the kinetic Vlasov-Fokker-Planck equation in bounded spatial domains with specular reflection type boundary conditions. The method of proof involves the construction of a particular class of test functions to be chosen in the weak formulation of the kinetic model. This involves t…
First order discretizations of Langevin diffusion can achieve better generalization error with additional smoothness assumptions.
problem Analyzing generalization error for first order discretizations of Langevin diffusion.
method Providing a sufficient smoothness condition to show that first order methods can achieve arbitrarily runtime complexity for a given expected generalization error.
result First order methods can achieve arbitrarily runtime complexity with additional smoothness assumptions.
Physics analogies explain machine learning overfitting control.
problem Understanding and controlling overfitting in machine learning.
method Analogies from physics and biology applied to algorithmic stability and GAN models.
result Physics formulas provide insights into reducing overfitting in machine learning.
We introduce and analyze a linear kinetic model that describes the evolution of the probability density of the number of firms in a society, in which the microscopic rate of change obeys to the so-called law of proportional effect proposed by Gibrat. Despite its apparent simplicity, the possible mean field limits of th…
SGLDiff approximates Bayesian posterior distributions with subsampling error.
problem Approximating Bayesian posterior distributions in large-scale data settings.
method Stochastic Gradient Langevin Diffusion (SGLDiff) with subsampling.
result The Wasserstein distance between the posterior and SGLDiff's limiting distribution is bounded by a fractional power of the mean waiting time.
Unified approach for sampling non-differentiable and heavy-tailed targets.
problem Sampling non-differentiable and heavy-tailed distributions using Langevin algorithms.
method Anchored Langevin dynamics, which modifies the Langevin diffusion with a smooth reference potential and multiplicative scaling.
result Non-asymptotic guarantees in the 2-Wasserstein distance to the target distribution.
Paper defends diffusion models from membership inference attacks using Langevin dynamics.
problem Defending diffusion models against membership inference attacks.
method Uses critically-damped higher-order Langevin dynamics with auxiliary variables.
result Demonstrates improved resistance to membership inference attacks through theoretical investigation and validation.
NSGLD improves SGLD for non-convex optimization problems.
problem Optimizing non-convex objectives efficiently.
method Introducing non-reversible SGLD by adding an anti-symmetric matrix to the drift term of the Langevin diffusion.
result NSGLD converges faster to the same stationary distribution with non-asymptotic guarantees.
Improved Langevin algorithms with prior diffusion achieve dimension-independent convergence for non-log-concave distributions.
problem Understanding the dimension dependency of computational complexity in high-dimensional sampling.
method Investigation of prior diffusion technique for log-Sobolev inequality target distributions.
result Modified Langevin algorithm achieves dimension-independent KL divergence convergence.
Poisson Midpoint Method improves Langevin Dynamics for diffusion models.
problem Slow convergence of LMC in diffusion models requiring many small steps.
method Poisson Midpoint Method approximates LMC with larger steps, proving quadratic speed up.
result Poisson Midpoint Method maintains quality of DDPM with fewer calls.
Combining diffusion models with Langevin dynamics improves posterior sampling efficiency.
problem Sampling from noisy posterior distributions efficiently.
method Annealed Langevin dynamics combined with diffusion models.
result Achieves posterior sampling in polynomial time with a weaker score error bound.
A new diffusion method approximates Schrödinger bridge with improved convergence.
problem Approximating Schrödinger bridge with Langevin diffusion.
method Leveraging Langevin diffusion to approximate Schrödinger bridge.
result The difference between the two approximations is proportional to the score function.
Langevin diffusion is a commonly used tool for sampling from a given distribution. In this work, we establish that when the target density p∗ is such that logp∗ is L smooth and m strongly convex, discrete Langevin diffusion produces a distribution p with KL(p∣∣p∗)≤ε in O~(εd) steps,…
TOLD++ improves convergence of diffusion models by critically damping the forward transition matrix.
problem Improving the convergence of Denoising Diffusion Probabilistic Models.
method Critically damping the Third-Order Langevin Dynamics (TOLD) forward transition matrix using eigen-analysis.
result TOLD++ converges faster than TOLD, verified on toy and real datasets.
The paper analyzes sampling and estimation on manifolds using Langevin diffusion.
problem Sampling and estimation on compact Riemannian manifolds.
method Discretization of Langevin diffusion with error bounds derived.
result First-order error bounds for bias and variance in estimators.
A new algorithm solves semidefinite programs using Langevin diffusion.
problem Optimizing semidefinite programs with diagonal constraints.
method Langevin diffusion on a product manifold of spheres.
result Langevin algorithm achieves ε accuracy in Ω(ε^-5) iterations.
Discretizations of Langevin diffusions provide a powerful method for sampling and Bayesian inference. However, such discretizations require evaluation of the gradient of the potential function. In several real-world scenarios, obtaining gradient evaluations might either be computationally expensive, or simply impossibl…
We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion. We analyze these methods under a uniform set of assumptions on the log-posterior distribution, assuming it to be smo…
DMPS uses diffusion maps and LAWGD for efficient generative modeling.
problem Efficiently modeling complex data distributions.
method Diffusion maps for manifold learning and LAWGD for sampling.
result DMPS outperforms other methods on moderate-dimensional data.
The paper analyzes the randomized midpoint method for Langevin diffusions, revealing biases and asymptotic properties.
problem Analyzing biases and asymptotic properties of the randomized midpoint method for Langevin diffusions.
method Characterization of stationary distribution and asymptotic normality for numerical integration.
result The step-size needs to go to zero for the method to be asymptotically unbiased.
Recent technological development has enabled researchers to study social phenomena scientifically in detail and financial markets has particularly attracted physicists since the Brownian motion has played the key role as in physics. In our previous report (arXiv:1703.06739; to appear in Phys. Rev. Lett.), we have prese…
Langevin dynamics fails to produce accurate samples even with small score function errors.
problem Robustness of Langevin dynamics to score function errors.
method Analysis of Langevin dynamics and score function errors.
result Langevin dynamics produces a distribution far from the target distribution in TV distance even with small L2 errors in the score function. Adaptive algorithm improves convergence rate of Langevin dynamics.
problem Improving convergence rate of Langevin dynamics.
method Adaptive non-reversible stochastic gradient Langevin dynamics algorithm.
result Improved convergence rate of the algorithm.
Optimal preconditioning improves Langevin sampling efficiency.
problem Improving sampling efficiency in high-dimensional target distributions.
method Optimal preconditioning using Fisher information, applied to MALA.
result Adaptive MCMC scheme significantly outperforms other methods.
Algorithm estimates graph structure with prior information and Langevin diffusion.
problem Support estimation of partially known Gaussian graphical models.
method Proposes an algorithm using annealed Langevin diffusion and graph neural networks to estimate the posterior distribution of the graph.
result Demonstrates the benefits of the approach through numerical experiments.
Improved generative models using critically-damped Langevin diffusion.
problem Current score-based generative models (SGMs) use overly simplistic diffusion processes, leading to complex denoising tasks and suboptimal performance.
method Proposed a novel critically-damped Langevin diffusion (CLD) and derived a score matching objective and sampling scheme.
result CLD-based SGMs achieve superior performance in synthesis quality compared to previous methods.
Accelerates convergence in global non-convex optimization with reversible diffusion.
problem Global non-convex optimization challenges.
method Utilizes reversible diffusion processes with adaptive diffusion coefficients.
result Accelerated convergence with reduced discretization error.
New method improves sampling from score-based models by correcting bias.
problem Bias in sampling from score-based diffusion models.
method Metropolis-Hastings or Barker's accept-reject steps to correct bias, using the score function.
result Improves sample quality on synthetic and image datasets, yielding consistent gains in FID.
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
An Euler discretization of the Langevin diffusion is known to converge to the global minimizers of certain convex and non-convex optimization problems. We show that this property holds for any suitably smooth diffusion and that different diffusions are suitable for optimizing different classes of convex and non-convex …