Survey on W W W -entropy formulas for heat equations and Langevin deformation on Riemannian manifolds.
problem Entropy formulas for heat equations and Langevin deformation on Riemannian manifolds.
method Proving W W W -entropy formulas for heat equations and Langevin deformation. result Proved W W W -entropy formulas for heat equations and Langevin deformation. Langevin algorithms improve training of very deep neural networks, especially for image classification.
problem Training very deep neural networks is challenging due to increased non-linearity and the risk of getting stuck in local minima.
method Comparison of Langevin and non-Langevin algorithms for training deep neural networks, introduction of Layer Langevin algorithm.
result Langevin algorithms, especially Layer Langevin, lead to significant improvements in training deep neural networks, particularly for image classification tasks.
SLMC improves sampling efficiency for high-dimensional distributions.
problem Sampling from high-dimensional distributions is computationally challenging.
method SLMC projects Langevin updates onto subsampled eigenblocks of a time-varying preconditioner.
result SLMC offers superior adaptability and computational efficiency compared to traditional methods.
Improved error bounds for Langevin MCMC with scaling.
problem Improving convergence rates of Langevin MCMC.
method Introducing scaling terms in underdamped Langevin equation and analyzing conditions for improved error bounds.
result Appropriate scaling improves error bounds in terms of condition number.
Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
New algorithms improve sampling from complex distributions.
problem Sampling from complex probability distributions efficiently.
method Regime-switching Langevin dynamics and Monte Carlo algorithms.
result Convergence guarantees and iteration complexities provided.
Underdamped Langevin MCMC achieves faster convergence than overdamped methods.
problem Improving the efficiency of MCMC algorithms for sampling from complex distributions.
method Discretization of underdamped Langevin diffusion.
result Achieves ε error in 2-Wasserstein distance in O(√d/ε) steps.
Study non-asymptotic Langevin Monte Carlo for Gibbs distributions.
problem Sampling from Gibbs distributions with dissipative potentials.
method Langevin-type algorithms based on Liptser--Shiryaev theory and Poincaré inequalities.
result Upper bound on 2-Wasserstein distance for accurate approximation.
Unified approach for sampling non-differentiable and heavy-tailed targets.
problem Sampling non-differentiable and heavy-tailed distributions using Langevin algorithms.
method Anchored Langevin dynamics, which modifies the Langevin diffusion with a smooth reference potential and multiplicative scaling.
result Non-asymptotic guarantees in the 2-Wasserstein distance to the target distribution.
Novel geometry-informed irreversible perturbation accelerates Langevin dynamics convergence.
problem Accelerating convergence of Langevin dynamics for Bayesian computation.
method Geometry-informed irreversible perturbation of Riemannian manifold Langevin dynamics.
result Improves estimation performance over irreversible perturbations that ignore geometry.
New methods use transport maps to improve Langevin dynamics for sampling.
problem Sampling high-dimensional, non-Gaussian distributions efficiently.
method Apply transport maps to accelerate Langevin dynamics convergence.
result Discretized processes converge to target distribution with non-asymptotic bounds.
In this article we develop geometric versions of the classical Langevin equation on regular submanifolds in euclidean space in an easy, natural way and combine them with a bunch of applications. The equations are formulated as Stratonovich stochastic differential equations on manifolds. The first version of the geometr…
The paper analyzes convergence of Langevin dynamics with time-dependent metrics.
problem Analyzing convergence of Langevin dynamics with time-dependent metrics.
method Formulated a modified gradient flow of the Kullback-Leibler divergence, selected a time-dependent relative Fisher information functional, and developed a time-dependent Hessian matrix condition.
result Proved convergence conditions for various Langevin dynamics.
Langevin MCMC samples efficiently from Riemannian manifolds with geometric Euler-Murayama analysis.
problem Efficient sampling from Gibbs distributions on Riemannian manifolds.
method Geometric Langevin MCMC, discretization error bound, contraction guarantee for Langevin Diffusion.
result Langevin MCMC iterates converge to the target distribution after a number of steps proportional to the inverse square of the desired accuracy.
Unified analysis of Langevin dynamics for nonconvex optimization with improved convergence rates.
problem Global convergence of Langevin dynamics based algorithms for nonconvex optimization.
method Unified framework analyzing numerical approximations to Langevin dynamics.
result Improved convergence rates for gradient Langevin dynamics and stochastic gradient Langevin dynamics.
Study of Langevin algorithm in noisy high-dimensional inference.
problem Analyzing the Langevin algorithm's performance in noisy high-dimensional inference.
method Analytic study of Langevin algorithm's performances using the spiked matrix-tensor model.
result The algorithmic threshold of the Langevin algorithm is sub-optimal compared to AMP.
Accelerated Langevin algorithm improves MCMC sampling efficiency.
problem Improving the efficiency of MCMC sampling methods.
method Formulated gradient-based MCMC as optimization on probability measures, showing underdamped Langevin performs accelerated gradient descent.
result Accelerated rates can be achieved for nonconvex functions using the Langevin algorithm.
Unified bounds for random subset generalization error and improved SGD Langevin dynamics.
problem Generalization error bounds for random subsets and stochastic gradient Langevin dynamics.
method Unified framework based on Hellström and Durisi's work, extending bounds for Langevin dynamics.
result Unified and refined bounds for generalization error in stochastic gradient Langevin dynamics.
Langevin DQN achieves deep exploration using Gaussian noise.
problem Deep exploration in reinforcement learning.
method Developed Langevin DQN, a variation of DQN with Gaussian noise.
result Langevin DQN achieves deep exploration.
Error estimates found between SGD with momentum and Langevin diffusion.
problem Quantifying the difference between SGD with momentum and Langevin diffusion.
method Established error estimates using 1-Wasserstein and total variation distances.
result Quantitative error estimates between SGD with momentum and underdamped Langevin diffusion.
Paper proposes new Langevin samplers for sampling from log-concave distributions with superlinear gradient growth.
problem Sampling from log-concave distributions with superlinear gradient growth.
method Proposes two novel discretizations of kinetic Langevin SDEs, showing contractivity and log-Sobolev inequality.
result Establishes non-asymptotic bounds in 2-Wasserstein distance between sampled distributions and target measures.
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
Optimal preconditioning improves Langevin sampling efficiency.
problem Improving sampling efficiency in high-dimensional target distributions.
method Optimal preconditioning using Fisher information, applied to MALA.
result Adaptive MCMC scheme significantly outperforms other methods.
First order discretizations of Langevin diffusion can achieve better generalization error with additional smoothness assumptions.
problem Analyzing generalization error for first order discretizations of Langevin diffusion.
method Providing a sufficient smoothness condition to show that first order methods can achieve arbitrarily runtime complexity for a given expected generalization error.
result First order methods can achieve arbitrarily runtime complexity with additional smoothness assumptions.
The paper shows how Langevin diffusion converges to target distributions in KL-divergence.
problem Sampling from complex distributions efficiently.
method Analysis of Langevin diffusion as a gradient flow in the space of probability distributions.
result Langevin diffusion converges to target distributions in KL-divergence with a rate of $O( rac{d}{ε})$ steps.
Extends Langevin dynamics for constrained domains.
problem Optimization of constrained probability measures.
method Mirror mean-field Langevin dynamics (MMFLD).
result Linear convergence guarantees and propagation of chaos results.
SGLB boosts machine learning with Langevin diffusion for multimodal loss functions.
problem Dealing with multimodal loss functions in machine learning.
method Stochastic Gradient Langevin Boosting (SGLB) based on Langevin diffusion equation.
result SGLB guarantees global convergence for multimodal loss functions.
Langevin Dynamics speeds up mixing time with manifold hypothesis and multi-scale approach.
problem Langevin Dynamics struggles in high dimensions and nonconvex landscapes.
method Utilizes manifold hypothesis to reduce mixing time and employs multi-scale approach to improve image generation quality.
result Mixing time depends on intrinsic dimension rather than ambient dimension, significantly reducing computational complexity.
New algorithm improves sampling from constrained spaces.
problem Sampling from constrained spaces efficiently.
method Metropolis-adjusted Mirror Langevin algorithm.
result Unbiased sampling with improved mixing time.
NSGLD improves SGLD for non-convex optimization problems.
problem Optimizing non-convex objectives efficiently.
method Introducing non-reversible SGLD by adding an anti-symmetric matrix to the drift term of the Langevin diffusion.
result NSGLD converges faster to the same stationary distribution with non-asymptotic guarantees.
The Langevin Algorithm's stationary distribution is shown to be sub-exponential or sub-Gaussian under certain conditions.
problem Understanding the properties of the Langevin Algorithm's stationary distribution.
method Analysis using a rotation-invariant moment generating function (Bessel function) to study the stationary dynamics of the Langevin Algorithm.
result Concentration results for the Langevin Algorithm's stationary distribution π η π_η π η are established, showing it is sub-exponential or sub-Gaussian under convex or strongly convex potential conditions. Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
New Langevin algorithms improve sampling efficiency in high dimensions.
problem Sampling from log-concave and smooth distributions in high dimensions.
method Combining splitting and accurate integration methods for P P P -th order Langevin dynamics. result LMC algorithms converge faster with better dimension dependence as P P P increases. New analysis of Langevin Monte Carlo via convex optimization.
problem Sampling from logconcave smooth and non-smooth target distributions.
method Formulation as a convex optimization problem, analysis using convex optimization techniques.
result Non-asymptotic analysis of Unadjusted Langevin Algorithm and new sampling methods.
New method controls bias in unadjusted Hamiltonian Monte Carlo and underdamped Langevin.
problem Bias in unadjusted Hamiltonian Monte Carlo and underdamped Langevin samplers.
method Delocalization of bias technique applied to these samplers.
result Control W 2 W_2 W 2 bias with O ( K ) O(\sqrt{K}) O ( K ) integration steps for high-dimensional distributions. Study shows uniform-time chaos propagation in mean field Langevin dynamics.
problem Understanding the convergence of marginal distributions in mean field dynamics.
method Assumed functional convexity of energy, used L p L^p L p -convergence and Wasserstein metrics. result Uniform-in-time propagation of chaos proved in both L 2 L^2 L 2 -Wasserstein and relative entropy. A new method for sampling from complex distributions using Langevin samplers.
problem Sampling from unnormalized Boltzmann densities.
method Probability flow ODE derived from linear stochastic interpolants, employing Langevin samplers.
result Efficient simulation of the flow with non-asymptotic convergence rate.
Study Langevin Monte Carlo for sampling non-log-concave distributions.
problem Sampling from non-log-concave distributions, especially Gaussian mixtures.
method Discretizations of overdamped Langevin diffusions.
result Numerical simulations compare Langevin Monte Carlo algorithms' performance.
Adaptive algorithm improves convergence rate of Langevin dynamics.
problem Improving convergence rate of Langevin dynamics.
method Adaptive non-reversible stochastic gradient Langevin dynamics algorithm.
result Improved convergence rate of the algorithm.
Improved Langevin algorithms with prior diffusion achieve dimension-independent convergence for non-log-concave distributions.
problem Understanding the dimension dependency of computational complexity in high-dimensional sampling.
method Investigation of prior diffusion technique for log-Sobolev inequality target distributions.
result Modified Langevin algorithm achieves dimension-independent KL divergence convergence.
Study on Langevin dynamics convergence rates and their application to GAN training.
problem Understanding the long-term behavior of Langevin dynamics equations.
method Analytical and numerical methods to study convergence rates of underdamped mean-field Langevin dynamics.
result Exponential convergence rate results for the Langevin dynamics under various conditions.
Adaptive Langevin dynamics reduces bias in Bayesian inference with mini-batching.
problem Bias in posterior sampling due to mini-batching in Bayesian inference.
method Adaptive Langevin dynamics with dynamical friction to correct noise.
result Quantified bias in posterior distribution due to mini-batching.
Langevin autoencoders improve deep latent variable models with efficient posterior sampling.
problem Efficient posterior sampling in deep latent variable models using MCMC.
method Amortized Langevin dynamics (ALD) replaces datapoint-wise sampling with encoder updates.
result ALD is valid as an MCMC algorithm with the target posterior as a stationary distribution.
Langevin dynamics fails to produce accurate samples even with small score function errors.
problem Robustness of Langevin dynamics to score function errors.
method Analysis of Langevin dynamics and score function errors.
result Langevin dynamics produces a distribution far from the target distribution in TV distance even with small L 2 L^2 L 2 errors in the score function. Study on convergence of Langevin dynamics for zero-sum games in probability distributions.
problem Analyzing convergence of Langevin dynamics for zero-sum games in probability distributions.
method Proved exponential and biased convergence guarantees for mean-field and finite-particle min-max Langevin dynamics.
result Explicit iteration complexity for finite-particle algorithms to approximate equilibrium distributions.
A new algorithm SLA reduces bias in sampling from measures using Langevin dynamics.
problem Reducing bias in sampling from complex target measures using Langevin dynamics.
method Proposed symmetrized Langevin algorithm (SLA) to correct bias in ULA.
result SLA is consistent for Gaussian target measures, while ULA is not.
A new sampling algorithm speeds up Langevin sampling for multimodal distributions.
problem Efficient sampling from multimodal distributions in Bayesian inference.
method Birth-death mechanism applied to Langevin diffusion.
result The algorithm accelerates mixing of Langevin diffusion, independent of potential barriers.
New non-reversible Langevin dynamics improve global optimization efficiency.
problem Optimizing non-convex functions efficiently.
method Underdamped and non-symmetric drift Langevin dynamics.
result Non-reversible variants can exit local minima faster and explore state space better.