Paper approximates risk measures using SGD with Langevin dynamics.
problem Approximating arbitrary law invariant risk measures.
method Stochastic Gradient Langevin Dynamics (SGD-Langevin) for general risk measures.
result Non-asymptotic convergence rates of the approximation algorithm.
Estimates generalization gap for overparameterized models using Langevin approximation.
problem Estimating the difference between training and generalization performance in overparameterized models.
method Functional variance and Langevin approximation of functional variance.
result Demonstrates efficient estimation of generalization gaps for overparameterized models.
A new diffusion method approximates Schrödinger bridge with improved convergence.
problem Approximating Schrödinger bridge with Langevin diffusion.
method Leveraging Langevin diffusion to approximate Schrödinger bridge.
result The difference between the two approximations is proportional to the score function.
Improved sampling for high-dimensional posteriors with underdamped Langevin.
problem Scalability issues in high-dimensional problems with approximate Thompson sampling.
method Underdamped Langevin Monte Carlo for accelerated posterior concentration.
result Logarithmic regret improvement from i l d e O ( d ) \mathcal{ ilde O}(d) i l d e O ( d ) to i l d e O ( d ) \mathcal{ ilde O}(\sqrt{d}) i l d e O ( d ) . New algorithms improve Langevin Monte Carlo efficiency.
problem High computational cost of classical Langevin Monte Carlo.
method Integrates ensemble feature into LMC, constraining gradient approximations.
result Constrained Ensemble Langevin Monte Carlo reduces gradient computation.
A new TS-SA method alleviates non-stationarity in TS algorithms for bandits.
problem Non-stationarity in existing TS algorithms for multi-armed bandits.
method Integrates stochastic approximation within TS framework, using Langevin Monte Carlo and SA steps.
result Establishes near-optimal regret bounds for TS-SA, with simplified analysis.
New algorithm improves sampling from constrained spaces.
problem Sampling from constrained spaces efficiently.
method Metropolis-adjusted Mirror Langevin algorithm.
result Unbiased sampling with improved mixing time.
Combining diffusion models with Langevin dynamics improves posterior sampling efficiency.
problem Sampling from noisy posterior distributions efficiently.
method Annealed Langevin dynamics combined with diffusion models.
result Achieves posterior sampling in polynomial time with a weaker score error bound.
SAGD uses Langevin algorithm for efficient gradient descent.
problem Efficiently approximating gradients in complex models.
method Langevin algorithm for biased but asymptotically accurate gradients.
result Theoretical convergence guarantee for SAGD.
Study non-asymptotic Langevin Monte Carlo for Gibbs distributions.
problem Sampling from Gibbs distributions with dissipative potentials.
method Langevin-type algorithms based on Liptser--Shiryaev theory and Poincaré inequalities.
result Upper bound on 2-Wasserstein distance for accurate approximation.
Efficiently simulates Langevin dynamics on manifold using diffusion maps and finite volume schemes.
problem Simulating Langevin dynamics on high-dimensional manifolds with limited data.
method Diffusion maps, Fokker-Planck equation, finite volume scheme, explicit time discretization.
result Data-driven finite volume scheme approximates Langevin dynamics on manifold with good properties.
A scalable framework uses Langevin sampling to approximate neural network models of evolving processes.
problem Uncertainty quantification in neural network models of dynamic systems.
method Flexible data model based on NODE, joint learning of data model and posterior parameters, Langevin sampling.
result Demonstrated performance on chemical reaction and material physics data, compared favorably to variational inference.
Study on convergence of Langevin dynamics for zero-sum games in probability distributions.
problem Analyzing convergence of Langevin dynamics for zero-sum games in probability distributions.
method Proved exponential and biased convergence guarantees for mean-field and finite-particle min-max Langevin dynamics.
result Explicit iteration complexity for finite-particle algorithms to approximate equilibrium distributions.
A new method learns latent space normalizing flow for approximate inference in generator models.
problem Approximate inference in generator models with complex posterior distributions.
method Jointly learns latent space normalizing flow and generator model using MCMC-based maximum likelihood.
result The short-run Langevin flow approximates the posterior and aligns with the normalizing flow prior.
The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.
problem Sampling from complex target distributions efficiently.
method A neural network architecture inspired by Langevin Monte Carlo is proposed to map samples from a simple reference distribution to samples from the target.
result The proposed neural network architecture achieves approximation rates in the Wasserstein-2 distance for smooth, log-concave target distributions.
Improved Thompson sampling for bandits with Langevin algorithms.
problem Thompson sampling's computational inefficiency in generating posterior samples.
method Developed Langevin algorithms for approximate sampling with posterior concentration guarantees.
result Logarithmic regret with constant computational complexity.
Improved convergence rates for MFLD in various gradient estimators.
problem Proving convergence rates for mean-field Langevin dynamics with stochastic gradient updates.
method General framework for propagation of chaos, including finite-particle approximation, time-discretization, and stochastic gradient approximation.
result Improved convergence rates for SGD and SVRG settings.
The paper develops methods for sampling from log-concave distributions with constraints.
problem Sampling from log-concave distributions with constraints.
method Randomized midpoint discretization of Langevin diffusions with various projections.
result New convergence guarantees for constrained Langevin algorithms.
SGLDiff approximates Bayesian posterior distributions with subsampling error.
problem Approximating Bayesian posterior distributions in large-scale data settings.
method Stochastic Gradient Langevin Diffusion (SGLDiff) with subsampling.
result The Wasserstein distance between the posterior and SGLDiff's limiting distribution is bounded by a fractional power of the mean waiting time.
Paper tackles Bayesian image restoration in low-photon Poisson imaging problems.
problem Bayesian inference in challenging low-photon Poisson imaging problems.
method Plug-and-play (PnP) Langevin sampling strategies with accelerated methods and mirror sampling.
result Effective PnP Langevin sampling methods for low-photon Poisson imaging problems.
New algorithm learns LQR with O ( T ) O(\sqrt{T}) O ( T ) regret using Langevin dynamics and excitation.
problem Learning LQR with a O ( T ) O(\sqrt{T}) O ( T ) regret bound. method Thompson sampling with Langevin dynamics and excitation mechanism.
result Achieved O ( T ) O(\sqrt{T}) O ( T ) regret bound for LQR learning. We present a time-dependent Langevin description of dynamics of stock prices. Based on a simple sliding-window algorithm, the fluctuation of stock prices is discussed in the view of a time-dependent linear restoring force which is the linear approximation of the drift parameter in Langevin equation estimated from the f…
This paper optimizes Bayesian estimation for log-concave models using Langevin Monte-Carlo.
problem Optimizing Bayesian estimators for log-concave models with Langevin Monte-Carlo.
method Quantitative statistical bounds and numerical approximation of Gibbs measures.
result Established optimal numerical strategy and its cost for Bayesian posterior mean approximation.
New methods use transport maps to improve Langevin dynamics for sampling.
problem Sampling high-dimensional, non-Gaussian distributions efficiently.
method Apply transport maps to accelerate Langevin dynamics convergence.
result Discretized processes converge to target distribution with non-asymptotic bounds.
New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
Study Langevin Monte Carlo for sampling non-log-concave distributions.
problem Sampling from non-log-concave distributions, especially Gaussian mixtures.
method Discretizations of overdamped Langevin diffusions.
result Numerical simulations compare Langevin Monte Carlo algorithms' performance.
Paper analyzes Annealed Langevin Dynamics for multimodal sampling stability.
problem Ensuring stability of Annealed Langevin Dynamics across dimensions.
method Uniform-in-dimension analysis of ALD for Gaussian-mixture targets.
result ALD achieves prescribed accuracy in KL divergence with spectral conditions.
Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin alg…
We present a unified framework to analyze the global convergence of Langevin dynamics based algorithms for nonconvex finite-sum optimization with n n n component functions. At the core of our analysis is a direct analysis of the ergodicity of the numerical approximations to Langevin dynamics, which leads to faster conver…
Improved particle approximation for mean-field neural networks.
problem Particle approximation error for mean-field neural networks.
method Improved particle approximation error by leveraging the problem structure in risk minimization.
result Established an LSI-constant-free particle approximation error concerning the objective gap.
One way to avoid overfitting in machine learning is to use model parameters distributed according to a Bayesian posterior given the data, rather than the maximum likelihood estimator. Stochastic gradient Langevin dynamics (SGLD) is one algorithm to approximate such Bayesian posteriors for large models and datasets. SGL…
New method reduces variance in random coordinate descent for Langevin Monte Carlo.
problem Efficient sampling from log-concave distributions in high dimensions.
method Introduces RCAD, a variance reduction technique for RCD-LMC.
result RCAD-O-LMC and RCAD-U-LMC converge within the same number of iterations as classical LMC methods, saving computational cost.
FSGLD improves federated data sampling by correcting noisy gradients.
problem Noisy gradients and delayed communication in federated data.
method Conductive gradients to correct noisy gradients in distributed SGLD.
result FSGLD converges to true posterior even with delayed communication.
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
Langevin autoencoders improve deep latent variable models with efficient posterior sampling.
problem Efficient posterior sampling in deep latent variable models using MCMC.
method Amortized Langevin dynamics (ALD) replaces datapoint-wise sampling with encoder updates.
result ALD is valid as an MCMC algorithm with the target posterior as a stationary distribution.
We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows. Here two information metrics are considered, including both the Fisher-Rao metric and the Wasserstein-2 metric. A known fact is that overdamped Langevin dynamics correspond to Wasserstein gradien…
Efficiently samples posterior distributions using Langevin dynamics.
problem Challenges in generating diverse posterior samples in high-dimensional spaces.
method Simulates Langevin dynamics in the noise space of a pre-trained generative model.
result Noise-space Langevin dynamics approximates the posterior without restarting the full sampling chain.
Tractable model explains market dynamics using Langevin and SUSY QM.
problem Understanding non-linear market dynamics and option pricing.
method Langevin dynamics mapped to QM, using SUSY to find solutions.
result NES model provides accurate option pricing with a single volatility parameter.
New sampling method guarantees approximate first-order stationary points for non-convex functions.
problem Sampling from non-log-concave densities with non-convex potential functions.
method Averaged Langevin Monte Carlo with complexity analysis.
result Langevin Monte Carlo outputs a sample with ε-relative Fisher information after O(L²d²/ε²) iterations.
New method approximates sampling from smooth potential distributions using a vanishing penalty.
problem Sampling from smooth potential distributions on high-dimensional spaces.
method Penalized Langevin dynamics (PLD) with vanishing penalty.
result Established upper bound on Wasserstein-2 distance for PLD approximation.
In this paper, we provide new insights on the Unadjusted Langevin Algorithm. We show that this method can be formulated as a first order optimization algorithm of an objective functional defined on the Wasserstein space of order 2 2 2 . Using this interpretation and techniques borrowed from convex optimization, we give a …
Proves convergence of PSGLA for sampling non-convex potentials.
problem Sampling from non-convex potentials with stability.
method Combines ULA and proximal optimization with stability analysis.
result First proof of convergence for PSGLA on non-convex potentials.
Proposes a new Langevin flow approach for VAEs.
problem Difficulty in constructing low variance ELBO for VAEs with large datasets.
method Integrates Langevin dynamic with quasi-symplectic integrator to improve posterior estimation.
result Shows theoretical and practical effectiveness compared to gradient flow-based methods.
Stochastic EM with biased MCMC improves inference stability.
problem Intractable E-step in EM algorithm.
method Stochastic approximation with biased MCMC.
result ULA is more stable and sometimes faster than MALA.
DMPS uses diffusion maps and LAWGD for efficient generative modeling.
problem Efficiently modeling complex data distributions.
method Diffusion maps for manifold learning and LAWGD for sampling.
result DMPS outperforms other methods on moderate-dimensional data.
Paper analyzes Gibbs and Langevin Monte Carlo for interpolation regime, showing generalization from low errors.
problem Analyzing Gibbs and Langevin Monte Carlo in overparameterized interpolation regime.
method Data-dependent bounds and stability under approximation with Langevin Monte Carlo.
result Generalization is signaled by small training errors in noisy regime, with bounds stable under approximation.
Improved PoC for MFLD reduces approximation error and provides model ensemble guarantees.
problem Quantifying optimization complexity in mean-field Langevin dynamics.
method Refined defective log-Sobolev inequality for neural network training.
result Improved PoC result with reduced approximation error and theoretical model ensemble guarantees.
Annealed Langevin dynamics improves sampling from composite scores in SBI.
problem Irreducible bias in sampling from composite scores of SBI methods.
method Derive Wasserstein bounds and decision rules for hyperparameters.
result Explicit decision rules for hyperparameters guarantee prescribed sampling accuracy.