New Langevin Monte Carlo algorithms for sampling from nonsmooth distributions.
problem Sampling from distributions with nonsmooth convex composite potentials.
method Leveraging Bregman--Moreau envelopes and proximal operators in mirror descent.
result Efficiency in sampling from nonsmooth distributions, extending existing methods.
PLA improves sampling from distributions under isoperimetry with faster KL divergence convergence.
problem Sampling from distributions with KL divergence under isoperimetry.
method Proximal Langevin Algorithm (PLA) with KL and Rényi divergence convergence guarantees.
result PLA achieves faster KL divergence convergence rates than ULA under log-Sobolev inequality.
The paper analyzes convergence rates of Langevin dynamics and Proximal Sampler using Φ-divergence.
problem Analyzing convergence rates of Langevin dynamics and Proximal Sampler.
method Extending mixing time analyses to Φ-divergence, using strong data processing inequalities. result Convergence of Φ-divergence to 0 exponentially fast along Unadjusted Langevin Algorithm and Proximal Sampler. New algorithms improve inference in non-differentiable models.
problem Inference and learning in latent variable models with non-differentiable densities.
method Proximal interacting particle Langevin algorithms (PIPLA).
result Nonasymptotic bounds and effectiveness demonstrated in various models.
New algorithm speeds up sampling from complex distributions.
problem Efficiently sampling from non-log-concave distributions.
method Stochastic Proximal Samplers (SPS) based on SGLD and MALA.
result SPS-SGLD and SPS-MALA achieve faster sampling with reduced gradient complexity.
We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPLA)---for sampling from a log concave distribution. Our method is a generalization of the Langevin algorithm to potentials expressed as the sum of one stochastic smooth term and multiple stochastic nonsmooth terms. In each iteration, our splitting t…
DE-PSGLD samples from constrained distributions in a decentralized manner.
problem Sampling from log-concave distributions with constraints.
method Decentralized Proximal Stochastic Gradient Langevin Dynamics with proximal regularization.
result DE-PSGLD converges to a regularized Gibbs distribution and maintains posterior concentration.
New algorithm extends LMC to more complex potentials.
problem Addressing limitations of existing LMC methods.
method Inexact Proximal Langevin Algorithm (IPLA).
result Improved convergence rates and broader applicability.
Proves convergence of PSGLA for sampling non-convex potentials.
problem Sampling from non-convex potentials with stability.
method Combines ULA and proximal optimization with stability analysis.
result First proof of convergence for PSGLA on non-convex potentials.
We study sampling as optimization in the space of measures. We focus on gradient flow-based optimization with the Langevin dynamics as a case study. We investigate the source of the bias of the unadjusted Langevin algorithm (ULA) in discrete time, and consider how to remove or reduce the bias. We point out the difficul…
New method accelerates Bayesian imaging using Langevin sampling.
problem Bayesian inference in imaging inverse problems with convex geometry.
method Stochastic relaxed proximal-point iteration targeting posterior distribution.
result Accelerated convergence for κ-strongly log-concave targets. Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
Optimal scaling for proximal MALA in high dimensions confirmed.
problem Optimizing sampling efficiency in high-dimensional target densities.
method Introduced and analyzed the proximal MALA algorithm, showing it maintains optimal scaling.
result Proximal MALA achieves optimal scaling in high dimensions with an average acceptance probability of 0.574.
Noise-free sampling method using Wasserstein proximal for faster convergence.
problem Sampling from distributions governed by potential functions.
method Deterministic score-based MCMC with regularized Wasserstein proximal.
result Improved mixing time bounds for Gaussian distributions compared to ULA and MALA.
Paper analyzes complexity of PSGLA for sampling log-concave distributions.
problem Sampling from log-concave distributions with composite potentials.
method Uses primal-dual interpretation and duality gap to analyze PSGLA complexity.
result Complexity of PSGLA is O(1/ε2) for strongly convex potentials. Study Langevin Monte Carlo for sampling non-log-concave distributions.
problem Sampling from non-log-concave distributions, especially Gaussian mixtures.
method Discretizations of overdamped Langevin diffusions.
result Numerical simulations compare Langevin Monte Carlo algorithms' performance.
New transport method simplifies cutoff phenomenon for Markov processes.
problem Understanding the cutoff phenomenon for Markov processes.
method A new W-TV transport inequality combined with a parabolic regularization estimate.
result Recovery and extension of previous results on cutoff phenomena.
The paper develops methods for sampling from log-concave distributions with constraints.
problem Sampling from log-concave distributions with constraints.
method Randomized midpoint discretization of Langevin diffusions with various projections.
result New convergence guarantees for constrained Langevin algorithms.
The paper analyzes the mean field Langevin dynamics and its convergence rate.
problem The convergence property of the mean field Langevin dynamics in the context of neural networks.
method The analysis uses a proximal Gibbs distribution and techniques from convex optimization.
result A concise convergence rate analysis of the mean field Langevin dynamics in both continuous and discrete time settings.
New method for credible intervals of Covid19 reproduction number.
problem Lack of credibility intervals in existing estimates.
method Combines Langevin Monte Carlo with Proximal operators.
result Produces credible intervals for reproduction number estimates.
New analysis for learning and applying preconditioners in MCMC improves efficiency.
problem Improving efficiency of MCMC algorithms by modifying them with preconditioners.
method Analyzes and compares computational costs of MCMC schemes with and without preconditioners.
result Establishes non-asymptotic guarantees for MCMC algorithms that learn and use preconditioners.
Study improves sampling from non-log-concave distributions using Fisher information.
problem Sampling from non-log-concave distributions with high Fisher information guarantees.
method Proximal sampler with RGO implementation, leveraging log-concave sampling results.
result Improved complexity guarantee in relative Fisher information for non-log-concave sampling.
The paper studies how quickly samples from Langevin dynamics become independent.
problem Understanding the dependence between samples along Langevin dynamics and related algorithms.
method Measures dependence via Φ-mutual information and proves strong data processing inequalities. result The Φ-mutual information between samples decreases exponentially to zero. Optimization is at the heart of machine learning, statistics and many applied scientific disciplines. It also has a long history in physics, ranging from the minimal action principle to finding ground states of disordered systems such as spin glasses. Proximal algorithms form a class of methods that are broadly applica…
Langevin algorithms improve training of very deep neural networks, especially for image classification.
problem Training very deep neural networks is challenging due to increased non-linearity and the risk of getting stuck in local minima.
method Comparison of Langevin and non-Langevin algorithms for training deep neural networks, introduction of Layer Langevin algorithm.
result Langevin algorithms, especially Layer Langevin, lead to significant improvements in training deep neural networks, particularly for image classification tasks.
New algorithms improve sampling from complex distributions.
problem Sampling from complex probability distributions efficiently.
method Regime-switching Langevin dynamics and Monte Carlo algorithms.
result Convergence guarantees and iteration complexities provided.
Study non-asymptotic Langevin Monte Carlo for Gibbs distributions.
problem Sampling from Gibbs distributions with dissipative potentials.
method Langevin-type algorithms based on Liptser--Shiryaev theory and Poincaré inequalities.
result Upper bound on 2-Wasserstein distance for accurate approximation.
Improved log-concave sampling to O(d1/2) with warm starts.
problem Sampling from strongly log-concave distributions efficiently.
method Warm starts and discretized underdamped Langevin diffusion.
result Achieved O(d1/2) complexity for high-accuracy sampling. Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
Adaptive algorithm improves convergence rate of Langevin dynamics.
problem Improving convergence rate of Langevin dynamics.
method Adaptive non-reversible stochastic gradient Langevin dynamics algorithm.
result Improved convergence rate of the algorithm.
New Langevin algorithms improve sampling efficiency in high dimensions.
problem Sampling from log-concave and smooth distributions in high dimensions.
method Combining splitting and accurate integration methods for P-th order Langevin dynamics. result LMC algorithms converge faster with better dimension dependence as P increases. Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin alg…
Langevin DQN achieves deep exploration using Gaussian noise.
problem Deep exploration in reinforcement learning.
method Developed Langevin DQN, a variation of DQN with Gaussian noise.
result Langevin DQN achieves deep exploration.
Unified approach for sampling non-differentiable and heavy-tailed targets.
problem Sampling non-differentiable and heavy-tailed distributions using Langevin algorithms.
method Anchored Langevin dynamics, which modifies the Langevin diffusion with a smooth reference potential and multiplicative scaling.
result Non-asymptotic guarantees in the 2-Wasserstein distance to the target distribution.
New algorithm improves sampling from constrained spaces.
problem Sampling from constrained spaces efficiently.
method Metropolis-adjusted Mirror Langevin algorithm.
result Unbiased sampling with improved mixing time.
The Langevin Algorithm's stationary distribution is shown to be sub-exponential or sub-Gaussian under certain conditions.
problem Understanding the properties of the Langevin Algorithm's stationary distribution.
method Analysis using a rotation-invariant moment generating function (Bessel function) to study the stationary dynamics of the Langevin Algorithm.
result Concentration results for the Langevin Algorithm's stationary distribution πη are established, showing it is sub-exponential or sub-Gaussian under convex or strongly convex potential conditions. Paper analyzes and compares ELF algorithms for federated learning.
problem Improving efficiency and privacy in federated learning.
method Proposes P-ELF, D-ELF, and B-ELF algorithms with primal, dual, and bidirectional compression.
result Provides non-asymptotic convergence guarantees under Log-Sobolev inequality.
Optimal preconditioning improves Langevin sampling efficiency.
problem Improving sampling efficiency in high-dimensional target distributions.
method Optimal preconditioning using Fisher information, applied to MALA.
result Adaptive MCMC scheme significantly outperforms other methods.
Improved Langevin algorithms with prior diffusion achieve dimension-independent convergence for non-log-concave distributions.
problem Understanding the dimension dependency of computational complexity in high-dimensional sampling.
method Investigation of prior diffusion technique for log-Sobolev inequality target distributions.
result Modified Langevin algorithm achieves dimension-independent KL divergence convergence.
Paper proposes Langevin dynamics for adaptive IRL of stochastic gradient algorithms.
problem Estimating reward functions from noisy gradient estimates of stochastic gradient agents.
method Generalized Langevin dynamics algorithm for IRL.
result Proposed algorithms asymptotically generate samples proportional to exp(R(θ)).
Noise-corrected Langevin algorithm improves sampling from noisy data.
problem Sampling from noisy data with biased score function.
method Noise-corrected Langevin algorithm using noisy score function.
result Bias due to noisy data is removed, improving sampling accuracy.
Langevin algorithms enhance training of deep neural networks for stochastic control problems.
problem Training acceleration for deep neural networks in stochastic control problems.
method Application of Langevin algorithms to minimize the loss of deep neural networks in stochastic control problems.
result Langevin algorithms improve training on various stochastic control problems.
This paper resolves the Langevin Algorithm's mixing time for log-concave distributions.
problem Resolving the mixing time of the Langevin Algorithm for log-concave sampling.
method Introducing Privacy Amplification by Iteration to analyze Rényi divergence and Optimal Transport smoothing.
result Optimal mixing bounds for the Langevin Algorithm in log-concave sampling settings.
A new algorithm solves semidefinite programs using Langevin diffusion.
problem Optimizing semidefinite programs with diagonal constraints.
method Langevin diffusion on a product manifold of spheres.
result Langevin algorithm achieves ε accuracy in Ω(ε^-5) iterations.
Introduces PPMM algorithm for nonconvex robust regression problems.
problem Nonconvex tuning-free robust regression problems.
method PPMM algorithm with inner subproblems solved by SSN-PPA.
result Converges to d-stationary point with KL property.
In machine learning research, the proximal gradient methods are popular for solving various optimization problems with non-smooth regularization. Inexact proximal gradient methods are extremely important when exactly solving the proximal operator is time-consuming, or the proximal operator does not have an analytic sol…
New Langevin method achieves third order convergence for strongly log-concave distributions.
problem Sampling from complex distributions efficiently.
method Underdamped Langevin diffusion with third order convergence.
result Achieves 2-Wasserstein error of ε in O(√d/ε^1/3) steps under additional Lipschitz condition.
Paper proposes new Langevin samplers for sampling from log-concave distributions with superlinear gradient growth.
problem Sampling from log-concave distributions with superlinear gradient growth.
method Proposes two novel discretizations of kinetic Langevin SDEs, showing contractivity and log-Sobolev inequality.
result Establishes non-asymptotic bounds in 2-Wasserstein distance between sampled distributions and target measures.