Uniform diffusion approximation for SGD in non-convex settings.
problem Finite-time diffusion approximation for SGD.
method Establishing uniform-in-time diffusion approximation with strong convexity and mild conditions.
result Uniform-in-time diffusion approximation of SGD without convexity of each loss function.
Improved error estimate for SGLD sampling algorithm.
problem Establishing a precise error bound for SGLD.
method Sharp uniform-in-time error estimate for SGLD under mild assumptions.
result Uniform-in-time O(η2) bound for KL-divergence between SGLD and Langevin diffusion. Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.
problem Understanding long-term behavior of finite-particle systems in relation to their mean-field limits.
method Developed uniform-in-time propagation-of-chaos results for continuous-time SVGD using cutoff strategies and finite-dimensional theories.
result Uniform-in-time propagation-of-chaos bounds in various metrics, including Langevin kernel Stein discrepancy, Wasserstein-1, and Wasserstein-2 distances.
Study shows uniform-time chaos propagation in mean field Langevin dynamics.
problem Understanding the convergence of marginal distributions in mean field dynamics.
method Assumed functional convexity of energy, used Lp-convergence and Wasserstein metrics. result Uniform-in-time propagation of chaos proved in both L2-Wasserstein and relative entropy. Continuous-time PCD for MLE with explicit error bounds.
problem Maximum likelihood estimation of unnormalised densities.
method Continuous-time formulation as coupled SDEs, deriving UiT bounds.
result Explicit error bounds between PCD iterates and MLE solution.
Uniform bounds for neural network convergence without strong convexity assumptions.
problem Understanding the convergence of neural networks in the feature-learning regime.
method Establishing uniform-in-time weak propagation-of-chaos via mean-field deterministic Wasserstein-gradient-flow dynamics.
result Uniform bounds on the difference between infinite-width and finite-width neural network outputs, showing that fewer neurons can achieve a desired loss.
Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the …
Improved particle approximation for mean-field neural networks.
problem Particle approximation error for mean-field neural networks.
method Improved particle approximation error by leveraging the problem structure in risk minimization.
result Established an LSI-constant-free particle approximation error concerning the objective gap.
Stability result for a popular algorithm in optimal transport.
problem Stability of the Iterative Proportional Fitting Procedure in time and metric.
method Uniform stability analysis in the 1-Wasserstein metric.
result Quantitative stability result for entropy-regularized Optimal Transport and Schrödinger bridges.
In this paper we consider the class A of those solutions u(x,t) to the conjugate heat equation dtdu=−Δu+Ru on compact Kähler manifolds M with c1>0 (where g(t) changes by the unnormalized Kähler Ricci flow, blowing up at T<∞), which satisfy Perelman's differential Harnack i…
Extends Langevin dynamics for constrained domains.
problem Optimization of constrained probability measures.
method Mirror mean-field Langevin dynamics (MMFLD).
result Linear convergence guarantees and propagation of chaos results.
We develop a framework for the analysis of deep neural networks and neural ODE models that are trained with stochastic gradient algorithms. We do that by identifying the connections between control theory, deep learning and theory of statistical sampling. We derive Pontryagin's optimality principle and study the corres…
Optimized AIS scheme reduces bias and MSE for general proposals.
problem Performing Monte Carlo integration with general proposals.
method Global optimization of χ²-divergence using stochastic gradient Langevin dynamics.
result Explicit theoretical guarantees for uniform-in-time MSE reduction.
New algorithm for solving minimax problems over distributions converges to Nash equilibrium.
problem Solving minimax problems over probability distributions.
method Symmetric Mean-field Langevin Dynamics (MFL-AG and MFL-ABR) with weighted averaging and best response dynamics.
result Converges to mixed Nash equilibrium with average-iterate and last-iterate convergence.
Improved sampling from mean-field stationary distributions.
problem Sampling from the stationary distribution of mean-field SDEs.
method Decoupling the problem into two aspects: approximation of mean-field SDE and sampling from finite-particle distribution.
result Improved guarantees in various settings, including optimizing neural networks.
Paper shows no finite time singularities for smooth conformal heat flow of harmonic maps.
problem Smoothness of conformal heat flow of harmonic maps.
method Combines harmonic map flow with metric evolution in conformal direction.
result No finite time singularity occurs for the flow, and under certain conditions, maps converge to a point.
Infinite-time blow-up in high-dimensional mean curvature flow.
problem High-dimensional mean curvature flow with exponential asymptotic behavior.
method New zero number argument approach to handle degenerate equations.
result Flow propagates at exponential asymptotic speed, gradients and speeds increase to infinity.
Improved PoC for MFLD reduces approximation error and provides model ensemble guarantees.
problem Quantifying optimization complexity in mean-field Langevin dynamics.
method Refined defective log-Sobolev inequality for neural network training.
result Improved PoC result with reduced approximation error and theoretical model ensemble guarantees.
Improved SGD bounds for machine learning models with Markovian noise.
problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.
method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.
result Matching 1/k decay rate for expected suboptimality. We develop a framework that allows the use of the multi-level Monte Carlo (MLMC) methodology (Giles2015) to calculate expectations with respect to the invariant measure of an ergodic SDE. In that context, we study the (over-damped) Langevin equations with a strongly concave potential. We show that, when appropriate con…
The paper studies mean curvature flow with contact angles in high-dimensional cylinders.
problem Mean curvature flow with prescribed contact angles in a high-dimensional cylinder.
method Derives uniform-in-time gradient bounds and presents a trichotomy result for asymptotic behavior.
result The solution converges to a translating solution with positive speed when a specific condition is met.
New sampling algorithms for complex distributions without log-concavity.
problem Efficient sampling from complex, high-dimensional distributions.
method Randomized splitting Langevin Monte Carlo (RSLMC) algorithm.
result Uniform-in-time error bounds for RSLMC and RLMC algorithms.
Improved convergence rates for MFLD in various gradient estimators.
problem Proving convergence rates for mean-field Langevin dynamics with stochastic gradient updates.
method General framework for propagation of chaos, including finite-particle approximation, time-discretization, and stochastic gradient approximation.
result Improved convergence rates for SGD and SVRG settings.
Work on SGDm under heavy-tailed noise, revealing its generalization properties.
problem Understanding generalization of SGDm under heavy-tailed noise.
method Analysis of continuous-time limit (SDE) and discrete-time SGDm, establishing generalization bounds.
result SGDm can have worse generalization in the presence of heavy-tailed noise for quadratic loss functions.
Gaussian-SVGD dynamics converge to Gaussian distributions under certain conditions.
problem Understanding the theoretical properties of SVGD, especially for Gaussian targets.
method Detailed theoretical study of Gaussian-SVGD dynamics, considering both mean-field PDE and discrete particle systems.
result Gaussian-SVGD dynamics converge linearly to the Gaussian distribution closest to the target in KL divergence.
Develops a new algorithm for estimating model parameters using interacting particle systems.
problem Estimating parameters of latent variable models.
method Interacting Particle Langevin Algorithm (IPLA) based on Langevin diffusion.
result Nonasymptotic optimisation error bounds for the estimator.
Unified stability bounds for noisy SGD across convex and non-convex losses.
problem Deriving generalization bounds for noisy stochastic gradient descent.
method Unified approach using Lyapunov functions and applied probability.
result Time-uniform stability bounds for SGD on various loss functions.
Wide neural networks learn features under μP, identifying weights and decomposing support.
problem Feature learning in wide neural networks under μP. method Proving mean-field limit, characterizing identifiability, sparse-dictionary decomposition, and feature-learning-error decomposition.
result The triple (w∗,Dorb∗,S∗) identifies the natural learning cell of the architecture-data pair (σ,ρ).