Study shows uniform-time chaos propagation in mean field Langevin dynamics.
problem Understanding the convergence of marginal distributions in mean field dynamics.
method Assumed functional convexity of energy, used Lp-convergence and Wasserstein metrics. result Uniform-in-time propagation of chaos proved in both L2-Wasserstein and relative entropy. Extends Langevin dynamics for constrained domains.
problem Optimization of constrained probability measures.
method Mirror mean-field Langevin dynamics (MMFLD).
result Linear convergence guarantees and propagation of chaos results.
The paper analyzes the mean field Langevin dynamics and its convergence rate.
problem The convergence property of the mean field Langevin dynamics in the context of neural networks.
method The analysis uses a proximal Gibbs distribution and techniques from convex optimization.
result A concise convergence rate analysis of the mean field Langevin dynamics in both continuous and discrete time settings.
Study on Langevin dynamics convergence rates and their application to GAN training.
problem Understanding the long-term behavior of Langevin dynamics equations.
method Analytical and numerical methods to study convergence rates of underdamped mean-field Langevin dynamics.
result Exponential convergence rate results for the Langevin dynamics under various conditions.
New algorithm for solving minimax problems over distributions converges to Nash equilibrium.
problem Solving minimax problems over probability distributions.
method Symmetric Mean-field Langevin Dynamics (MFL-AG and MFL-ABR) with weighted averaging and best response dynamics.
result Converges to mixed Nash equilibrium with average-iterate and last-iterate convergence.
New method optimizes non-linear functionals over probability measures.
problem Optimizing non-linear functionals defined over probability measures.
method N-particle underdamped Langevin algorithm with spacetime discretization.
result Converges globally in total variation distance.
Study on convergence of Langevin dynamics for zero-sum games in probability distributions.
problem Analyzing convergence of Langevin dynamics for zero-sum games in probability distributions.
method Proved exponential and biased convergence guarantees for mean-field and finite-particle min-max Langevin dynamics.
result Explicit iteration complexity for finite-particle algorithms to approximate equilibrium distributions.
Algorithm generates private continuous-time data for sensitive domains.
problem Private generation of continuous-time data for sensitive domains.
method Mean-field Langevin dynamics and noisy particle gradient descent.
result Strong privacy guarantees for one-time data contributions.
Improved convergence rates for MFLD in various gradient estimators.
problem Proving convergence rates for mean-field Langevin dynamics with stochastic gradient updates.
method General framework for propagation of chaos, including finite-particle approximation, time-discretization, and stochastic gradient approximation.
result Improved convergence rates for SGD and SVRG settings.
Improved particle approximation for mean-field neural networks.
problem Particle approximation error for mean-field neural networks.
method Improved particle approximation error by leveraging the problem structure in risk minimization.
result Established an LSI-constant-free particle approximation error concerning the objective gap.
Improved PoC for MFLD reduces approximation error and provides model ensemble guarantees.
problem Quantifying optimization complexity in mean-field Langevin dynamics.
method Refined defective log-Sobolev inequality for neural network training.
result Improved PoC result with reduced approximation error and theoretical model ensemble guarantees.
Improved sampling from mean-field stationary distributions.
problem Sampling from the stationary distribution of mean-field SDEs.
method Decoupling the problem into two aspects: approximation of mean-field SDE and sampling from finite-particle distribution.
result Improved guarantees in various settings, including optimizing neural networks.
Study uses neural nets to learn multi-index models in high dimensions, reducing complexity.
problem Learning multi-index models in high-dimensional data.
method Mean-field Langevin dynamics with neural networks.
result Effective dimension controls sample and computational complexity, potentially reducing it.
PDA method optimizes neural networks with global convergence rate analysis.
problem Quantitative convergence rate for neural network optimization in mean field regime.
method Particle dual averaging (PDA) method, combining Langevin algorithm and outer loop optimization.
result Established quantitative global convergence for two-layer mean field neural networks.
Method learns radial basis function distributions from samples.
problem Learning radial basis function distributions from training samples.
method Projected particle Langevin optimization method with distributionally robust optimization.
result Empirical measure of Langevin particles converges to a reflected Itô diffusion-drift process.
Paper extends MLFD to signed measures via bilevel approach.
problem Risk minimization for infinite width neural networks and sparse deconvolution.
method Bilevel reduction to extend MLFD to signed measures, investigating convergence rates.
result Improved convergence rates for bilevel MFLD in low-noise regime and local exponential convergence for single neuron learning.
We introduce a mean-field type approximation for description of company's income statistics. Utilizing huge company data we show that a discrete version of Langevin equation with additive and multiplicative noises can appropriately describe the time evolution of a company's income fluctuation in statistical sense. The …
New method infers population dynamics from snapshots using path space optimization.
problem Recover dynamics of a population from its temporal marginals.
method Grid-free algorithm using Schrödinger bridges coupled via noisy gradient descent in mean-field limit.
result Global convergence to min-entropy estimator with end-to-end theoretical guarantees.
Mean-field neural nets approximate functions using a free energy functional and controlled dynamics.
problem Function approximation by two-layer neural nets in the mean-field regime.
method Phrasing function approximation as global minimization of a free energy functional, examining dynamics in the space of probability measures over weights.
result Characterization of the unique global minimizer and dynamics achieving it, including the Föllmer drift.
A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions. Due to metastability, multimodal distributions are difficult to sample using standard Markov chain Monte Carlo methods. We propose a new sampling algorithm based on a birth-death mechanis…
New method uses birth-death process and exploration component to accelerate sampling from multimodal distributions.
problem Sampling from multimodal probability distributions efficiently.
method Combines birth-death process and exploration component to accelerate sampling.
result Proves exponential asymptotic convergence under mild assumptions.
Unified analysis of neural networks in NPIV using 2SLS and MFLD.
problem Global convergence of neural networks in NPIV.
method Lifted perspective through MFLD, penalty gradient approach for bilevel optimization.
result First global convergence result of neural networks for 2SLS in NPIV.
Two-layer neural networks learn efficiently using kernel methods in mean-field analysis.
problem Feature learning ability of two-layer neural networks in the mean-field regime.
method Mean-field analysis through kernel methods, focusing on dynamics of the first layer's kernel.
result Two-layer neural networks can learn a union of multiple reproducing kernel Hilbert spaces more efficiently than kernel methods.
Gradient descent variants improve phase retrieval accuracy.
problem Phase retrieval problem in high-dimensional spaces.
method Gradient descent, stochastic gradient descent, Langevin algorithm, dynamical mean-field theory.
result Stochastic variants of gradient descent achieve better generalization in phase retrieval.
New model explains market dynamics with phase transitions and non-linear interactions.
problem Understanding complex multi-asset market dynamics with phase transitions.
method Developed a Multi-Asset Non-Equilibrium Skew (MANES) model based on Langevin dynamics and McKean-Vlasov equation.
result The model accurately predicts market returns and phase transitions in both benign and distressed markets.
A model for collaborative learning with principal-agent interaction.
problem Optimizing parameter estimates in a collaborative learning setting.
method Decision-theoretic model with aggregation coefficients and Langevin dynamics.
result Advantages in stability and generalization due to cooperative behavior.
New algorithms for sampling and optimization without tuning.
problem Efficient sampling and optimization over probability measures.
method Optimization on the space of probability measures, using gradient flows.
result Strong theoretical guarantees and similar performance to optimally tuned algorithms.
Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.
problem Understanding long-term behavior of finite-particle systems in relation to their mean-field limits.
method Developed uniform-in-time propagation-of-chaos results for continuous-time SVGD using cutoff strategies and finite-dimensional theories.
result Uniform-in-time propagation-of-chaos bounds in various metrics, including Langevin kernel Stein discrepancy, Wasserstein-1, and Wasserstein-2 distances.
New method for variational inference without conjugacy constraints.
problem Efficient variational inference with flexible prior and approximation families.
method Wasserstein gradient flow for mean-field approximation.
result Improved convergence and efficiency of variational inference.
We develop a framework for the analysis of deep neural networks and neural ODE models that are trained with stochastic gradient algorithms. We do that by identifying the connections between control theory, deep learning and theory of statistical sampling. We derive Pontryagin's optimality principle and study the corres…
Paper optimizes approximating high-dimensional diffusions by independent coordinates.
problem Optimizing approximations of high-dimensional diffusions by independent coordinates.
method Introduces independent projection as optimal for two criteria.
result Independent projection is optimal for two criteria related to entropy and convergence.
New framework analyzes deep learning optimization with finite width networks, revealing generalization gaps and excess risks.
problem Analyzing generalization error of deep learning with finite width networks.
method Formulating neural network training as transportation map estimation and analyzing via infinite dimensional Langevin dynamics.
result Achieves fast learning rate and minimax optimal rates for classification and regression problems.
Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already observed in the works of Mei, Montanari and Nguyen (2018), Chizat and Bach (2018) as …
We review some approaches to the understanding of fluctuations in some models used to describe socio and economic systems. Our approach builds on the development of a simple Langevin equation that characterises stochastic processes. This provides a unifying approach that allows first a straightforward description of th…
Wide neural networks learn features under μP, identifying weights and decomposing support.
problem Feature learning in wide neural networks under μP. method Proving mean-field limit, characterizing identifiability, sparse-dictionary decomposition, and feature-learning-error decomposition.
result The triple (w∗,Dorb∗,S∗) identifies the natural learning cell of the architecture-data pair (σ,ρ). Method identifies mixed Nash equilibria in high dimensions for training mixtures of GANs.
problem Finding Nash equilibria in two-player zero-sum continuous games, especially in high dimensions.
method Parametrizing mixed strategies as mixtures of particles, updating their positions and weights using gradient descent-ascent.
result Global convergence to an approximate equilibrium for the related Langevin gradient-ascent dynamic.
We analyze the linear response of a market network to shocks based on the bipartite market model we introduced in an earlier paper, which we claimed to be able to identify the time-line of the 2009-2011 Eurozone crisis correctly. We show that this model has three distinct phases that can broadly be categorized as "stab…
Advocates for a new posterior that predicts better than classical and generalised Bayes.
problem Combining parameter inference and density estimation for better predictive models.
method Predictively Oriented (PrO) posterior using mean field Langevin dynamics.
result PrO posteriors converge to the predictively optimal model average, adapting to model misspecification.
Langevin algorithms improve training of very deep neural networks, especially for image classification.
problem Training very deep neural networks is challenging due to increased non-linearity and the risk of getting stuck in local minima.
method Comparison of Langevin and non-Langevin algorithms for training deep neural networks, introduction of Layer Langevin algorithm.
result Langevin algorithms, especially Layer Langevin, lead to significant improvements in training deep neural networks, particularly for image classification tasks.
SLMC improves sampling efficiency for high-dimensional distributions.
problem Sampling from high-dimensional distributions is computationally challenging.
method SLMC projects Langevin updates onto subsampled eigenblocks of a time-varying preconditioner.
result SLMC offers superior adaptability and computational efficiency compared to traditional methods.
Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
New algorithms improve sampling from complex distributions.
problem Sampling from complex probability distributions efficiently.
method Regime-switching Langevin dynamics and Monte Carlo algorithms.
result Convergence guarantees and iteration complexities provided.
Study non-asymptotic Langevin Monte Carlo for Gibbs distributions.
problem Sampling from Gibbs distributions with dissipative potentials.
method Langevin-type algorithms based on Liptser--Shiryaev theory and Poincaré inequalities.
result Upper bound on 2-Wasserstein distance for accurate approximation.
Paper adds Fisher Information to mean field optimization for faster convergence.
problem Mean field optimization in neural networks training.
method Developed energy-dissipation method and gradient flow on probability space.
result Marginal distributions converge exponentially to minimizer.
Unified approach for sampling non-differentiable and heavy-tailed targets.
problem Sampling non-differentiable and heavy-tailed distributions using Langevin algorithms.
method Anchored Langevin dynamics, which modifies the Langevin diffusion with a smooth reference potential and multiplicative scaling.
result Non-asymptotic guarantees in the 2-Wasserstein distance to the target distribution.
Novel geometry-informed irreversible perturbation accelerates Langevin dynamics convergence.
problem Accelerating convergence of Langevin dynamics for Bayesian computation.
method Geometry-informed irreversible perturbation of Riemannian manifold Langevin dynamics.
result Improves estimation performance over irreversible perturbations that ignore geometry.
Study explores optimal strategies in games with multiple players and mean-field interactions.
problem Optimal strategies in games with multiple players and mean-field interactions.
method Exploration of three different notions of optimality, including mean-field control solution, mean-field coarse correlated equilibria, and mean-field Nash equilibria.
result Approximation of cooperative and competitive equilibria in large N-player games by mean-field control and mean-field equilibria. New methods use transport maps to improve Langevin dynamics for sampling.
problem Sampling high-dimensional, non-Gaussian distributions efficiently.
method Apply transport maps to accelerate Langevin dynamics convergence.
result Discretized processes converge to target distribution with non-asymptotic bounds.