A new method uses Mean Field Games to optimize mixture models of Bernoulli and categorical distributions.
problem Optimizing parameters of finite mixture models of Bernoulli and categorical distributions.
method Mean Field Games theory applied to multi-population systems.
result The Mean Field Games approach provides a method to compute mixture model parameters.
Existence of strong randomized equilibria in mean-field games with common noise.
problem Existence of strong solutions in mean-field games of optimal stopping.
method Connection with Bank-El Karoui's representation problem and continuity assumptions.
result Existence of strong randomized mean-field equilibrium under certain conditions.
The paper analyzes the mean field Langevin dynamics and its convergence rate.
problem The convergence property of the mean field Langevin dynamics in the context of neural networks.
method The analysis uses a proximal Gibbs distribution and techniques from convex optimization.
result A concise convergence rate analysis of the mean field Langevin dynamics in both continuous and discrete time settings.
Two-layer neural networks learn efficiently using kernel methods in mean-field analysis.
problem Feature learning ability of two-layer neural networks in the mean-field regime.
method Mean-field analysis through kernel methods, focusing on dynamics of the first layer's kernel.
result Two-layer neural networks can learn a union of multiple reproducing kernel Hilbert spaces more efficiently than kernel methods.
Study explores optimal strategies in games with multiple players and mean-field interactions.
problem Optimal strategies in games with multiple players and mean-field interactions.
method Exploration of three different notions of optimality, including mean-field control solution, mean-field coarse correlated equilibria, and mean-field Nash equilibria.
result Approximation of cooperative and competitive equilibria in large N-player games by mean-field control and mean-field equilibria. Revises mean-field theory of Santa Fe model using kinetic theory.
problem Deriving a solid mathematical foundation for the Santa Fe model.
method Systematic derivation of BBGKY hierarchy from exact master equation.
result Explicit and closed-form solutions for mean-field equations.
New algorithm tackles multi-agent reinforcement learning issues.
problem Multi-agent reinforcement learning suffers from the curse of many agents.
method Proposes MF-FQI algorithm based on mean embeddings of distributions.
result Establishes a non-asymptotic analysis for MF-FQI algorithm.
In this work we introduce a model of default contagion that combines the approaches of Eisenberg-Noe interbank networks and dynamic mean field interactions. The proposed contagion mechanism provides an endogenous rule for early defaults in a network of financial institutions. The main result is to demonstrate a mean fi…
Kernel methods are studied in a mean field limit for high-dimensional data.
problem Analyzing kernel methods in high-dimensional data with many variables.
method Investigation of kernel methods in the mean field limit of interacting particle systems.
result Rigorous mean field limit of kernels and detailed analysis of the limiting reproducing kernel Hilbert space.
Paper analyzes Transformer learning dynamics, proving benign landscape for in-context learning.
problem Understanding how Transformers learn in context with nonlinear features.
method Mean-field and two-timescale analysis of Transformer dynamics, proving nonconvex but benign landscape.
result Proves mean-field dynamics avoid saddle points, leading to improved optimization.
Study shows benefits of transfer learning with neural networks.
problem Understanding generalization errors in transfer learning.
method Mean-field analysis applied to α-ERM and fine-tuning. result Established conditions for generalization error and convergence rates.
This work shows linear convergence for two-layer neural networks in mean-field regime.
problem Optimizing two-layer neural networks in the mean-field regime.
method Mean-field analysis and continuous-time noisy gradient descent.
result Establishes linear convergence rate for two-layer neural networks.
ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
Global convergence proved for three-layer neural networks in mean field regime.
problem Optimization efficiency of multilayer neural networks in the mean field regime.
method Developed a rigorous framework for mean field limit of three-layer networks using stochastic gradient descent and neuronal embedding.
result Global convergence guarantee for unregularized feedforward three-layer networks in the mean field regime.
Extends LIBOR market model to reduce exploding scenarios.
problem Exploding scenarios in market-consistent guarantees valuation.
method Mean-field extension of the LIBOR market model.
result Existence and uniqueness of MF-LMM proved.
MF-TRPO optimizes MFGs with finite sample guarantees.
problem Computing approximate Nash equilibria in MFGs.
method Extends TRPO to MFGs, providing convergence guarantees.
result Theoretical guarantees on MF-TRPO's convergence.
New algorithm solves complex mean-field Schrödinger bridge problem.
problem Designing a controller for diffusion processes with nonlocal interaction.
method Generalized Hopf-Cole transform and Sinkhorn-type algorithm.
result Convergence guarantees for the proposed algorithm under mild assumptions.
Wide BNNs with odd activations fail to approximate data under mean-field inference.
problem Theoretical limitations of mean-field variational inference in wide, deep Bayesian neural networks.
method Analysis of mean-field variational inference in fully-connected BNNs with odd activation functions and Gaussian likelihood.
result The optimal mean-field variational posterior predictive distribution converges to the prior predictive distribution as network width increases.
New methods learn correlated equilibria in large games without structural assumptions.
problem Learning correlated equilibria in large, anonymous games with exponential player count.
method Developed Mean-Field correlated and coarse-correlated equilibria, and used classical algorithms to learn them efficiently.
result Efficiently learned correlated equilibria in all games without structural assumptions.
Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.
problem Analyzing convergence of proximal algorithm in general metric spaces.
method Analysis of the Wasserstein proximal algorithm without geodesic convexity assumption.
result Establishes unbiased and linear convergence rate for proximal algorithm under natural Wasserstein inequality.
Gradient descent finds global optima in ResNets with sufficient parameters.
problem Finding optimal parameters in ResNet models.
method Mean-field analysis and gradient-flow PDE to study convergence of first-order optimization methods.
result First-order methods can find global minimizers in overparameterized ResNets.
We conduct non-asymptotic analysis on the mean-field variational inference for approximating posterior distributions in complex Bayesian models that may involve latent variables. We show that the mean-field approximation to the posterior can be well-approximated relative to the Kullback-Leibler divergence discrepancy m…
Modeling pollution from competing firms using mean-field games.
problem Pollution regulation of competitive firms producing similar goods.
method Developed a mean-field game model with cap-and-trade regulation.
result Explicit solutions found through Riccati differential equations.
Unified analysis of DLNs using DMFT reveals dynamics of loss convergence and generalization trade-offs.
problem Understanding the overall dynamics of diagonal linear networks (DLNs) in neural network training.
method Dynamical Mean-Field Theory (DMFT) applied to DLNs.
result Derives low-dimensional effective process capturing high-dimensional gradient flow dynamics.
Paper analyzes SHB method for neural networks, proving stability, connectivity, and global convergence.
problem Theoretical understanding of SHB method for neural networks.
method Mean-field analysis of SHB dynamics related to a partial differential equation.
result SHB method converges to global optimum and exhibits stability and connectivity.
We consider learning two layer neural networks using stochastic gradient descent. The mean-field description of this learning dynamics approximates the evolution of the network weights by an evolution in the space of probability distributions in RD (where D is the number of parameters associated to each neuron). T…
An informed broker optimizes trading strategies in a market influenced by many traders.
problem Optimizing trading strategies for an informed broker in a market with many traders.
method Developed a mean-field game approach to derive equilibrium strategies for both the broker and traders.
result The broker's optimal strategy involves a Stackelberg equilibrium, leading and traders following.
Study shows how deep residual networks can be analyzed as shallow network ensembles for optimization.
problem Understanding why deep neural networks can be trained to zero loss despite non-convex optimization landscapes.
method Mean-field analysis of deep residual networks, focusing on their continuum limit as a two-layer network.
result Derives the first global convergence result for multilayer neural networks in the mean-field regime.
PDA method optimizes neural networks with global convergence rate analysis.
problem Quantitative convergence rate for neural network optimization in mean field regime.
method Particle dual averaging (PDA) method, combining Langevin algorithm and outer loop optimization.
result Established quantitative global convergence for two-layer mean field neural networks.
Study shows how neural networks generalize with minimal training data.
problem Understanding how neural networks generalize with limited data.
method Mean-field analysis of KL-regularized empirical risk minimization.
result Generalization error rate is O(1/n) for large n. New MFG model for MV portfolio management with peer-based risk aversion.
problem Time-inconsistent mean-variance portfolio management with peer-based risk aversion.
method Mean-field game, smooth regularization, fixed-point arguments, convergence analysis.
result Existence of mean-field equilibrium in time-inconsistent MFG.
One of the key socioeconomic phenomena to explain is the distribution of wealth. Bouchaud and Mézard have proposed an interesting model of economy [Bouchaud and Mézard (2000)] based on trade and investments of agents. In the mean-field approximation, the model produces a stationary wealth distribution with a power-law …
Study shows policy gradient convergence for entropy-regularized MDPs with neural nets in mean-field regime.
problem Global convergence of policy gradient for entropy-regularized MDPs with neural network approximation.
method Softmax policy with neural network approximation in mean-field regime, gradient flow in 2-Wasserstein metric, exponential convergence under sufficient regularization.
result Gradient flow converges exponentially fast to the unique stationary solution under sufficient regularization.
In this paper we consider a mean-field model of interacting diffusions for the monetary reserves in which the reserves are subjected to a self- and cross-exciting shock. This is motivated by the financial acceleration and fire sales observed in the market. We derive a mean-field limit using a weak convergence analysis …
We rigorously prove a central limit theorem for neural network models with a single hidden layer. The central limit theorem is proven in the asymptotic regime of simultaneously (A) large numbers of hidden units and (B) large numbers of stochastic gradient descent training iterations. Our result describes the neural net…
Study on PG learning for LQ MFC problems with common noise, proving convergence and sample complexity.
problem Optimal policy learning in LQ MFC problems with common noise and entropy regularization.
method Comprehensive error analysis of PG algorithms in both model-based and model-free settings.
result Global linear convergence and sample complexity of PG algorithms in model-free setting.
In this paper we study a continuous time equilibrium model of limit order book (LOB) in which the liquidity dynamics follows a non-local, reflected mean-field stochastic differential equation (SDE) with evolving intensity. Generalizing the basic idea of Ma et al. (2015), we argue that the frontier of the LOB (e.g., the…
The paper studies stability of mean-field variational inference for log-concave distributions.
problem Stability of mean-field variational inference for log-concave distributions.
method Novel approach via linearized optimal transport, lifting non-convex problem to convex optimization over transport maps.
result Dimension-free Lipschitz continuity of the MFVI optimizer with respect to the target distribution, measured in 2-Wasserstein distance.
Study analyzes adversarial training dynamics without data distribution assumptions.
problem Understanding training dynamics of adversarial training without data distribution assumptions.
method Mean field theory approach to analyze adversarial training in random deep neural networks.
result Upper bounds of adversarial loss derived empirically and theoretically.
Study uses Mean Field Game to analyze Bitcoin mining hashpower dynamics.
problem Analyzing the hashpower distribution in Bitcoin mining.
method Mean Field Game framework and master equation approach.
result Hashpower reaches steady state or increases with demand.
Study Transformer layers under cross-entropy training using mean field control.
problem Understanding the behavior of Transformer layers in cross-entropy training.
method Continuous-depth mean field control analysis, treating depth as time and layer parameters as controls.
result Derivation of a Pontryagin condition for the limiting population problem, involving the softmax residual.
Mean field theory has been successfully used to analyze deep neural networks (DNN) in the infinite size limit. Given the finite size of realistic DNN, we utilize the large deviation theory and path integral analysis to study the deviation of functions represented by DNN from their typical mean field solutions. The para…
Analysis of SGD for Gaussian mixture classification using dynamical mean-field theory.
problem Learning dynamics of SGD for a neural network classifying Gaussian mixture.
method Applying dynamical mean-field theory to track SGD dynamics in high dimensions.
result Reveals how SGD navigates the non-convex loss landscape.
The paper analyzes the dynamics of tokens in transformer models at moderate interaction levels.
problem Understanding the evolution of tokens in transformer models at moderate interaction levels.
method Modeling transformer models as a system of particles interacting in a mean-field way and studying the corresponding dynamics.
result Characterization and convergence of the limiting dynamics in different phases of the system.
Study mutual insurance market dynamics using mean field games.
problem Understanding strategic interactions and wealth distribution in mutual insurance companies.
method Extended mean field game framework, mean field forward-backward stochastic differential equations (MF-FBSDE), deep BSDE algorithm.
result Established global-in-time existence and uniqueness of Nash equilibrium strategy.
Rotates MFVI for better Gaussian approximations.
problem Improving variational approximations for complex distributions.
method Rotated coordinate system, PCA-based rotation, iterative Gaussianization.
result Significantly more accurate approximations with lower computational cost.
Paper adds Fisher Information to mean field optimization for faster convergence.
problem Mean field optimization in neural networks training.
method Developed energy-dissipation method and gradient flow on probability space.
result Marginal distributions converge exponentially to minimizer.
Analyzes SGD dynamics in two-layer networks, bridging different regimes.
problem Understanding SGD dynamics in high-dimensional and mean-field settings.
method Rigorous analysis via deterministic low-dimensional description of sufficient statistics.
result Infinite-width dynamics remains close to a low-dimensional subspace.