The paper models asset pricing in a partially observed market using mean field game theory and exponential quadratic Gaussian framework.
problem Asset pricing in a market with partial observation and heterogeneous agents.
method Mean field game theory, exponential quadratic Gaussian framework, Kalman-Bucy filtering theory.
result Characterization of equilibrium risk premium through mean field BSDE and construction of unobservable risk premium process.
Analysis of SGD for Gaussian mixture classification using dynamical mean-field theory.
problem Learning dynamics of SGD for a neural network classifying Gaussian mixture.
method Applying dynamical mean-field theory to track SGD dynamics in high dimensions.
result Reveals how SGD navigates the non-convex loss landscape.
Let Xbe a complex hyperelliptic curve of genus two equipped with the canonical metric ds2. We study mean field equations on complex hyperelliptic curves and show that the Gaussian curvature function of (X,ds2) determines an explicit solution to a mean field equation.
Rotates MFVI for better Gaussian approximations.
problem Improving variational approximations for complex distributions.
method Rotated coordinate system, PCA-based rotation, iterative Gaussianization.
result Significantly more accurate approximations with lower computational cost.
Gaussian-SVGD dynamics converge to Gaussian distributions under certain conditions.
problem Understanding the theoretical properties of SVGD, especially for Gaussian targets.
method Detailed theoretical study of Gaussian-SVGD dynamics, considering both mean-field PDE and discrete particle systems.
result Gaussian-SVGD dynamics converge linearly to the Gaussian distribution closest to the target in KL divergence.
Unified theory for deep and recurrent networks using Gaussian processes.
problem Understanding capabilities and limitations of different network architectures.
method Unified derivation of mean-field theory from statistical physics of disordered systems.
result Gaussian processes yield identical Gaussian kernels for both architectures at a single time point or layer.
Proves equations for high-dimensional gradient-based methods from Gaussian data.
problem High-dimensional asymptotics of gradient-based learning algorithms.
method Closed-form equations derived from dynamical mean-field theory.
result Equations match those from discretized DMFT for gradient flow.
Paper extends multi-task Gaussian Cox processes for heterogeneous tasks.
problem Modeling multiple heterogeneous correlated tasks jointly.
method Data augmentation and mean-field approximation for non-conjugate Bayesian inference.
result Demonstrates improved performance and inference on synthetic and real data.
A new MFG framework for evolving clusters from Gaussian mixtures.
problem Evolutionary clustering of time-dependent Gaussian mixtures.
method Control-theoretic framework based on Mean Field Games (MFG) with coupled HJB and Fokker-Planck systems.
result MFG dynamics recover classical EM algorithm trajectories with mass conservation.
This paper analyzes MFVBI for GMM using statistical mechanics.
problem Approximate fast computation of Gaussian Mixture Model.
method Statistical mechanics and MFVBI applied to GMM.
result Rigorous analysis and mathematical foundation for MFVBI applied to GMM.
Compact parameterization improves Bayesian neural network performance.
problem Improving performance of Bayesian neural networks using variational methods.
method Restricting variational distribution to a k-tied Normal distribution with low-rank factorization.
result Compact parameterization improves signal-to-noise ratio and convergence speed.
This paper studies gradient flows for sampling using various metrics and their affine invariance.
problem Sampling from probability distributions with unknown normalizations.
method Gradient flows in the space of probability measures, focusing on Kullback-Leibler divergence and affine invariance of metrics.
result Gradient flows of Kullback-Leibler divergence do not depend on the normalization constant, and affine invariance is achieved for certain metrics.
Model asset pricing with habit formation in a large market.
problem Understanding asset pricing in large heterogeneous markets with habit formation.
method Mean field game theory and quadratic-growth mean field BSDEs.
result Derives a semi-analytic solution for asset pricing model.
Variational inference with a factorized Gaussian posterior estimate is a widely used approach for learning parameters and hidden variables. Empirically, a regularizing effect can be observed that is poorly understood. In this work, we show how mean field inference improves generalization by limiting mutual information …
New GM layers improve neural network performance.
problem Improving neural network performance.
method Employing Gaussian mixture models and Wasserstein gradient flows.
result GM layers achieve comparable performance to two-layer networks.
A new method for uncertainty estimation in neural networks using Gaussian-softmax integration.
problem Quantifying uncertainty in neural network predictions.
method Proposes a single-model approach integrating Gaussian distribution with softmax outputs, using mean-field approximation.
result Competitive performance on uncertainty estimation tasks and outperforms many methods on out-of-distribution detection.
A new method for steering large agent populations efficiently.
problem Controlling the configuration of a swarm of identical, interacting cooperative agents.
method Mean-Field Schrodinger Bridges with Gaussian Mixture Models.
result A highly efficient parameterization to approximate optimal solutions of the MFSB problem in closed form.
A Gaussian restricted Boltzmann machine (GRBM) is a Boltzmann machine defined on a bipartite graph and is an extension of usual restricted Boltzmann machines. A GRBM consists of two different layers: a visible layer composed of continuous visible variables and a hidden layer composed of discrete hidden variables. In th…
We analyze training dynamics in Gaussian mixture models using a comparison theorem.
problem Analyzing training algorithms with Gaussian mixture data.
method Applying a Gaussian comparison theorem to a specific family of training algorithms.
result Validated dynamic mean-field expressions and provided iterative refinement schemes.
Nonnegative Boltzmann machines (NNBMs) are recurrent probabilistic neural network models that can describe multi-modal nonnegative data. NNBMs form rectified Gaussian distributions that appear in biological neural network models, positive matrix factorization, nonnegative matrix factorization, and so on. In this paper,…
Enhanced ensemble filters use machine learning to improve accuracy in filtering models.
problem Accuracy limitations of traditional ensemble Kalman filters.
method Introduces a measure neural mapping (MNM) to map joint predicted state and observation to updated state estimates.
result Superior root-mean-square-error performance compared to leading methods in filtering models.
Paper solves curvature prescription problem on surfaces with boundary.
problem Prescribing Gaussian and geodesic curvatures on compact surfaces with boundary.
method Mean field-type formulation and variational techniques.
result Existence results for positive, zero, and negative Euler characteristics.
We identify a new variational inference scheme for dynamical systems whose transition function is modelled by a Gaussian process. Inference in this setting has either employed computationally intensive MCMC methods, or relied on factorisations of the variational posterior. As we demonstrate in our experiments, the fact…
Muon dynamics study uses spectral Wasserstein flow for optimization stability.
problem Optimizing deep learning models with gradient normalization.
method Introduces Spectral Wasserstein distances for matrix flows, proving equivalence with Benamou--Brenier formulation.
result Gradient-flow interpretation of mean-field normalized training dynamics.
Beta process is the standard nonparametric Bayesian prior for latent factor model. In this paper, we derive a structured mean-field variational inference algorithm for a beta process non-negative matrix factorization (NMF) model with Poisson likelihood. Unlike the linear Gaussian model, which is well-studied in the non…
ALO-CV approximates leave-one-out error in proportional regime.
problem Estimating generalization error in high-dimensional settings.
method Developed new analysis for ALO-CV, showed consistency under strong convexity.
result ALO-CV approximates leave-one-out error up to negligible error.
Develops asset pricing models with mean field game theory for heterogeneous agents.
problem Tackles equilibrium asset pricing in incomplete markets with heterogeneous agents.
method Uses mean field game theory and mean field backward stochastic differential equations (BSDEs).
result Derives equilibrium risk premium and shows market clearing in the large population limit.
Gradient matching with Gaussian processes is a promising tool for learning parameters of ordinary differential equations (ODE's). The essence of gradient matching is to model the prior over state variables as a Gaussian process which implies that the joint distribution given the ODE's and GP kernels is also Gaussian di…
Deep learning relies on good initialization schemes and hyperparameter choices prior to training a neural network. Random weight initializations induce random network ensembles, which give rise to the trainability, training speed, and sometimes also generalization ability of an instance. In addition, such ensembles pro…
Improved model for non-smooth signals with complex spectra.
problem Current models struggle with non-smooth signals and complex spectral structures.
method CGPCM and RGPCM models with causality and Bayesian nonparametric interpretations, improved variational inference.
result Proposed models show better performance on synthetic and real-world data.
Ensemble Kalman Filter improves GPSSM inference for online learning.
problem Non-mean-field variational inference issues in GPSSM.
method Combining EnKF with NMF variational inference.
result Improved online learning performance and data-fitting accuracy.
Wide BNNs with odd activations fail to approximate data under mean-field inference.
problem Theoretical limitations of mean-field variational inference in wide, deep Bayesian neural networks.
method Analysis of mean-field variational inference in fully-connected BNNs with odd activation functions and Gaussian likelihood.
result The optimal mean-field variational posterior predictive distribution converges to the prior predictive distribution as network width increases.
New control methods improve dynamic measure transport paths.
problem Improving paths for dynamic measure transport.
method Connecting mean-field games to optimization problems for learning paths, advocating for smoothness of velocities.
result Our method recovers more efficient and smooth transport models compared to untilted paths.
Explaining how overparametrized neural networks simultaneously achieve low risk and zero empirical risk on benchmark datasets is an open problem. PAC-Bayes bounds optimized using variational inference (VI) have been recently proposed as a promising direction in obtaining non-vacuous bounds. We show empirically that thi…
We rigorously prove a central limit theorem for neural network models with a single hidden layer. The central limit theorem is proven in the asymptotic regime of simultaneously (A) large numbers of hidden units and (B) large numbers of stochastic gradient descent training iterations. Our result describes the neural net…
MF-PID uses interacting samples to efficiently transport probability mass.
problem Efficiently transporting probability mass in generative models.
method Introducing Mean-Field Path-Integral Diffusion (MF-PID) where samples become interacting agents.
result MF-PID achieves 19-24% reductions in control energy for demand-response control of energy systems.
Researchers develop methods to recover agent behavior from sparse data using Gaussian processes.
problem Recovering agent behavior from limited, noisy data in potential mean field games.
method Two Gaussian process-based frameworks: inf-sup formulation and bilevel approach.
result Surrogate MFG models can accurately reproduce observed data, even when prior information is limited.
The Perona-Malik model has been very successful at restoring images from noisy input. In this paper, we reinterpret the Perona-Malik model in the language of Gaussian scale mixtures and derive some extensions of the model. Specifically, we show that the expectation-maximization (EM) algorithm applied to Gaussian scale …
We reconsider a nonparametric density model based on Gaussian processes. By augmenting the model with latent Pólya--Gamma random variables and a latent marked Poisson process we obtain a new likelihood which is conjugate to the model's Gaussian process prior. The augmented posterior allows for efficient inference by Gi…
Bayesian neural networks can be simplified by parameterizing weights as rank-r matrices, reducing parameter count and improving performance.
problem High parameter count in standard Bayesian neural networks.
method Parameterize weights as W=ABop with A∈Rmimesr, B∈Rnimesr, inducing a singular posterior. result PAC-Bayes generalization bounds and loss bounds show improved performance with fewer parameters.
Deep Gaussian processes (DGPs) can model complex marginal densities as well as complex mappings. Non-Gaussian marginals are essential for modelling real-world data, and can be generated from the DGP by incorporating uncorrelated variables to the model. Previous work on DGP models has introduced noise additively and use…
Bayesian neural networks ignore data in infinite units limit.
problem Pathological behavior of posterior in over-parameterized networks.
method Mean-field variational inference in infinite hidden units limit.
result Posterior mean converges to zero, ignoring data.
Mean-field neural nets approximate functions using a free energy functional and controlled dynamics.
problem Function approximation by two-layer neural nets in the mean-field regime.
method Phrasing function approximation as global minimization of a free energy functional, examining dynamics in the space of probability measures over weights.
result Characterization of the unique global minimizer and dynamics achieving it, including the Föllmer drift.
A scalable factorized Gaussian process VAE for faster inference.
problem Inference bottlenecks in Gaussian process VAEs.
method Factorizes latent kernel across auxiliary features, leveraging independence.
result Significant speed-up in inference time (in theory and practice).
Learning in Gaussian Process models occurs through the adaptation of hyperparameters of the mean and the covariance function. The classical approach entails maximizing the marginal likelihood yielding fixed point estimates (an approach called \textit{Type II maximum likelihood} or ML-II). An alternative learning proced…
New theory predicts deep neural networks can operate in an extended critical regime without fine-tuning.
problem Understanding the dynamics and computational principles of deep neural networks.
method Combining theories of heavy-tailed random matrices and non-equilibrium statistical physics.
result Deep neural networks can operate in an extended critical regime without fine-tuning parameters.
Analyzes SGD dynamics in two-layer networks, bridging different regimes.
problem Understanding SGD dynamics in high-dimensional and mean-field settings.
method Rigorous analysis via deterministic low-dimensional description of sufficient statistics.
result Infinite-width dynamics remains close to a low-dimensional subspace.
Study SGD dynamics in high-dimensional models, revealing consistent behavior across different batch sizes and learning rates.
problem Understanding SGD dynamics in high-dimensional multi-index models.
method Asymptotic analysis of SGD, developing mean-field equations and Gaussian diffusion approximations.
result Consistent SGD dynamics across different batch sizes and learning rates, distinct from gradient flow and online SGD.