Statistical field theory aids in understanding deep learning complexities.
problem Complexity and lack of theoretical understanding in deep learning.
method Statistical field theory as a theoretical framework.
result Field theory provides insights into generalization, bias, and feature learning.
Simplified neural network EFTs reveal a single critical condition.
problem Understanding neuron statistics in neural networks at initialization.
method Diagrammatic approach to effective field theories (EFTs).
result A single condition governs criticality of all neuron preactivations.
Non-linear image reconstruction and signal analysis deal with complex inverse problems. To tackle such problems in a systematic way, I present information field theory (IFT) as a means of Bayesian, data based inference on spatially distributed signal fields. IFT is a statistical field theory, which permits the construc…
Renormalization in neural networks linked to quantum field theory.
problem Implementing renormalization in neural networks.
method Mapping neural networks to quantum field theory, applying renormalization techniques.
result Changing weight standard deviation corresponds to a renormalization flow.
Develops a dynamic mean field theory for reinforcement learning.
problem Finite state and action Bayesian reinforcement learning in large state spaces.
method Analogies with statistical physics, interpreting probabilities as couplings and values as spins, solving mean field equations.
result State-action values are statistically independent in the asymptotic state space limit, with exact or approximate equations for computation.
Extreme value theory enhances statistical learning extrapolation for rare events.
problem Challenges in traditional machine learning methods for extreme data.
method Asymptotic theory and statistical tools for tail behavior.
result Effective extrapolation methods for extreme quantiles and anomalies.
New algebraic geometry and statistical manifold connections proven.
problem Understanding the structure of statistical manifolds and their algebraic properties.
method Developed relations between algebraic geometry, information theory, and Topological Field Theory.
result Statistical pre-Frobenius manifolds form algebraic varieties and have hexagonal, isoclinic webs.
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function. To describe the network state statistics of the restricted Boltzmann machine, we develop an advanced mean field theory based on the Bethe approximation. Our theory provides an efficient message pas…
This work uses statistical mechanics to explain AI learning.
problem Understanding the statistical principles behind AI learning.
method Starting from sample concentration behaviors, the study applies statistical mechanics principles to AI and machine learning.
result Exponential families and statistical quantities are key in AI and machine learning.
The paper derives Einstein tensors for a family of α-connections on quasi-statistical manifolds.
problem Deriving Einstein tensors for a new family of connections.
method Developed mathematical foundations of statistical and quasi-statistical manifolds, including dual and equiaffine connections.
result Explicit expressions for curvatures and Einstein tensors of the α-connections.
Study on kernel methods in large-scale machine learning problems.
problem Large-scale machine learning with many interacting variables.
method Mean field limit analysis of kernels and their Hilbert spaces.
result Mean field convergence of empirical and infinite-sample solutions.
Statistical learning theory provides the theoretical basis for many of today's machine learning algorithms. In this article we attempt to give a gentle, non-technical overview over the key ideas and insights of statistical learning theory. We target at a broad audience, not necessarily machine learning researchers. Thi…
High-dimensional statistics advances in complex data domains.
problem Complex, rich datasets challenge traditional methods.
method Evolved to address sophisticated estimation and inference problems.
result Deepened connections with optimization, concentration, and information theory.
These notes gather recent results on robust statistical learning theory. The goal is to stress the main principles underlying the construction and theoretical analysis of these estimators rather than provide an exhaustive account on this rapidly growing field. The notes are the basis of lectures given at the conference…
Revises mean-field theory of Santa Fe model using kinetic theory.
problem Deriving a solid mathematical foundation for the Santa Fe model.
method Systematic derivation of BBGKY hierarchy from exact master equation.
result Explicit and closed-form solutions for mean-field equations.
The paper develops a new algorithm for RBMs using dynamical mean-field theory.
problem Learning in Restricted Boltzmann Machines (RBMs) with complex dependencies.
method Dynamical mean-field theory applied to RBMs with rectangular coupling matrices drawn from a bi-rotation invariant ensemble.
result The algorithm converges globally under a stability criterion, with rates matching numerical simulations.
Machine learning should incorporate maximum likelihood for better estimation.
problem Lack of rigorous foundational theory in machine learning.
method Integrate maximum likelihood estimation into machine learning models.
result Foundationally rigorous machine learning models have greater practical impact.
The differential geometry of Kenmotsu manifold is a valuable part of contact geometry with nice applications in other fields such as theoretical physics. In fact, its statistical counterpart, that is, Kenmotsu statistical manifold also has same importance as that of Kenmotsu manifold. Theoretical physicists have also b…
Paper connects RL and non-equilibrium statistical mechanics for entropy-regularized RL.
problem Obtaining analytical solutions for entropy-regularized RL.
method Mapping RL to non-equilibrium statistical mechanics, applying large deviation theory.
result Derives exact analytical results for optimal policy and dynamics in MDPs.
Paper explores the Jones polynomial and its impact on knot theory and related fields.
problem Exploring the Jones polynomial and its applications in knot theory.
method Recalling the Jones polynomial and its development, discussing its connections to various mathematical and physical contexts.
result The Jones polynomial has wide-ranging applications and connections in mathematics and physics.
Study finds Calabi-Yau models' operator spectra match random matrix theory.
problem Understanding spectra of Calabi-Yau sigma models.
method Numerical methods for Ricci-flat metrics, averaging over complex structure moduli space.
result Spectrum matches Gaussian orthogonal ensemble of random matrix theory.
The question of how best to estimate a continuous probability density from finite data is an intriguing open problem at the interface of statistics and physics. Previous work has argued that this problem can be addressed in a natural way using methods from statistical field theory. Here I describe new results that allo…
Quantum theory improves counting overlapping clusters.
problem Counting overlapping clusters in machine learning.
method Applied quantum theory using path integral technique.
result Quantum theory provides a robust statistical method for counting clusters.
The need to estimate smooth probability distributions (a.k.a. probability densities) from finite sampled data is ubiquitous in science. Many approaches to this problem have been described, but none is yet regarded as providing a definitive solution. Maximum entropy estimation and Bayesian field theory are two such appr…
To formulate the universal constraints of quantum statistics data of generic long-range entangled quantum systems, we introduce the geometric-topology surgery theory on spacetime manifolds where quantum systems reside, cutting and gluing the associated quantum amplitudes, specifically in 2+1 and 3+1 spacetime dimension…
Study on Alexander polynomials in braids, linking number theory and topology.
problem Distribution of Alexander polynomials in specific families of braids.
method Exploration of arithmetic invariants and analogies with number theory.
result New directions in arithmetic topology and statistics.
Global EQG sums boundary states over manifold diffeomorphism classes.
problem Summing boundary states over manifold diffeomorphism classes.
method Formulated as classical statistical physics, weights determined by general principles.
result Hartle-Hawking state as a probability measure.
Unified theory for deep and recurrent networks using Gaussian processes.
problem Understanding capabilities and limitations of different network architectures.
method Unified derivation of mean-field theory from statistical physics of disordered systems.
result Gaussian processes yield identical Gaussian kernels for both architectures at a single time point or layer.
New theory predicts deep neural networks can operate in an extended critical regime without fine-tuning.
problem Understanding the dynamics and computational principles of deep neural networks.
method Combining theories of heavy-tailed random matrices and non-equilibrium statistical physics.
result Deep neural networks can operate in an extended critical regime without fine-tuning parameters.
Paper uses SLT to improve model selection for SHM.
problem Model selection for SHM using data-based systems.
method Utilizes Statistical Learning Theory to rigorously estimate generalisation.
result Incorporating domain knowledge improves model generalisation.
Sharp statistical theory for conditional diffusion models.
problem Lack of theoretical foundation for conditional diffusion models.
method Sharp statistical theory with approximation of conditional score function.
result Sample complexity bound that adapts to data distribution smoothness.
In this comment on "Solving Statistical Mechanics Using Variational Autoregressive Networks" by Wu et al., we propose a subtle yet powerful modification of their approach. We show that the inherent sampling error of their method can be corrected by using neural network-based MCMC or importance sampling which leads to a…
Explains how geometry and statistics intertwine, focusing on information geometry.
problem Understanding the interplay between geometry and statistics.
method Introduces differential topology, geometry, probability, and (pre-)Frobenius manifolds.
result Discovers connections between geometry and statistics, particularly in information geometry.
Paper reviews algebraic research in machine learning theory.
problem Understanding phase transitions in machine learning models.
method Algebraic approaches in statistical mechanics.
result Algebraic methods are essential for analyzing machine learning models with singularities.
The theory of learning under the uniform distribution is rich and deep, with connections to cryptography, computational complexity, and the analysis of boolean functions to name a few areas. This theory however is very limited due to the fact that the uniform distribution and the corresponding Fourier basis are rarely …
We apply the geometric-topology surgery theory on spacetime manifolds to study the constraints of quantum statistics data in 2+1 and 3+1 spacetime dimensions. First, we introduce the fusion data for worldline and worldsheet operators capable creating anyon excitations of particles and strings, well-defined in gapped st…
Model financial time series using φ^4 quantum field theory.
problem Inaccuracies in Ising models for financial data.
method φ^4 quantum field theory with inhomogeneous couplings.
result Accurately reproduces higher-order statistics like market kurtosis.
Canonical quantization of abelian BF-type topological field theory coupled to extended sources on generic d-dimensional manifolds and with curved line bundles is studied. Sheaf cohomology is used to construct the appropriate topological extension of the action and the topological flux quantization conditions, in terms …
Statistical learning is the process of estimating an unknown probabilistic input-output relationship of a system using a limited number of observations. A statistical learning machine (SLM) is the algorithm, function, model, or rule, that learns such a process; and machine learning (ML) is the conventional name of this…
The simplest field theory description of the multivariate statistics of forward rate variations over time and maturities, involves a quadratic action containing a gradient squared rigidity term. However, this choice leads to a spurious kink (infinite curvature) of the normalized correlation function for coinciding matu…
Analysis of SGD for Gaussian mixture classification using dynamical mean-field theory.
problem Learning dynamics of SGD for a neural network classifying Gaussian mixture.
method Applying dynamical mean-field theory to track SGD dynamics in high dimensions.
result Reveals how SGD navigates the non-convex loss landscape.
Topological Quantum Field Theories (TQFTs) pertinent to some emergent low energy phenomena of condensed matter lattice models in 2+1 and 3+1D are explored. Many of our field theories are highly-interacting without free quadratic analogs. Some of our bosonic TQFTs can be regarded as the continuum field theory formulatio…
Study loop corrections in random feature models affecting training and test errors.
problem Analyzing loop corrections in random feature models to understand training and test errors.
method Statistical physics and effective field theory approach to study loop corrections.
result Derived loop corrections to training error, test error, and generalization gap.
Theory proposes neural networks can be initialized for optimal information transmission.
problem Optimizing neural networks for optimal information transmission and representation.
method Developed a corrected mean-field framework to study neural networks as information channels, proving mutual information maximization at dynamic isometry.
result Mutual information maximization is realized between inputs and propagated signals when neural networks are initialized at dynamic isometry.
Proves equations for high-dimensional gradient-based methods from Gaussian data.
problem High-dimensional asymptotics of gradient-based learning algorithms.
method Closed-form equations derived from dynamical mean-field theory.
result Equations match those from discretized DMFT for gradient flow.
Improves survey sampling with unbiased machine learning methods.
problem Design-consistent model-assisted estimation lacks a general theory for machine learning.
method Proposes a subsampling Rao-Blackwell method for design-unbiased estimation.
result Yields efficiency gains over standard methods while ensuring valid estimation.
We propose a new statistical model suitable for machine learning of systems with long distance correlations such as natural languages. The model is based on directed acyclic graph decorated by multi-linear tensor maps in the vertices and vector spaces in the edges, called tensor network. Such tensor networks have been …
Paper characterizes gradient descent in high-dimensional learning problems.
problem Understanding gradient descent dynamics in high-dimensional statistical learning.
method Non-asymptotic joint distributional characterization of gradient descent iterates and debiased statistics.
result Gradient descent iterates approximate normality after debiasing correction.