Generative model learns spin-glass dynamics and properties.
problem Complex behavior of many-body systems in statistical physics and computer science.
method Self-supervised learning with normalizing flows.
result Key physical and computational properties of spin-glasses are learned.
We consider the problem of rational decision making in the presence of nonlinear constraints. By using tools borrowed from spin glass and random matrix theory, we focus on the portfolio optimisation problem. We show that the number of ``optimal'' solutions is generically exponentially large: rationality is thus de fact…
In an informal way, a number of thoughts on the financial crisis 2008 are presented from a physicist's viewpoint, considering the problem as a nonergodicity transition of a spin-glass type of system. Some tentative suggestions concerning the way out of the crisis are also discussed, concerning Keynesian "deficit spendi…
We analyze operational risk in terms of a spin glass model. Several regimes are investigated, as a functions of the parameters that characterize the dynamics. The system is found to be robust against variations of these parameters. We unveil the presence of limit cycles and scrutinize the features of the asymptotic sta…
The study examines the retrieval capabilities of RBMs and generalized Hopfield networks under various prior distributions.
problem Characterizing the state of RBMs and Hopfield networks under different prior distributions.
method Equivalence between RBMs and generalized Hopfield networks, analysis of phase transitions, and study of retrieval capabilities.
result The retrieval phase is robust and exists at low load for every pattern distribution.
This thesis explores emergent intelligence in disordered systems like spin glasses and neural networks.
problem Understanding the principles behind emergent intelligent behaviors in disordered systems.
method Statistical physics approach to charting learning mechanisms and dynamics.
result Uncovering relationships between learning mechanisms and physical dynamics.
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
MPF method improves parameter estimation in probabilistic models.
problem Difficulty in fitting probabilistic models due to intractable partition function.
method Minimum Probability Flow (MPF) method for parameter estimation.
result MPF outperforms existing techniques in convergence time and accuracy.
Derives TAP approximation for Bayesian linear regression.
problem Log-normalizing constant of posterior distribution in high-dimensional linear regression.
method Variational representation and Thouless-Anderson-Palmer approximation.
result Proves TAP approximation for spherical prior in proportional asymptotic regime.
Study higher-order spin glass models for social network behavior with peer-group effects.
problem Modeling correlation phenomena on social networks with peer-group effects.
method Inference in higher-order Ising models to recover coefficients and peer-group effects.
result Strong concavity of log pseudo-likelihood implies statistical error rate of sqrt(d/n) for MPLE.
A new framework trains RBMs deterministically for unsupervised learning.
problem Training and evaluation of RBMs with weak interactions.
method TAP mean-field approximation for generalized latent-variable models.
result Effective deterministic training and interesting unsupervised learning features demonstrated.
A statistical physics model for the time evolutions of stock portfolios is proposed. In this model the time series of price changes are coded into the sequences of up and down spins. The Hamiltonian of the system is introduced and is expressed by spin-spin interactions as in spin glass models of disordered magnetic sys…
Study free energy in spherical spin glasses, proving universality dichotomy.
problem Analyzing free energy in spherical spin glass models with different tail exponents.
method Introduced a tail-adapted normalization and used universality dichotomy.
result Sharp universality dichotomy for free energy across different tail exponents.
Injectivity of ReLU networks studied using statistical physics.
problem When can the input of a ReLU neural network be inferred from its output?
method Connection to spherical integral geometry and statistical physics.
result Replica symmetry-breaking theory and Gordon's min--max theorem provide insights into the injectivity threshold.
Researchers use statistical methods to infer transmission matrices in complex media.
problem Comprehending and exploiting photon scattering through disordered media.
method Pseudolikelihood decimation to learn the coupling matrix via random sampling.
result Transmission matrices can be inferred and used like normal optical elements.
In this article we use the Mean-Variance Model in order to measure the current market state. In our study we take the approach of detecting the overall alignment of portfolios in the spin picture. The projection to the ground-states enables us to use physical observables in order to describe the current state of the ex…
Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.
problem Understanding learnability in overparameterized quadratic neural networks.
method Mapping ERM to convex matrix sensing with nuclear norm penalization.
result Characterization of global minima and precise generalization thresholds.
New insights into optimal portfolios and ecological equilibria reveal surprising complexity.
problem Optimal portfolio construction with ecological constraints.
method Computational analysis of multispecies Lotka-Volterra equations with unit rank interaction matrices.
result Logarithm of the average number of solutions grows as \(N^{2/3}\), with most likely solutions being much smaller.
A model studies deep neural networks with binary synapses under connection removal.
problem Understanding the mechanism of deep learning from a theoretical perspective.
method Random active path model with diluted binary synapses under removal perturbation.
result A critical value of perturbation separates spin glass and paramagnetic phases, with the latter having poor generalization performance.
Deep neural networks are optimizable due to their multilayered structure.
problem Understanding why deep neural networks are easily optimizable despite their non-convex loss functions.
method Analysis of a spin glass model of deep neural networks using random matrix theory and algebraic geometry.
result The multilayered structure of deep neural networks leads to fewer stationary points, more clustered minima, and less severe tradeoffs between depth and width of minima.
The study reveals the spectral structure of attention layers and its implications for generalization.
problem Understanding the spectral structure and generalization of trained attention layers.
method Empirical risk minimization in a single-head tied-attention layer, using random matrix theory, spin-glass theory, and approximate message passing.
result Exact high-dimensional characterization of training and test error, interpolation and recovery thresholds, and spectrum of the key and query matrices.
Two methods find typical sums of log-normal variates in GBM trajectories.
problem Finding typical sums of log-normal variates in GBM trajectories.
method Mapped to spin glasses and used Ito calculus.
result Qualitative and quantitative agreement between methods.
We analyze the statistics of daily price change of stock market in the framework of a statistical physics model for the collective fluctuation of stock portfolio. In this model the time series of price changes are coded into the sequences of up and down spins, and the Hamiltonian of the system is expressed by spin-spin…
Financial markets are a classical example of complex systems as they comprise many interacting stocks. As such, we can obtain a surprisingly good description of their structure by making the rough simplification of binary daily returns. Spin glass models have been applied and gave some valuable results but at the price…
Minimal DAMs can recognize patterns in high noise, even with minimal data.
problem Pattern recognition in high noise conditions with limited data.
method Interpolating between DAMs and spin glasses, using minimal dense associative networks and extremizing quenched free-energy.
result Minimal DAMs can correctly recognize patterns even when the signal is very weak and noise is high.
In this paper I give a brief introduction to a family of simple but non-trivial models designed to increase our understanding of collective processes in markets, the so-called Minority Games, and their non-equilibrium statistical mathematical analysis. Since the most commonly studied members of this family define disor…
Study on detecting a single spike in high-dimensional data matrices.
problem Detecting a single unknown spike in high-dimensional rectangular data matrices.
method Analysis of likelihood ratio between spiked and null models, using Gaussian fluctuations and Talagrand's interpretation of cavity method.
result Asymptotic Gaussian fluctuations of the likelihood ratio below the BBP threshold, with open maximal parameter region.
Model explains surprising properties of neural network training landscapes.
problem Understanding the geometry of neural network loss landscapes.
method Developed a simple theoretical model of gradients and Hessians.
result Unified model accounts for 4 surprising properties of neural loss landscapes.
Recurring international financial crises have adverse socioeconomic effects and demand novel regulatory instruments or strategies for risk management and market stabilization. However, the complex web of market interactions often impedes rational decisions that would absolutely minimize the risk. Here we show that, for…
Study on the optimization landscape of half-rectified networks without simplifying assumptions.
problem Understanding the optimization landscape of deep neural networks, focusing on half-rectified networks.
method Theoretical analysis and empirical study of gradient descent on half-rectified networks.
result Proves that half-rectified single layer networks are asymptotically connected and provides bounds on the interplay between data distribution and model over-parametrization.
Gradient span algorithms show consistent progress in high dimensions.
problem Understanding consistent training progress in large machine learning models.
method Proving deterministic behavior of gradient span algorithms on Gaussian random functions.
result Gradient span algorithms have asymptotically deterministic behavior in high dimensions.
We propose a hierarchy for approximate inference based on the Dobrushin, Lanford, Ruelle (DLR) equations. This hierarchy includes existing algorithms, such as belief propagation, and also motivates novel algorithms such as factorized neighbors (FN) algorithms and variants of mean field (MF) algorithms. In particular, w…
Unified approach learns Ising models from various dynamics and data types.
problem Efficiently learning Ising model parameters from data under diverse conditions.
method Simple logistic regression approach, generalizing existing algorithms.
result Logistic regression succeeds in multiple new settings where assumptions are violated.
Finding minima of a real valued non-convex function over a high dimensional space is a major challenge in science. We provide evidence that some such functions that are defined on high dimensional domains have a narrow band of values whose pre-image contains the bulk of its critical points. This is in contrast with the…
New approach uses random matrix theory to understand tensor estimation performance.
problem Understanding the performance of estimators for low-rank signals in noisy tensors.
method Developed a new approach using random matrix theory to study random tensors.
result Discovered a fixed-point equation that matches the performance of the maximum likelihood estimator.
DNAMite creates interpretable, calibrated survival analysis models.
problem Limited interpretability in survival analysis models, especially for healthcare applications.
method Feature discretization and kernel smoothing in embedding module for flexible shape functions.
result DNAMite produces calibrated shape functions interpretable as contributions to cumulative incidence function.
Unified analysis of mean-field and convex hierarchies for estimating Ising model free energy.
problem Estimating the free energy of Ising models in various regimes.
method Unified analysis using mean-field approximation and convex hierarchies, proving tight bounds and optimality.
result Unified tight bounds for both mean-field and convex hierarchies, showing they are within O((n∥J∥F)2/3) of the free energy. New system learns programs from descriptions, outperforming brute-force methods.
problem Learning to write programs from descriptions.
method Intelligent search system using glass-box loss function.
result Significant improvements in accuracy and time compared to brute-force search.
This article is a follow-up of a short essay that appeared in Nature 455, 1181 (2008) [arXiv:0810.5306]. It has become increasingly clear that the erratic dynamics of markets is mostly endogenous and not due to the rational processing of exogenous news. I elaborate on the idea that spin-glass type of problems, where th…
GLASS Flows improves flow and diffusion model performance by optimizing sampling efficiency.
problem Efficiency bottleneck in sampling Markov transitions for flow and diffusion models.
method Introduces GLASS Flows, a new sampling paradigm that simulates a 'flow matching model within a flow matching model' to sample Markov transitions efficiently.
result Eliminates the trade-off between stochastic evolution and efficiency in large-scale text-to-image models.
Explains deep learning models and their geometric properties.
problem Understanding the geometric intuition behind deep learning models.
method Geometrical intuition and novel insights into loss surfaces of deep learning models.
result Deep neural networks carve out manifolds with multiplication neurons.
Bayesian model improves BCI performance for ALS users.
problem Classifying EEG signals for P300 BCIs with low SNR and complex correlations.
method GLASS model with Gaussian Latent channel and Sparse time-varying effects.
result GLASS substantially improves BCI performance in ALS users.
New method generates equilibrium glass configurations efficiently.
problem Sampling equilibrium configurations of amorphous materials is slow and difficult.
method Riemannian stochastic interpolation framework combining Riemannian stochastic interpolant and equivariant flow matching.
result Enforcing geometric and symmetry constraints significantly improves generative performance.
Study energy landscapes in glass models, focusing on Gaussian and spiked-tensor functions.
problem Characterize statistical properties and phase transitions of high-dimensional energy landscapes.
method Developed a Kac-Rice method framework to compute landscape complexity and analyze phase transitions rigorously.
result Characterized the ruggedness and arrangements of local minima in energy landscapes.
Paper presents a privacy-preserving algorithm for estimating peer effects using the Ising model.
problem Privacy concerns in estimating peer effects using network data.
method Developed a (ε,δ)-differentially private algorithm using Ising model. result Established regret bounds and validated performance on synthetic and real-world networks.
HDMR provides insights into machine learning models, aiding in both prediction and explanation.
problem Understanding and interpreting complex machine learning models.
method High Dimensional Model Representation (HDMR) and its applications in machine learning.
result HDMR offers a glass box approach to machine learning models, enhancing both prediction and explanation.
We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…
Study on Langevin dynamics for recovering planted signals in spiked matrix models.
problem Recovering a planted signal in spiked matrix models.
method Path-wise characterization of overlap using integro-differential equations and explicit formula derivation.
result Sharp phase transition in limiting overlap: positive in one regime, zero in another due to injected noise.