Generative model learns spin-glass dynamics and properties.
problem Complex behavior of many-body systems in statistical physics and computer science.
method Self-supervised learning with normalizing flows.
result Key physical and computational properties of spin-glasses are learned.
This thesis explores emergent intelligence in disordered systems like spin glasses and neural networks.
problem Understanding the principles behind emergent intelligent behaviors in disordered systems.
method Statistical physics approach to charting learning mechanisms and dynamics.
result Uncovering relationships between learning mechanisms and physical dynamics.
Study energy landscapes in glass models, focusing on Gaussian and spiked-tensor functions.
problem Characterize statistical properties and phase transitions of high-dimensional energy landscapes.
method Developed a Kac-Rice method framework to compute landscape complexity and analyze phase transitions rigorously.
result Characterized the ruggedness and arrangements of local minima in energy landscapes.
Injectivity of ReLU networks studied using statistical physics.
problem When can the input of a ReLU neural network be inferred from its output?
method Connection to spherical integral geometry and statistical physics.
result Replica symmetry-breaking theory and Gordon's min--max theorem provide insights into the injectivity threshold.
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
New method generates equilibrium glass configurations efficiently.
problem Sampling equilibrium configurations of amorphous materials is slow and difficult.
method Riemannian stochastic interpolation framework combining Riemannian stochastic interpolant and equivariant flow matching.
result Enforcing geometric and symmetry constraints significantly improves generative performance.
Bayesian model improves BCI performance for ALS users.
problem Classifying EEG signals for P300 BCIs with low SNR and complex correlations.
method GLASS model with Gaussian Latent channel and Sparse time-varying effects.
result GLASS substantially improves BCI performance in ALS users.
Bayesian hybrid models correct for missing physics in machine learning.
problem Systematic bias in machine learning models.
method Fusing physics-based insights with machine learning constructs, using Bayesian calibration and stochastic programming.
result Bayesian hybrid models outperform pure machine learning approaches with less data.
In this article we use the Mean-Variance Model in order to measure the current market state. In our study we take the approach of detecting the overall alignment of portfolios in the spin picture. The projection to the ground-states enables us to use physical observables in order to describe the current state of the ex…
Combining ML and physics for understanding glassy systems.
problem Understanding supercooled liquids and glasses due to disorder and non-equilibrium effects.
method Data-driven approach using machine learning with physical intuition.
result Building a phenomenological theory of disordered materials.
A statistical physics model for the time evolutions of stock portfolios is proposed. In this model the time series of price changes are coded into the sequences of up and down spins. The Hamiltonian of the system is introduced and is expressed by spin-spin interactions as in spin glass models of disordered magnetic sys…
We consider the problem of rational decision making in the presence of nonlinear constraints. By using tools borrowed from spin glass and random matrix theory, we focus on the portfolio optimisation problem. We show that the number of ``optimal'' solutions is generically exponentially large: rationality is thus de fact…
This article is a follow-up of a short essay that appeared in Nature 455, 1181 (2008) [arXiv:0810.5306]. It has become increasingly clear that the erratic dynamics of markets is mostly endogenous and not due to the rational processing of exogenous news. I elaborate on the idea that spin-glass type of problems, where th…
Recurring international financial crises have adverse socioeconomic effects and demand novel regulatory instruments or strategies for risk management and market stabilization. However, the complex web of market interactions often impedes rational decisions that would absolutely minimize the risk. Here we show that, for…
In an informal way, a number of thoughts on the financial crisis 2008 are presented from a physicist's viewpoint, considering the problem as a nonergodicity transition of a spin-glass type of system. Some tentative suggestions concerning the way out of the crisis are also discussed, concerning Keynesian "deficit spendi…
DNAMite creates interpretable, calibrated survival analysis models.
problem Limited interpretability in survival analysis models, especially for healthcare applications.
method Feature discretization and kernel smoothing in embedding module for flexible shape functions.
result DNAMite produces calibrated shape functions interpretable as contributions to cumulative incidence function.
New system learns programs from descriptions, outperforming brute-force methods.
problem Learning to write programs from descriptions.
method Intelligent search system using glass-box loss function.
result Significant improvements in accuracy and time compared to brute-force search.
GLASS Flows improves flow and diffusion model performance by optimizing sampling efficiency.
problem Efficiency bottleneck in sampling Markov transitions for flow and diffusion models.
method Introduces GLASS Flows, a new sampling paradigm that simulates a 'flow matching model within a flow matching model' to sample Markov transitions efficiently.
result Eliminates the trade-off between stochastic evolution and efficiency in large-scale text-to-image models.
We analyze operational risk in terms of a spin glass model. Several regimes are investigated, as a functions of the parameters that characterize the dynamics. The system is found to be robust against variations of these parameters. We unveil the presence of limit cycles and scrutinize the features of the asymptotic sta…
Many questions of fundamental interest in todays science can be formulated as inference problems: Some partial, or noisy, observations are performed over a set of variables and the goal is to recover, or infer, the values of the variables based on the indirect information contained in the measurements. For such problem…
HDMR provides insights into machine learning models, aiding in both prediction and explanation.
problem Understanding and interpreting complex machine learning models.
method High Dimensional Model Representation (HDMR) and its applications in machine learning.
result HDMR offers a glass box approach to machine learning models, enhancing both prediction and explanation.
We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…
Deep learning improves classification and characterization of amorphous materials.
problem Challenges in quantifying structure-property relationships and identifying structural features in amorphous materials.
method Application of convolutional neural networks and message passing neural networks to molecular dynamics simulations.
result Message passing neural networks outperform convolutional neural networks in classifying and characterizing amorphous materials.
Restricted Boltzmann Machines are described by the Gibbs measure of a bipartite spin glass, which in turn corresponds to the one of a generalised Hopfield network. This equivalence allows us to characterise the state of these systems in terms of retrieval capabilities, both at low and high load. We study the paramagnet…
Modeling financial markets with memory using fractional calculus and Brownian motion.
problem Capturing memory effects in financial markets using stochastic models.
method Fractional Langevin equation with colored noise generated by fractional Brownian motion.
result Anomalous marginal glass phase observed in some regions of the system.
Recent experimental advances in neuroscience have opened new vistas into the immense complexity of neuronal networks. This proliferation of data challenges us on two parallel fronts. First, how can we form adequate theoretical frameworks for understanding how dynamical network processes cooperate across widely disparat…
We analyze the statistics of daily price change of stock market in the framework of a statistical physics model for the collective fluctuation of stock portfolio. In this model the time series of price changes are coded into the sequences of up and down spins, and the Hamiltonian of the system is expressed by spin-spin…
RPNN-EOFs model improves time series forecasting accuracy.
problem Improving time series forecasting accuracy for complex systems.
method Combines higher-order neural networks with error-output feedbacks.
result RPNN-EOFs outperformed other models in forecasting the Mackey-Glass time series.
Improved DeepONet variants using Transformer cross-conditioning enhance PDE solution efficiency.
problem Solving partial differential equations efficiently and accurately.
method Transformer-inspired DeepONet variants with bidirectional cross-conditioning.
result Improved efficiency and accuracy compared to modified DeepONet, with variant effectiveness tied to PDE characteristics.
MPF method improves parameter estimation in probabilistic models.
problem Difficulty in fitting probabilistic models due to intractable partition function.
method Minimum Probability Flow (MPF) method for parameter estimation.
result MPF outperforms existing techniques in convergence time and accuracy.
We discuss the portfolio optimization problem with the obligatory deposits constraint. Recently it has been shown that as a consequence of this nonlinear constraint, the solution consists of an exponentially large number of optimal portfolios, completely different from each other, and extremely sensitive to any changes…
Derives TAP approximation for Bayesian linear regression.
problem Log-normalizing constant of posterior distribution in high-dimensional linear regression.
method Variational representation and Thouless-Anderson-Palmer approximation.
result Proves TAP approximation for spherical prior in proportional asymptotic regime.
New framework to understand and exploit curvature in deep learning loss landscapes.
problem Understanding and optimizing the loss landscape in deep learning models.
method New conceptual framework and techniques to estimate and exploit curvature of expected loss changes.
result Alice algorithm optimizes training by incorporating curvature terms and step bounds.
Study higher-order spin glass models for social network behavior with peer-group effects.
problem Modeling correlation phenomena on social networks with peer-group effects.
method Inference in higher-order Ising models to recover coefficients and peer-group effects.
result Strong concavity of log pseudo-likelihood implies statistical error rate of sqrt(d/n) for MPLE.
Paper proposes hybrid approach for transparent credit scoring models.
problem Lack of transparency in machine learning models limits their use in regulated environments.
method Post-hoc interpretation of black-box models guides feature selection, followed by training glass-box models.
result Reduces feature usage from 106 to 10 while maintaining comparable performance.
Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…
Paper calculates the benefit of unlabeled data in semi-supervised learning for Gaussian mixtures.
problem Improving performance in semi-supervised learning with unlabeled data.
method Analytical computation of Bayes risk gap between supervised and semi-supervised approaches for Gaussian mixture models.
result Quantifies the accuracy increase due to unlabeled data in a Bayesian setting.
Unified analysis of mean-field and convex hierarchies for estimating Ising model free energy.
problem Estimating the free energy of Ising models in various regimes.
method Unified analysis using mean-field approximation and convex hierarchies, proving tight bounds and optimality.
result Unified tight bounds for both mean-field and convex hierarchies, showing they are within O((n∥J∥F)2/3) of the free energy. New method clusters weighted networks and data with high accuracy.
problem Clustering weighted and directed networks with varying weights.
method Extends message passing algorithms to Potts model at critical temperature, solving marginals using belief propagation.
result Significantly outperforms existing algorithms in community detection and clustering tasks.
Researchers use statistical methods to infer transmission matrices in complex media.
problem Comprehending and exploiting photon scattering through disordered media.
method Pseudolikelihood decimation to learn the coupling matrix via random sampling.
result Transmission matrices can be inferred and used like normal optical elements.
Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.
problem Understanding learnability in overparameterized quadratic neural networks.
method Mapping ERM to convex matrix sensing with nuclear norm penalization.
result Characterization of global minima and precise generalization thresholds.
Spatio-temporal RBF neural networks improve chaotic time series prediction.
problem Predicting chaotic time series due to their dynamic nature.
method Proposes an spatio-temporal extension of RBF neural networks.
result Spatio-temporal RBF outperforms standard RBF in chaotic time series prediction.
Deep networks show less disorder and more monotonic behavior.
problem Understanding the disorder and monotonicity in deep neural networks.
method Computing frustration and analyzing near-monotonicity in signed graphs of deep networks.
result Deep networks exhibit less disorder and more monotonic behavior than expected.
Method uses elastic black-boxes to create interpretable models from complex ones.
problem Lack of trust and stability in opaque models and time-consuming feature engineering in interpretable models.
method Surrogate assisted feature extraction for model learning.
result Trains interpretable and accurate models without time-consuming feature engineering.
Deep learning model simulates noisy dynamical systems without distributional assumptions.
problem Simulating noisy dynamical systems with unknown distributional properties.
method DE-LSTM model using LSTM network for multi-label classification and penalized maximum log likelihood.
result DE-LSTM makes accurate predictions of probability distributions for noisy dynamical systems.
Automated feature engineering improves interpretable models without manual work.
problem Lack of interpretability in complex models causes trust and stability issues.
method Use elastic black-box models to create simpler, interpretable glass-box models.
result Extracted features from complex models improve linear model performance.
Study free energy in spherical spin glasses, proving universality dichotomy.
problem Analyzing free energy in spherical spin glass models with different tail exponents.
method Introduced a tail-adapted normalization and used universality dichotomy.
result Sharp universality dichotomy for free energy across different tail exponents.
Project aims to diagnose and track IPF disease using deep learning.
problem Diagnosing and tracking IPF disease in lung images.
method Developed a deep learning model for identifying honeycombing and ground glass patterns in HRCT lung images.
result Deep learning model achieved high accuracy in identifying specific lung regions.