We consider the problem of rational decision making in the presence of nonlinear constraints. By using tools borrowed from spin glass and random matrix theory, we focus on the portfolio optimisation problem. We show that the number of ``optimal'' solutions is generically exponentially large: rationality is thus de fact…
Statistical learning theory connects to spin glass models via Rademacher complexity and replica theory.
problem Bounding generalization gap in statistical learning theory.
method Linking Rademacher complexity in statistical learning to synthetic models in statistical physics.
result Rademacher complexity is closely related to ground state energy in spin glass models.
Project aims to diagnose and track IPF disease using deep learning.
problem Diagnosing and tracking IPF disease in lung images.
method Developed a deep learning model for identifying honeycombing and ground glass patterns in HRCT lung images.
result Deep learning model achieved high accuracy in identifying specific lung regions.
In this article we use the Mean-Variance Model in order to measure the current market state. In our study we take the approach of detecting the overall alignment of portfolios in the spin picture. The projection to the ground-states enables us to use physical observables in order to describe the current state of the ex…
Injectivity of ReLU networks studied using statistical physics.
problem When can the input of a ReLU neural network be inferred from its output?
method Connection to spherical integral geometry and statistical physics.
result Replica symmetry-breaking theory and Gordon's min--max theorem provide insights into the injectivity threshold.
Generative model learns spin-glass dynamics and properties.
problem Complex behavior of many-body systems in statistical physics and computer science.
method Self-supervised learning with normalizing flows.
result Key physical and computational properties of spin-glasses are learned.
Recurring international financial crises have adverse socioeconomic effects and demand novel regulatory instruments or strategies for risk management and market stabilization. However, the complex web of market interactions often impedes rational decisions that would absolutely minimize the risk. Here we show that, for…
New algorithm nearly achieves ground state free energy of SK model.
problem Determining the ground state free energy of the SK model.
method Controlled Loosening-up (CLuP) algorithm applied to SK models.
result Achieves ground state free energy of ~0.76 for n in the thousands.
In an informal way, a number of thoughts on the financial crisis 2008 are presented from a physicist's viewpoint, considering the problem as a nonergodicity transition of a spin-glass type of system. Some tentative suggestions concerning the way out of the crisis are also discussed, concerning Keynesian "deficit spendi…
DNAMite creates interpretable, calibrated survival analysis models.
problem Limited interpretability in survival analysis models, especially for healthcare applications.
method Feature discretization and kernel smoothing in embedding module for flexible shape functions.
result DNAMite produces calibrated shape functions interpretable as contributions to cumulative incidence function.
GLASS Flows improves flow and diffusion model performance by optimizing sampling efficiency.
problem Efficiency bottleneck in sampling Markov transitions for flow and diffusion models.
method Introduces GLASS Flows, a new sampling paradigm that simulates a 'flow matching model within a flow matching model' to sample Markov transitions efficiently.
result Eliminates the trade-off between stochastic evolution and efficiency in large-scale text-to-image models.
We analyze operational risk in terms of a spin glass model. Several regimes are investigated, as a functions of the parameters that characterize the dynamics. The system is found to be robust against variations of these parameters. We unveil the presence of limit cycles and scrutinize the features of the asymptotic sta…
Bayesian model improves BCI performance for ALS users.
problem Classifying EEG signals for P300 BCIs with low SNR and complex correlations.
method GLASS model with Gaussian Latent channel and Sparse time-varying effects.
result GLASS substantially improves BCI performance in ALS users.
New method generates equilibrium glass configurations efficiently.
problem Sampling equilibrium configurations of amorphous materials is slow and difficult.
method Riemannian stochastic interpolation framework combining Riemannian stochastic interpolant and equivariant flow matching.
result Enforcing geometric and symmetry constraints significantly improves generative performance.
Recently proposed models which learn to write computer programs from data use either input/output examples or rich execution traces. Instead, we argue that a novel alternative is to use a glass-box loss function, given as a program itself that can be directly inspected. Glass-box optimization covers a wide range of pro…
We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of futu…
Deep learning improves classification and characterization of amorphous materials.
problem Challenges in quantifying structure-property relationships and identifying structural features in amorphous materials.
method Application of convolutional neural networks and message passing neural networks to molecular dynamics simulations.
result Message passing neural networks outperform convolutional neural networks in classifying and characterizing amorphous materials.
Prediction and explanation are key objects in supervised machine learning, where predictive models are known as black boxes and explanatory models are known as glass boxes. Explanation provides the necessary and sufficient information to interpret the model output in terms of the model input. It includes assessments of…
Restricted Boltzmann Machines are described by the Gibbs measure of a bipartite spin glass, which in turn corresponds to the one of a generalised Hopfield network. This equivalence allows us to characterise the state of these systems in terms of retrieval capabilities, both at low and high load. We study the paramagnet…
RPNN-EOFs model improves time series forecasting accuracy.
problem Improving time series forecasting accuracy for complex systems.
method Combines higher-order neural networks with error-output feedbacks.
result RPNN-EOFs outperformed other models in forecasting the Mackey-Glass time series.
This thesis explores emergent intelligence in disordered systems like spin glasses and neural networks.
problem Understanding the principles behind emergent intelligent behaviors in disordered systems.
method Statistical physics approach to charting learning mechanisms and dynamics.
result Uncovering relationships between learning mechanisms and physical dynamics.
MPF method improves parameter estimation in probabilistic models.
problem Difficulty in fitting probabilistic models due to intractable partition function.
method Minimum Probability Flow (MPF) method for parameter estimation.
result MPF outperforms existing techniques in convergence time and accuracy.
Derives TAP approximation for Bayesian linear regression.
problem Log-normalizing constant of posterior distribution in high-dimensional linear regression.
method Variational representation and Thouless-Anderson-Palmer approximation.
result Proves TAP approximation for spherical prior in proportional asymptotic regime.
Paper proposes a weak supervision technique for CNN semantic segmentation of lung diseases using partially annotated data.
problem Creating annotated datasets for semantic segmentation of lung diseases is laborious and time-consuming.
method Proposes a weak supervision technique that utilizes partially annotated datasets to improve CNN semantic segmentation accuracy.
result Significantly improved segmentation accuracy using partially annotated datasets.
New framework to understand and exploit curvature in deep learning loss landscapes.
problem Understanding and optimizing the loss landscape in deep learning models.
method New conceptual framework and techniques to estimate and exploit curvature of expected loss changes.
result Alice algorithm optimizes training by incorporating curvature terms and step bounds.
Study higher-order spin glass models for social network behavior with peer-group effects.
problem Modeling correlation phenomena on social networks with peer-group effects.
method Inference in higher-order Ising models to recover coefficients and peer-group effects.
result Strong concavity of log pseudo-likelihood implies statistical error rate of sqrt(d/n) for MPLE.
Paper proposes hybrid approach for transparent credit scoring models.
problem Lack of transparency in machine learning models limits their use in regulated environments.
method Post-hoc interpretation of black-box models guides feature selection, followed by training glass-box models.
result Reduces feature usage from 106 to 10 while maintaining comparable performance.
Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the stati…
Researchers use statistical methods to infer transmission matrices in complex media.
problem Comprehending and exploiting photon scattering through disordered media.
method Pseudolikelihood decimation to learn the coupling matrix via random sampling.
result Transmission matrices can be inferred and used like normal optical elements.
Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.
problem Understanding learnability in overparameterized quadratic neural networks.
method Mapping ERM to convex matrix sensing with nuclear norm penalization.
result Characterization of global minima and precise generalization thresholds.
Spatio-temporal RBF neural networks improve chaotic time series prediction.
problem Predicting chaotic time series due to their dynamic nature.
method Proposes an spatio-temporal extension of RBF neural networks.
result Spatio-temporal RBF outperforms standard RBF in chaotic time series prediction.
Method uses elastic black-boxes to create interpretable models from complex ones.
problem Lack of trust and stability in opaque models and time-consuming feature engineering in interpretable models.
method Surrogate assisted feature extraction for model learning.
result Trains interpretable and accurate models without time-consuming feature engineering.
Automated feature engineering improves interpretable models without manual work.
problem Lack of interpretability in complex models causes trust and stability issues.
method Use elastic black-box models to create simpler, interpretable glass-box models.
result Extracted features from complex models improve linear model performance.
Study free energy in spherical spin glasses, proving universality dichotomy.
problem Analyzing free energy in spherical spin glass models with different tail exponents.
method Introduced a tail-adapted normalization and used universality dichotomy.
result Sharp universality dichotomy for free energy across different tail exponents.
The study reveals the spectral structure of attention layers and its implications for generalization.
problem Understanding the spectral structure and generalization of trained attention layers.
method Empirical risk minimization in a single-head tied-attention layer, using random matrix theory, spin-glass theory, and approximate message passing.
result Exact high-dimensional characterization of training and test error, interpolation and recovery thresholds, and spectrum of the key and query matrices.
This paper gives necessary and sufficient conditions that the free product with amalgamation of circularly-ordered groups admit a circular ordering extending the given orderings of the factors. Our result follows from establishing a categorical framework that allows the problem to be restated in terms of amalgamating c…
We show that every irreducible toroidal integer homology sphere graph manifold has a left-orderable fundamental group. This is established by way of a specialization of a result due to Bludov and Glass for the almagamated products that arise, and in this setting work of Boyer, Rolfsen and Wiest may be applied. Our resu…
We investigate the bi-orderability of two-bridge knot groups and the groups of knots with 12 or fewer crossings by applying recent theorems of Chiswell, Glass and Wilson. Amongst all knots with 12 or fewer crossings (of which there are 2977), previous theorems were only able to determine bi-orderability of 599 of the c…
New insights into optimal portfolios and ecological equilibria reveal surprising complexity.
problem Optimal portfolio construction with ecological constraints.
method Computational analysis of multispecies Lotka-Volterra equations with unit rank interaction matrices.
result Logarithm of the average number of solutions grows as \(N^{2/3}\), with most likely solutions being much smaller.
DIGIT is a low-cost tactile sensor for in-hand manipulation.
problem Difficulty in sensing contact forces limits robotic manipulation.
method DIGIT miniaturizes and improves a vision-based tactile sensor.
result DIGIT enables better control of interactions with the environment.
Financial markets are a classical example of complex systems as they comprise many interacting stocks. As such, we can obtain a surprisingly good description of their structure by making the rough simplification of binary daily returns. Spin glass models have been applied and gave some valuable results but at the price…
ParamBoost uses gradient boosting to create interpretable non-linear models with constraints.
problem Creating interpretable non-linear models with expert knowledge constraints.
method Gradient Boosting of cubic polynomials with specified constraints.
result ParamBoost outperforms state-of-the-art GAMs in real-world datasets.
We study the problem of detecting the presence of a single unknown spike in a rectangular data matrix, in a high-dimensional regime where the spike has fixed strength and the aspect ratio of the matrix converges to a finite limit. This setup includes Johnstone's spiked covariance model. We analyze the likelihood ratio …
Minimal DAMs can recognize patterns in high noise, even with minimal data.
problem Pattern recognition in high noise conditions with limited data.
method Interpolating between DAMs and spin glasses, using minimal dense associative networks and extremizing quenched free-energy.
result Minimal DAMs can correctly recognize patterns even when the signal is very weak and noise is high.
Data science principles enhance AI interpretability for better user control.
problem Risks from opaque AI models without clear impacts.
method Synthesizes principles from interpretability literature, emphasizing audience goals.
result Illustrates basic techniques and criteria for evaluating interpretability.
Dense Associative Memories outperform classical networks in robustness and signal processing.
problem Improving neural network performance in adversarial attacks and weak signal processing.
method Relaxing replica symmetry in statistical mechanics of spin glasses to analyze unsupervised and supervised learning.
result Explicit analytical investigation of phase diagrams and storage capacities for Dense Associative Memories.
New algorithm for mean estimation in add-remove model achieves optimal error.
problem Mean estimation in add-remove model of differential privacy.
method Proposed new algorithm achieving min-max optimality.
result Achieves best possible constant in mean squared error for all ε.