The paper models market crashes as phase transitions, finding dynamic transitions offer better predictions.
problem Understanding and predicting extreme financial events like market crashes.
method Employing phase transition theory, focusing on endogenous crashes, and comparing DPT, CPT, and SPT.
result Dynamic phase transitions provide more accurate predictions of market crashes compared to critical and stochastic models.
Data-driven approach learns effective equations for phase field interfaces.
problem Learning accurate equations for phase field interface dynamics.
method Data-driven identification of partial differential equations from phase field data.
result Data-driven equations outperform analytical approximations in certain regimes.
Gradient descent dynamics in quadratic regression models are analyzed, revealing five phases: monotonic, catapult, periodic, chaotic, and divergent.
problem Analyzing the dynamics of gradient descent in quadratic regression models.
method Fine-grained bifurcation analysis of gradient descent dynamics using a cubic map parameterized by the step-size.
result Gradient descent dynamics in quadratic regression models exhibit five distinct phases: monotonic, catapult, periodic, chaotic, and divergent.
Study shows how anisotropic data affects learning dynamics in phase retrieval.
problem Understanding learning dynamics in phase retrieval with anisotropic Gaussian inputs.
method Developed a tractable reduction to reveal a three-phase trajectory and derived scaling laws.
result Found that anisotropy leads to a three-phase trajectory: fast escape, slow convergence, and spectral-tail learning.
Study connects symmetries in dynamical systems to phase plane representations.
problem Understanding symmetries in dynamical systems and their phase plane realizations.
method Analysis of symmetries in differential equations and phase plane representations, establishing correspondence and lifting conditions.
result Every symmetry generator in one formulation corresponds uniquely to a generator in the other, with a lifting condition to solve.
In order to model volatile real-world network behavior, we analyze phase-flipping dynamical scale-free network in which nodes and links fail and recover. We investigate how stochasticity in a parameter governing the recovery process affects phase-flipping dynamics, and find the probability that no more than q% of nodes…
New model explains market dynamics with phase transitions and non-linear interactions.
problem Understanding complex multi-asset market dynamics with phase transitions.
method Developed a Multi-Asset Non-Equilibrium Skew (MANES) model based on Langevin dynamics and McKean-Vlasov equation.
result The model accurately predicts market returns and phase transitions in both benign and distressed markets.
New model explains deep learning performance at large learning rates.
problem Understanding deep learning performance at different learning rates.
method Developed neural networks with solvable training dynamics.
result Large learning rates lead to convergence to flatter minima.
Study of two-layer ReLU neural network phase diagram at infinite-width limit.
problem Characterize the dynamical regimes of two-layer ReLU neural networks.
method Combining experimental and theoretical approaches, including phase diagram analogy.
result Identification of three regimes: linear, critical, and condensed.
This paper resolves Breiman's dilemma in neural networks by analyzing phase transitions of margin dynamics.
problem Breiman's dilemma in neural networks: uniform margin improvement does not guarantee reduced generalization errors.
method Revisiting Breiman's dilemma in deep neural networks with spectrally normalized margins, analyzing phase transitions of normalized margin distributions.
result Margin-based generalization bounds can predict test error trends during training phase transitions.
Enhances neural network dynamics to boost computational capacity.
problem Improving computational capacity of neural networks.
method Introducing Phase Transition Adaptation to drive system dynamics towards edge of stability.
result Consistently achieves enhancement in computational capacity over multiple datasets.
We present a simple model of firm rating evolution. We consider two sources of defaults: individual dynamics of economic development and Potts-like interactions between firms. We show that such a defined model leads to phase transition, which results in collective defaults. The existence of the collective phase depends…
We study the phase transition of dynamical herd behaviors for the yen-dollar exchange rate in the Japanese financial market. It is obtained that the probability distribution of returns satisfies the power-law behavior with three different values of the scaling exponent 3.11 (one time lag τ = 1 minute), 2.81 (30 minut…
Minimizing non-convex and high-dimensional objective functions is challenging, especially when training modern deep neural networks. In this paper, a novel approach is proposed which divides the training process into two consecutive phases to obtain better generalization performance: Bayesian sampling and stochastic op…
Community-based system dynamics improves ML fairness by involving excluded stakeholders.
problem Bias in ML system development during problem formulation.
method Community-based system dynamics (CBSD) for stakeholder participation.
result CBSD facilitates deeper problem understanding and bias mitigation.
Noise Injection probes deep learning dynamics during training phases.
problem Understanding the learning mechanism of deep neural networks.
method Noise Injection Nodes (NINs) are used to perturb DNN architectures without altering the optimization algorithm.
result Distinct training phases are observed based on the scale of injected noise.
Study of phase separation and geometry on a closed elastic curve, including dynamics and free energy minimization.
problem Free energy and dynamics of a closed elastic filament coupled to a scalar concentration field.
method Analytical and numerical simulations of coupled Willmore flow and Cahn--Hilliard gradient flow on differential geometry.
result Qualitative changes in free energy landscape due to closure constraint, leading to metastable and stable multi-domain morphologies.
Gradient descent dynamics in neural networks show quenching and activation phases.
problem Understanding training dynamics in neural networks.
method Numerical and phenomenological study of gradient descent algorithm for two-layer neural networks.
result Gradient descent dynamics exhibit quenching and activation phases in under-parametrized networks.
This paper presents a phase diagram for two-layer neural networks under different initialization scales.
problem Understanding the behavior of neural networks under varying scales of initialization.
method Analysis of a phase diagram for two-layer neural networks.
result Condensation of weight vectors on isolated orientations during training.
LoRA fine-tuning causes forgetting, studied via particle system dynamics.
problem Catastrophic forgetting in LoRA fine-tuning.
method Mean-field self-attention model, partial differential equations, dynamical systems.
result Characterization of phase transitions in forgetting behavior.
We present an agent behavior based microscopic model that induces jumps, spikes and high volatility phases in the price process of a traded asset. We transfer dynamics of thermally activated jumps of an unexcited/ excited two state system discussed in the context of quantum mechanics to agent socio-economic behavior an…
The paper analyzes the dynamics of tokens in transformer models at moderate interaction levels.
problem Understanding the evolution of tokens in transformer models at moderate interaction levels.
method Modeling transformer models as a system of particles interacting in a mean-field way and studying the corresponding dynamics.
result Characterization and convergence of the limiting dynamics in different phases of the system.
We explain how neural networks learn to solve modular addition tasks.
problem How two-layer neural networks learn to solve modular addition tasks.
method Formalized a diversification condition during training, proving it allows the network to approximate the correct logic for modular addition.
result Neural networks can robustly identify the correct sum through phase symmetry and frequency diversification.
Gradient flow in phase retrieval escapes spurious minima with high probability.
problem Understanding gradient-based optimization in high-dimensional non-convex functions.
method Analytical and numerical study of gradient dynamics in phase retrieval.
result Gradient flow avoids spurious minima by drifting along unstable directions.
In this paper, from the viewpoint of completeness of Marsden-Weinstein reduction, we illustrate how to give the definitions of a controlled Hamiltonian (CH) system and a reducible controlled Hamiltonian system with symmetry; and how to describe the dynamics of a CH system and the controlled Hamiltonian equivalence; as …
Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.
problem Understanding the performance and capabilities of generative diffusion models.
method Reformulating generative diffusion models using statistical mechanics, focusing on phase transitions and symmetry breaking.
result Generative diffusion models undergo second-order phase transitions with mean-field universality, critical instability, and mean-field critical exponents.
Anomalous diffusion in SGD reveals interactions between hyperparameters and Hessian.
problem Understanding the limiting dynamics of SGD in deep neural networks.
method Continuous-time model of SGD as an underdamped Langevin equation, derived for linear regression.
result Anomalous diffusion is explained by modified loss and probability currents in phase space.
Proposes a new binary classification model inspired by fluid phase separation.
problem Binary classification challenges.
method Discretization of nonlinear reaction-diffusion equation coupled with ODE, inspired by fluid dynamics.
result PSBC model achieves comparable performance to traditional methods on MNIST.
The stability of money value is an important requisite for a functioning economy, yet it critically depends on the actions of participants in the market themselves. Here we model the value of money as a dynamical variable that results from trading between agents. The basic trading scenario can be recast into an Ising t…
Weight decay stabilizes training dynamics by slowing progressive sharpening.
problem Understanding how weight decay affects training stability in deep learning models.
method Analyzing weight decay effects at the Edge of Stability, developing a mathematical framework.
result Weight decay dampens oscillations and stabilizes sharpness in CNNs, causing a phase transition in MLPs.
We explore a computational model of an incompressible fluid with a multi-phase field in three-dimensional Euclidean space. By investigating an incompressible fluid with a two-phase field geometrically, we reformulate the expression of the surface tension for the two-phase field found by Lafaurie, Nardone, Scardovelli, …
Quadratic models explain neural network behavior during training.
problem Understanding neural network dynamics during training with large learning rates.
method Developed and tested Neural Quadratic Models.
result Neural Quadratic Models exhibit the 'catapult phase' similar to neural networks.
In this paper we explore the functional correlation approach to operational risk. We consider networks with heterogeneous a-priori conditional and unconditional failure probability. In the limit of sparse connectivity, self-consistent expressions for the dynamical evolution of order parameters are obtained. Under equil…
META-DES.H selects competent classifiers using meta-learning and dynamic weighting.
problem Selecting competent classifiers in dynamic ensemble selection.
method META-DES framework using meta-learning and dynamic weighting.
result Improvements in recognition accuracy on 30 datasets.
Gradient descent variants improve phase retrieval accuracy.
problem Phase retrieval problem in high-dimensional spaces.
method Gradient descent, stochastic gradient descent, Langevin algorithm, dynamical mean-field theory.
result Stochastic variants of gradient descent achieve better generalization in phase retrieval.
EP learns like BPTT but with local weight updates.
problem Existing EP lacks a local time learning rule.
method C-EP updates weights simultaneously with neuron dynamics.
result C-EP follows BPTT gradients and performs well.
Deep neural networks near edge of chaos show universal scaling laws.
problem Understanding the behavior of deep neural networks near critical points.
method Analogy to absorbing phase transitions in statistical mechanics, deterministic propagation dynamics, mean-field and directed percolation universality classes.
result Deep neural networks exhibit universal scaling laws near the edge of chaos.
Study sharp interface limit of Allen-Cahn equation to characterize mean curvature flow with boundary conditions.
problem Characterizing mean curvature flow with Dirichlet or dynamic boundary conditions.
method Varifold formulation, phase field method, extending Brakke flow.
result Sharp interface limit of Allen-Cahn equation converges to mean curvature flow with boundary conditions.
By generalizing the measurements on the game experiments of mixed strategy Nash equilibrium, we study the dynamical pattern in a representative dynamic stochastic general equilibrium (DSGE). The DSGE model describes the entanglements of the three variables (output gap [y], inflation [π] and nominal interest rate [$…
This research explores the dynamics of linearised neural nets, revealing distinct learning phases and layer growth rates.
problem Understanding the fundamental mechanics of neural nets and their learning dynamics.
method Derivation of properties of learning dynamics in general multi-layer linear neural nets, including orthogonal networks.
result Linear multi-layer neural nets exhibit distinct phases of learning with different layer growth rates, and nonlinearity affects these dynamics.
DLNs dynamics change with variance, leading to saddle-to-saddle training phases.
problem Understanding the dynamics of DLNs with varying initialization variance.
method Analyzing the phase transition of DLNs' dynamics as variance changes.
result Gradient descent visits a sequence of saddles, reaching a sparse global minimum.
Study of rotation angles in a rotating disc model.
problem Understanding geometric phase in rotating systems.
method Analyzes a simple kinematic model of rotating discs.
result Explicit form of geometric phase Δg found using Baumkuchen lemma. Study on multi-head softmax attention dynamics for in-context learning.
problem Understanding and optimizing multi-head softmax attention models for multi-task linear regression.
method Gradient flow analysis and spectral mapping technique.
result Gradient flow converges to optimal multi-head softmax attention model, with task allocation emerging during training.
Study reconstructs network interactions from oscillator dynamics data.
problem Reconstructing network interactions from observed oscillator dynamics.
method Machine learning methods applied to phase-oscillator networks.
result Reconstruction of network interactions and intrinsic dynamics parameters.
Geometric phases describe how in a continuous-time dynamical system the displacement of a variable (called phase variable) can be related to other variables (shape variables) undergoing a cyclic motion, according to an area rule. The aim of this paper is to show that geometric phases can exist also for discrete-time sy…
Motivated by the need for accurate frequency information, a novel algorithm for estimating the fundamental frequency and its rate of change in three-phase power systems is developed. This is achieved through two stages of Kalman filtering. In the first stage a quaternion extended Kalman filter, which provides a unified…
Gating units in GRUs and LSTMs create slow modes and control phase-space complexity.
problem Training challenges in RNNs due to exploding or vanishing gradients.
method Random matrix theory and mean-field theory applied to GRUs and LSTMs.
result Gates in GRUs and LSTMs lead to accumulation of slow modes and control phase-space complexity.
We study a class of heterogeneous agent-based models which are based on a basic set of principles, and the most fundamental operations of an economic system: trade and product transformations. A basic guiding principle is scale invariance, which means that the dynamics of the economy should not depend on the units used…