We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (1) the complexity of the loss landscape and of the dynamics within it, and (2) to what extent DNNs share similarities with glassy systems. O…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “glassy system”
Review of gradient-based algorithms in statistical inference problems.
problem Understanding the dynamics of gradient-based algorithms in statistical inference.
method Insights from physics of glassy systems.
result Quantitative and qualitative understanding of algorithm performance.
Study dynamics of alternating minimization for bilinear regression under large system limits.
problem Understanding the time evolution of alternating minimization for bilinear regression.
method Replica method applied to a multi-temperature glassy system.
result Dynamics of alternating minimization can be described by a two-dimensional discrete stochastic process.
Experimental fractal landscape dynamics observed in emulsions.
problem Understanding anomalous motions in soft glassy materials.
method Quantitative analysis of oil droplet trajectories in dense emulsions.
result Experimental fractal geometry matches computational model of soft glassy dynamics.
Glassy nature of the hard phase in inference problemscond-mat.dis-nn
An algorithmically hard phase was described in a range of inference problems: even if the signal can be reconstructed with a small error from an information theoretic point of view, known algorithms fail unless the noise-to-signal ratio is sufficiently small. This hard phase is typically understood as a metastable bran…
The dynamical behavior of the currency exchange rate after its large-scale catastrophe is discussed through a case study of the rate of Russian rubles to US dollars after its crash in 2014. It is shown that, similarly to the case of the stock market crash, the relaxation is characterized by a power law, which is in ana…
Our understanding of supercooled liquids and glasses has lagged significantly behind that of simple liquids and crystalline solids. This is in part due to the many possibly relevant degrees of freedom that are present due to the disorder inherent to these systems and in part to non-equilibrium effects which are difficu…
Paper introduces supervised and unsupervised TAM models for binary neurons.
problem Learning and retrieval of structured triplets of patterns in neural networks.
method Extends Hebbian paradigm to supervised and unsupervised protocols, using glassy statistical mechanical techniques.
result Obtained self-consistency equations for critical dataset sizes and retrieval performance.
Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin alg…
How the rich get richerphysics.soc-ph
In our model, traders interact with each other and with a central bank; they are taxed on the money they make, some of which is dissipated away by corruption. A generic feature of our model is that the richest trader always wins by 'consuming' all the others: another is the existence of a threshold wealth, below wh…
Approximate Survey Propagation for Statistical Inferencecond-mat.dis-nn
Approximate message passing algorithm enjoyed considerable attention in the last decade. In this paper we introduce a variant of the AMP algorithm that takes into account glassy nature of the system under consideration. We coin this algorithm as the approximate survey propagation (ASP) and derive it for a class of low-…
We demonstrate that graphs embedded on surfaces are a powerful and practical tool to generate, characterize and simulate networks with a broad range of properties. Remarkably, the study of topologically embedded graphs is non-restrictive because any network can be embedded on a surface with sufficiently high genus. The…
In this paper we provide a comprehensive analysis of a structural model for the dynamics of prices of assets traded in a market originally proposed in [1]. The model takes the form of an interacting generalization of the geometric Brownian motion model. It is formally equivalent to a model describing the stochastic dyn…
From complex to simple : hierarchical free-energy landscape renormalized in deep neural networkscond-mat.dis-nn
We develop a statistical mechanical approach based on the replica method to study the design space of deep and wide neural networks constrained to meet a large number of training data. Specifically, we analyze the configuration space of the synaptic weights and neurons in the hidden layers in a simple feed-forward perc…
Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitionscond-mat.dis-nn
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the stati…
Empirical data of supermarket sales show stylised facts that are similar to stock markets, with a broad (truncated) Levy distribution of weekly sales differences in the baseline sales [R.D. Groot, Physica A 353 (2005) 501]. To investigate the cause of this, the influence of social interactions and advertisements are st…
Overview of integrable systems with symmetries, focusing on toric and semitoric systems.
problem Classifying and understanding integrable systems with symmetries.
method Using decorated polygons and controlled bifurcations in one-parameter families of systems.
result Construction of explicit semitoric systems with prescribed invariants.
New method to derive integrable systems from existing Lax systems.
problem Deriving new integrable systems from existing ones.
method Systematic method of deriving new integrable systems from a given one.
result Examples of new integrable systems derived, including the dispersionless Hirota equation, the general heavenly equation, and the web equations.
Learning to control linear systems is statistically hard, especially for underactuated systems.
problem Statistical difficulty of learning to control linear systems, especially underactuated ones.
method Utilized minimax lower bounds and structural assumptions to prove learning complexity can be exponential.
result Learning complexity can be at most exponential with the controllability index of the system.
Discrete-time systems can be characterized by simple flat coordinates and their shifts.
problem Characterizing flatness of discrete-time systems.
method Developed a map from flat coordinates and their shifts to system state and input, fulfilling system equations identically.
result Derived necessary conditions for a system to be flat, without requiring differential geometry methods.
The paper explores when linear system identification is hard or easy, especially for under-actuated systems.
problem Statistical hardness of learning linear systems, especially under-actuated or under-excited systems.
method Using tools from minimax theory and recent statistical tools for finite sample analysis of system identification.
result The controllability index of linear systems affects the sample complexity of identification, making some systems hard to learn.
This paper improves system identification by reducing sample complexity for high-dimensional linear dynamical systems.
problem High sample complexity for learning partially observed linear dynamical systems in high dimensions.
method Introduces an -regularized estimation method that reduces sample complexity from linear to logarithmic with system dimension.
result Markov parameters can be learned with logarithmic number of samples relative to system dimension, improving sample complexity.
In integrable hydrodynamic systems, coordinates exist where generators and symmetries are simple.
problem Existence of Riemannian invariants for integrable systems of hydrodynamic type.
method Finding coordinates where the generator and all symmetries are diagonal.
result In integrable hydrodynamic systems, there exist coordinates where the generator and all symmetries are diagonal.
Nonholonomic Controlled Hamiltonian System: Symmetric Reduction and Hamilton-Jacobi Equationsmath.SG
This paper studies nonholonomic constraints in Hamiltonian systems, deriving equations and theorems.
problem Analyzing nonholonomic constraints in Hamiltonian systems.
method Deriving distributional RCH systems, geometric constraint conditions, and Hamilton-Jacobi theorems.
result Derives precise geometric constraint conditions and Hamilton-Jacobi theorems for nonholonomic systems.
New method models unknown systems with hidden parameters using neural networks.
problem Modeling unknown dynamical systems with hidden parameters.
method Training a deep neural network (DNN) model using trajectory data of the unknown system.
result DNN model accurately predicts unknown dynamical systems with new initial conditions.
Study absolute equivalence for Pfaffian systems, applying to control systems.
problem Absolute equivalence of Pfaffian systems with specific independence conditions.
method Structural results for Pfaffian systems of corank 3, applied to control systems.
result Dynamic feedback linearization of control systems with 2 inputs.
Estimates input from output of nonlinear systems using ANN.
problem Estimating unknown compositional input from system output.
method Artificial Neural Networks (ANNs) for nonlinear system inversion.
result ANNs can compete with optimal bounds for linear systems and demonstrate promising results for nonlinear systems.
Solves selecting the best optimizing system problems.
problem Selecting the best system among contenders with unknown performance.
method Adaptive algorithms integrating stochastic gradient descent and sequential elimination.
result Exponential rates of convergence to zero for false selection probability.
This paper considers control systems defined on Lie algebroids. After deriving basic controllability tests for general control systems, we specialize our discussion to the class of mechanical control systems on Lie algebroids. This class of systems includes mechanical systems subject to holonomic and nonholonomic const…
Superintegrable systems on curved manifolds found to have Hessian structures.
problem Characterizing superintegrable systems on curved manifolds.
method Identifying and computing Hessian coordinates for superintegrable systems.
result Examples of superintegrable systems in 2D and 3D have natural Hessian coordinates.
Measures of Systemic Riskq-fin.RM
Systemic risk refers to the risk that the financial system is susceptible to failures due to the characteristics of the system itself. The tremendous cost of systemic risk requires the design and implementation of tools for the efficient macroprudential regulation of financial institutions. The current paper proposes a…
Researchers solve boundary and scattering rigidity problems for magnetic systems.
problem Recovering magnetic systems from boundary or scattering data.
method Reduced to magnetic systems and applied results from [DPSU07].
result Recovering -system up to a gauge.
The paper develops methods to derive mixed superposition rules for Lie systems and applies them to various physical systems.
problem Finding general solutions for Lie systems.
method Develops mixed superposition rules for Lie systems with imprimitive Lie algebras and semidirect sums.
result Extends coalgebra method to Lie systems of partial differential equations.
Paper develops reduction theory for controlled Lagrangian systems with symmetry and momentum map.
problem Reduction of controlled Lagrangian systems with symmetry and momentum map.
method Using Legendre transformation and Euler-Lagrange vector field, the paper extends symmetric reduction theory.
result Established regular reduction theory for RCL systems with symmetry and momentum map.
The paper studies connections in superintegrable systems, revealing geometric insights.
problem Understanding non- and semi-degenerate superintegrable systems.
method Analyzes two torsion-free connections associated with superintegrable systems.
result Semi-degenerate secondary structure tensor is the Ricci curvature of a natural torsion-free connection.
This paper proposes a system-agnostic policy for dynamic scheduling.
problem Dynamic scheduling in changing systems is challenging due to system-specific optimal policies.
method Descriptive policy that learns a system-agnostic scheduling principle.
result System-agnostic meta-learning enables adaptation to unseen system characteristics.
Learn dynamics of a system using auxiliary data from similar systems.
problem Learning dynamics of a linear system with limited data.
method Weighted least squares approach, incorporating auxiliary data.
result Auxiliary data can help reduce intrinsic error due to noise.
The inability to see and quantify systemic financial risk comes at an immense social cost. Systemic risk in the financial system arises to a large extent as a consequence of the interconnectedness of its institutions, which are linked through networks of different types of financial contracts, such as credit, derivativ…
Smooth and F--smooth systemsmath.DG
Abstract reviews geometric theories of smooth and F-smooth systems.
problem Geometric theories of smooth and F-smooth systems.
method Reviews geometric theories of smooth and F-smooth systems.
result Discusses geometric theories of smooth and F-smooth systems.
New Lie systems derived from Goursat distributions with applications to differential equations.
problem Analyzing Lie systems associated with Goursat distributions and their applications.
method Analyzing bracket-generating distributions and their relation to Lie systems, focusing on reductions and reconstructions.
result Lie systems associated with Goursat distributions can be reduced and solutions reconstructed from reduced systems.
Elliptic systems are characterized by Darboux integrability.
problem Characterizing elliptic differential systems with holomorphic solutions.
method Using a complex manifold and associated holomorphic Pfaffian system.
result Elliptic systems are Darboux integrable under generic conditions.
The study analyzes stochastic Lie systems and their applications in various models.
problem Analyzing stochastic differential equations on manifolds.
method Coalgebra method for Hamiltonian stochastic Lie systems.
result New examples of stochastic Lie systems and Hamiltonian stochastic Lie systems are analyzed.
Reduces multisymplectic Lie systems through symmetry analysis.
problem Solving multisymplectic Lie systems using symmetry reduction.
method Using momentum maps for reduction and reconstruction of multisymplectic Lie systems.
result Solves the original problem by analyzing simpler multisymplectic Lie systems.
Study integrable discretizations of cyclic systems with circular coordinate lines.
problem Integrable discretizations of 3D cyclic systems with circular coordinate lines.
method Investigate circle congruences and flat connections in the context of discrete cyclic systems.
result Characterization of circle congruences and existence of certain flat connections.
A new chaotic financial system is proposed by considering ethics involvement in a four-dimensional financial system with market confidence. A five-dimensional conformable derivative financial system is presented by introducing conformable fractional calculus to the integer-order system. A discretization scheme is propo…
A distributed system identification method for LTI systems using reverse experience replay.
problem Online system identification of LTI systems over multi-agent networks.
method DSGD-RER, a distributed variant of SGD-RER with backward updates.
result The estimation error decreases as the network size grows.
This work extends reduction processes for nonholonomic discrete mechanical systems.
problem Nonholonomic discrete mechanical systems and their reductions.
method Introduces a category of discrete-time dynamical systems and a two-stage reduction process.
result Two-stage reduction process produces systems isomorphic to one-stage reduction.
Unified approach to deform Lie-Hamilton systems using Poisson-Hopf algebra.
problem Deforming Lie systems with quantum algebras.
method Poisson-Hopf algebra deformations applied to Lie-Hamilton systems.
result Unified approach to deformations of Lie-Hamilton systems on the real plane.