Deep networks learn features suddenly, akin to a phase transition.
problem Understanding sudden feature learning in deep networks.
method Adaptive kernel approach applied to two teacher-student models.
result Feature learning and Grokking are analogous to a first-order phase transition.
Improved simulation of phase transitions using hierarchical autoregressive networks.
problem Simulating phase transitions in complex systems.
method Hierarchical Autoregressive Neural (HAN) network sampling algorithm.
result Significant improvement in statistical uncertainty compared to the Wolff cluster algorithm.
New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.
problem Achieving best-of-both-worlds guarantees with data-dependent regret bounds in MDPs with unknown transitions.
method Optimistic follow-the-regularized-leader algorithm with new optimistic Q-function estimators and transition bonus.
result First-order, second-order, and path-length bounds with polylog(T) regret in the stochastic regime.
Two-dimensional transition rates improve life insurance reserve calculations.
problem Calculating life insurance reserves with Markov assumptions.
method Introducing two-dimensional forward and backward transition rates.
result Two-dimensional transition rates enable more accurate reserve calculations.
Machine learning predicts synchronization transitions in unknown systems.
problem Predicting synchronization transitions in systems with unknown equations.
method Developed a 'parameter-aware' machine learning scheme using reservoir computing or echo state networks.
result Machine learning accurately predicts synchronization transitions, including hysteresis loops.
New non-Kähler 3-folds constructed via log conifold transitions.
problem Constructing new non-Kähler 3-folds from Fano threefold pairs.
method Defining log conifold transitions and studying their deformation theory.
result Local smoothings of nodes can be lifted to global first-order deformations.
The paper models market crashes as phase transitions, finding dynamic transitions offer better predictions.
problem Understanding and predicting extreme financial events like market crashes.
method Employing phase transition theory, focusing on endogenous crashes, and comparing DPT, CPT, and SPT.
result Dynamic phase transitions provide more accurate predictions of market crashes compared to critical and stochastic models.
Derives FACT, an alternative to NFA for neural networks, explaining feature learning.
problem Understanding how neural networks learn representations.
method First-principles approach using first-order optimality conditions.
result FACT explains why NFA holds and provides a principled alternative.
This work interprets GELU and related activations via a first-order loss function.
problem Understanding and optimizing activation functions in neural networks.
method Complementary interpretation using the Gaussian first-order loss function.
result Calibrated or learned uniform-threshold gates are competitive and often outperform GELU, ReLU, and SiLU/Swish.
The stability of money value is an important requisite for a functioning economy, yet it critically depends on the actions of participants in the market themselves. Here we model the value of money as a dynamical variable that results from trading between agents. The basic trading scenario can be recast into an Ising t…
We show that for three dimensional gravity with higher genus boundary conditions, if the theory possesses a sufficiently light scalar, there is a second order phase transition where the scalar field condenses. This three dimensional version of the holographic superconducting phase transition occurs even though the pure…
Bayesian theory explains abrupt emergence of copy subcircuit in attention.
problem Understanding the abrupt emergence of the copy subcircuit in attention during training.
method Deriving a closed-form posterior over the attention matrix and reducing it to a low-dimensional order parameter space.
result Derive a phase transition in the amount of training data.
Recently, it was shown that there is a phase transition in the community detection problem. This transition was first computed using the cavity method, and has been proved rigorously in the case of q=2 groups. However, analytic calculations using the cavity method are challenging since they require us to understand p…
Within the framework of maximum entropy principle we show that the finite-size long-range Ising model is the adequate model for the description of homogeneous credit portfolios and the computation of credit risk when default correlations between the borrowers are included. The exact analysis of the model suggest that w…
Study analyzes order transitions in high, medium, and low market cap stocks using Markov chains.
problem Understanding order transitions in stocks of different market caps.
method First-order discrete-time Markov chain model applied to NASDAQ100 stocks.
result Limit orders exhibit higher inertia during opening hours but decrease in subsequent hours, while market orders increase.
Study dynamic risk measures with distributional uncertainty using optimal transport.
problem Risk robustification under distributional uncertainty in Markovian models.
method Characterize risk measures via convex monotone semigroups and optimal transport costs.
result Identify generator and correction terms for dynamic risk measures under different scaling regimes.
Novel algorithm for Markov decision processes using rank-one approximation.
problem Solving planning and learning problems of Markov decision processes.
method Policy iteration with rank-one approximation of transition probability matrix.
result The proposed algorithm consistently outperforms first-order algorithms and their accelerated versions.
The geometrical structure known as the Tulczyjew triple has proved to be very useful in describing mechanical systems, even those with singular Lagrangians or subject to constraints. Starting from basic concepts of variational calculus, we construct the Tulczyjew triple for first-order Field Theory. The important featu…
Statistical relational models provide compact encodings of probabilistic dependencies in relational domains, but result in highly intractable graphical models. The goal of lifted inference is to carry out probabilistic inference without needing to reason about each individual separately, by instead treating exchangeabl…
We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP also allows the specific dependence on history to be learned efficiently from da…
Study analyzes stock order transitions during US-China trade war using Markov chains.
problem Understanding order dynamics during extreme macroeconomic events.
method First-order time-homogeneous discrete-time Markov chain model.
result Active participation by different traders during high volatility days, influencing market outcomes.
We characterize stationary solutions to McKean-Vlasov equations on the circle.
problem Stationary solutions of McKean-Vlasov equations on the circle.
method Exact equivalence to an infinite-dimensional quadratic system of equations over Fourier coefficients, leading to explicit characterization of stationary states.
result Analytic expressions for the emergence, form, and shape of bifurcations involving multiple Fourier modes, and connections with discontinuous phase transitions.
An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive the result. The analysis is conducted through evaluation of the entropy, an expo…
Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets…
Deep networks learn clean structure before memorizing corrupted labels, leaving a spectral signature in gradient centered scatter.
problem Deep networks' transition from learning clean structure to memorizing corrupted labels under label noise.
method Analysis of the centered scatter of per-example last-layer gradients to identify Fisher Rank Inflation.
result Fisher Rank Inflation is a spectral signature of memorization under label noise, with effective rank expanding during memorization.
Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real…
This paper presents a Bayesian image segmentation model based on Potts prior and loopy belief propagation. The proposed Bayesian model involves several terms, including the pairwise interactions of Potts models, and the average vectors and covariant matrices of Gauss distributions in color image modeling. These terms a…
Neural networks parameterize time-varying Markov dynamics in financial time series.
problem Estimating Markov transition matrices in high-resolution, high-noise financial data.
method Introduces a neural network framework to generate explicit, time-varying Markov transition matrices, constraining neural outputs to formal stochastic operators.
result Learned operators capture regime shifts, with high-volatility regimes homogenizing transition dynamics.
New method for efficient probabilistic deep state-space models.
problem Efficient inference for probabilistic deep state-space models.
method Deterministic inference algorithm for ProDSSM with neural network weights.
result Superior balance between predictive performance and computational budget.
Deep learning has become a powerful and popular tool for a variety of machine learning tasks. However, it is challenging to understand the mechanism of deep learning from a theoretical perspective. In this work, we propose a random active path model to study collective properties of deep neural networks with binary syn…
Moving boundary problems allow to model systems with phase transition at an inner boundary. Driven by problems in economics and finance, in particular modeling of limit order books, we consider a stochastic and non-linear extension of the classical Stefan-problem in one space dimension, where the paths of the moving in…
Study shows pre-event L2 liquidity state predicts crypto futures liquidity better than event labels.
problem Understanding how crypto futures liquidity changes over time.
method Combining L2 order book data, trade-flow records, and macro-event windows to define discrete liquidity-state transitions and evaluate models.
result Pre-event L2 liquidity state predicts post-event liquidity regimes better than event labels, and order flow adds value only when layered on top of the state model.
Deep learning solves dynamic programming with recursive utility.
problem Challenges in solving high-dimensional discrete-time dynamic programming problems with recursive utility.
method Certainty Equivalent Learning (CEL) algorithm that learns certainty-equivalent value directly with neural networks.
result Accurate value and policy approximations in high-dimensional problems, comparable to VFI in some cases.
Examines how first-order differential operators can be equivalently transformed.
problem Equivalence of first-order linear differential operators.
method Discussion of equivalency transformations.
result Explains how first-order differential operators can be transformed equivalently.
New algorithms reduce regret in online MDPs by adapting to data and variance.
problem Adapting to both adversarial and stochastic environments in online MDPs.
method Develops algorithms based on global optimization and policy optimization, using optimistic follow-the-regularized-leader with log-barrier regularization.
result Achieves refined data-dependent and variance-dependent regret bounds.
Enhances inference of spreading processes using neural-network priors.
problem Estimating initial states of graph processes from partial observations.
method Bayesian framework with single-layer perceptron neural network for initial states; hybrid BP-AMP algorithm.
result Model exhibits first-order phase transitions, creating a statistical-to-computational gap.
New vector fields integrate first-order ODEs.
problem Integrating first-order ODEs.
method Relation between Riemannian manifolds and ODEs integration.
result Integration procedure for first-order ODEs.
Users form information trails as they browse the web, checkin with a geolocation, rate items, or consume media. A common problem is to predict what a user might do next for the purposes of guidance, recommendation, or prefetching. First-order and higher-order Markov chains have been widely used methods to study such se…
Study first-order locally convex Lie algebroids in Bastiani calculus.
problem Define and study first-order locally convex Lie algebroids.
method Define sheaves of Lie algebroid forms and morphisms, prove category structure, study representations and cohomology.
result First-order locally convex Lie algebroids form a category and have applications in Lie II theorems.
A new framework uses stochastic optimal control to estimate rare events more accurately.
problem Estimating rare events like chemical reactions in biomolecules is computationally challenging.
method The approach casts committor estimation as a stochastic optimal control problem, developing direct and off-policy Value Matching losses.
result The framework yields more accurate committor estimates, reaction rates, and equilibrium constants.
A new method solves complex constrained minimax problems.
problem Solving constrained minimax optimization problems.
method First-order augmented Lagrangian method.
result Established an operation complexity of O(ε−4logε−1). A guide for solving first-order elliptic boundary value problems.
problem Solving first-order elliptic boundary value problems on manifolds.
method Operator methods and general elliptic boundary conditions.
result Characterization of a new subclass of elliptic boundary conditions.
A Markov Chain approach for aligning generative models from pairwise human preferences.
problem Aligning generative models from pairwise human preferences.
method Markov Chain from Human Feedback (MCHF)
result MCHF converges geometrically fast to the stationary distribution.
First-order ODEs linked to flat surfaces, leading to integrability.
problem Integrating first-order ODEs.
method Defined Riemannian metrics on variable spaces, studied surface properties, and established connections between Jacobi fields and Lie point symmetries.
result Flat associated surfaces lead to integrable first-order ODEs.
Study efficient algorithms for nonconvex optimization with state-dependent Markov data.
problem Stochastic optimization with Markovian data and state-dependent transition kernels.
method Projection-based and projection-free algorithms for constrained nonconvex problems.
result The number of oracle calls to achieve an ε-stationary point is O(1/ε2.5). Extends a theorem for first-order elliptic operators on manifolds.
problem Proving the relative index theorem for general first-order elliptic operators.
method Using boundary value problems and graphical decomposition of elliptically regular boundary conditions.
result Proves the relative index theorem for general first-order elliptic operators.
Two classes of methods have been proposed for escaping from saddle points with one using the second-order information carried by the Hessian and the other adding the noise into the first-order information. The existing analysis for algorithms using noise in the first-order information is quite involved and hides the es…
For the implicit systems of first order ordinary differential equations on the plane there is presented the complete local classification of generic singularities of family of its phase curves up to smooth orbital equivalence. Besides the well known singularities of generic vector fields on the plane and the singularit…