Study finds phase transition in context-sensitive language model with short-range interactions.
problem Understanding phase transitions in language models with short-range interactions.
method Constructed a random language model with short-range interactions and investigated its statistical properties.
result Phase transition occurs in context-sensitive language models with constant context length.
Improved simulation of phase transitions using hierarchical autoregressive networks.
problem Simulating phase transitions in complex systems.
method Hierarchical Autoregressive Neural (HAN) network sampling algorithm.
result Significant improvement in statistical uncertainty compared to the Wolff cluster algorithm.
Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.
problem Understanding the performance and capabilities of generative diffusion models.
method Reformulating generative diffusion models using statistical mechanics, focusing on phase transitions and symmetry breaking.
result Generative diffusion models undergo second-order phase transitions with mean-field universality, critical instability, and mean-field critical exponents.
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
Characterizing the phase transitions of convex optimizations in recovering structured signals or data is of central importance in compressed sensing, machine learning and statistics. The phase transitions of many convex optimization signal recovery methods such as ℓ1 minimization and nuclear norm minimization are…
Persistent entropy detects phase transitions in complex systems.
problem Detecting phase transitions in complex systems.
method Established a general theorem for persistent entropy to reliably detect phase transitions, introduced operational framework for finite-time computations.
result Persistent entropy exhibits an asymptotically non-vanishing gap across phases, robust numerical signatures across experiments.
In this paper, we perform statistical segmentation and clustering analysis of the Dow Jones Industrial Average time series between January 1997 and August 2008. Modeling the index movements and log-index movements as stationary Gaussian processes, we find a total of 116 and 119 statistically stationary segments respect…
New algorithms handle phase retrieval with rank d measurements, revealing phase transitions.
problem Phase retrieval with rank d measurements.
method Random duality theory (RDT) and descending phase retrieval algorithms (dPR).
result Minimal sample complexity ratio for dPR's success exhibits phase transitions.
Descending phase retrieval algorithms show a phase transition with increasing sample complexity.
problem Theoretical limits of descending phase retrieval algorithms.
method Utilizing Random duality theory (RDT), the study develops a generic program to characterize algorithm performance.
result As sample complexity increases, the parametric manifold transitions from multi to single funneling points, leading to a phase transition in algorithm success.
Statistical mechanics reveals phase transitions in ε-SVR error.
problem Understanding task precision in neural representations with variability.
method Statistical mechanics applied to ε-SVR. result Double-descent phenomenon in generalization error due to ε. Abstract: Investigates the role of activation functions in neural networks and their physical basis.
problem Understanding the role of activation functions in neural networks and their physical basis.
method Formalizes the use of activation functions in neural inference by relating them to phase transitions in statistical physics.
result Reveals the physical justification for the performance of typical activation functions in neural networks.
High-dimensional U-statistics show surprising phase transitions, impacting kernel-based tests.
problem Understanding phase transitions in high-dimensional U-statistics.
method Proved a convergence theorem for U-statistics of degree two in high dimensions.
result High-dimensional U-statistics can have non-Gaussian limits with larger variance and asymmetry.
New insights into neural networks reveal unexpected phase transitions.
problem Understanding how overparameterized neural networks learn and generalize.
method Statistical physics of disordered systems applied to non-convex binary neural networks.
result Atypical phase transitions lead to better generalization in neural networks.
Deep heteroskedastic models overfit, showing a phase transition with regularization strength.
problem Overfitting in deep heteroskedastic regression models.
method Theoretical framework based on statistical field theory, empirical verification, and hyperparameter simplification.
result A phase transition in model behavior with varying regularization strength.
Study phase transitions in identifying infected individuals using group testing.
problem Identifying a set of k infected individuals from a population using pooled tests.
method Two random assignment designs (constant-column and Bernoulli) and polynomial-time inference procedures.
result Sharp phase transitions in statistical and computational limits for detection and recovery problems.
SLT explains neural network success by closing theory-practice gap.
problem Failure of classical inference and learning theory in modern neural networks.
method Physics-inspired Singular Learning Theory (SLT) applied to neural networks.
result SLT recovers known and novel scaling laws for neural network phase transitions.
We study the problem of approximate ranking from observations of pairwise interactions. The goal is to estimate the underlying ranks of n objects from data through interactions of comparison or collaboration. Under a general framework of approximate ranking models, we characterize the exact optimal statistical error …
The paper models market crashes as phase transitions, finding dynamic transitions offer better predictions.
problem Understanding and predicting extreme financial events like market crashes.
method Employing phase transition theory, focusing on endogenous crashes, and comparing DPT, CPT, and SPT.
result Dynamic phase transitions provide more accurate predictions of market crashes compared to critical and stochastic models.
Machine learning approximates phase transitions using Fisher information.
problem Understanding phase transitions from data using machine learning.
method Information geometry and Fisher information.
result Machine learning indicators approximate the square root of Fisher information.
High-dimensional models become unstable when sample size falls below a critical level, leading to a phase transition.
problem Instability in high-dimensional learning models when sample size is insufficient.
method Proved the necessity of a Fisher eigenvalue threshold for stability, introduced Fisher floor for verification.
result A sharp phase transition between reliable concentration and inevitable failure in high-dimensional learning.
If a given behavior of a multi-agent system restricts the phase variable to a invariant manifold, then we define a phase transition as change of physical characteristics such as speed, coordination, and structure. We define such a phase transition as splitting an underlying manifold into two sub-manifolds with distinct…
Multilayer graphs are commonly used for representing different relations between entities and handling heterogeneous data processing tasks. Non-standard multilayer graph clustering methods are needed for assigning clusters to a common multilayer node set and for combining information from each layer. This paper present…
This research highlights the secrecy potential of nonlinear generative models and their all-or-nothing phase transition.
problem Secrecy potential of nonlinear generative models in statistical learning.
method Replica method to derive asymptotic normalized cross entropy and statistical decoupling of Bayesian estimator.
result Strictly nonlinear models exhibit an all-or-nothing phase transition, leading to perfect secrecy.
Deep neural networks near edge of chaos show universal scaling laws.
problem Understanding the behavior of deep neural networks near critical points.
method Analogy to absorbing phase transitions in statistical mechanics, deterministic propagation dynamics, mean-field and directed percolation universality classes.
result Deep neural networks exhibit universal scaling laws near the edge of chaos.
Diffusion maps help learn complex quantum phase transitions from data.
problem Learning quantum phase transitions from experimental data is challenging.
method Diffusion maps for nonlinear dimensionality reduction and spectral clustering.
result Diffusion maps can learn complex phase transitions unsupervised.
Double-well transitions are stiffer than minimal surfaces.
problem Rigidity of double-well phase transitions compared to minimal hypersurfaces.
method Comparison of rigidity properties between double-well phase transitions and minimal hypersurfaces.
result Double-well phase transitions exhibit more rigidity than minimal hypersurfaces.
The optimal (`equilibrium') macroscopic properties of an economy with N industries endowed with different technologies, P commodities and one consumer are derived in the limit N→∞ with n=N/P fixed using the replica method. When technologies are strictly inefficient, a phase transition occurs upon increas…
This paper analyzes MFVBI for GMM using statistical mechanics.
problem Approximate fast computation of Gaussian Mixture Model.
method Statistical mechanics and MFVBI applied to GMM.
result Rigorous analysis and mathematical foundation for MFVBI applied to GMM.
An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive the result. The analysis is conducted through evaluation of the entropy, an expo…
Gradient flow in phase retrieval escapes spurious minima with high probability.
problem Understanding gradient-based optimization in high-dimensional non-convex functions.
method Analytical and numerical study of gradient dynamics in phase retrieval.
result Gradient flow avoids spurious minima by drifting along unstable directions.
Study phase transitions with prescribed mean curvature in Riemannian manifolds.
problem Understanding phase transitions with prescribed mean curvature in geometric settings.
method Analyzing solutions to inhomogeneous semilinear elliptic PDEs, establishing bounds and asymptotics.
result Established upper and lower bounds for eigenvalues of phase transition problems.
Paper analyzes latent space geometry in generative models using Fisher information.
problem Understanding the structure of latent spaces in generative models.
method Reconstructs Fisher information metric from generated samples and posterior distribution.
result Reveals fractal structure and abrupt changes in Fisher metric at phase boundaries.
Optimal spectral initializers impact phase retrieval phase transitions.
problem Understanding the limits of phase retrieval algorithms.
method Developed Random duality theory (RDT) to characterize optimal spectral initializers.
result Optimal spectral initializers can fall into flat regions of the phase retrieval manifold, making phase retrieval difficult.
Continuous phase transitions identified in Doi-Onsager, noisy transformer, and Hegselmann-Krause models.
problem Phase transitions in multimodal models and their properties.
method Sharp coercivity estimate and constrained Lebedev--Milin inequality.
result Continuous phase transitions at critical coupling strengths for Doi-Onsager, noisy transformer, and Hegselmann-Krause models.
Unsupervised learning is a discipline of machine learning which aims at discovering patterns in big data sets or classifying the data into several categories without being trained explicitly. We show that unsupervised learning techniques can be readily used to identify phases and phases transitions of many body systems…
The analysis of Belief Propagation and other algorithms for the {\em reconstruction problem} plays a key role in the analysis of community detection in inference on graphs, phylogenetic reconstruction in bioinformatics, and the cavity method in statistical physics. We prove a conjecture of Evans, Kenyon, Peres, and Sch…
In the Information Bottleneck (IB), when tuning the relative strength between compression and prediction terms, how do the two terms behave, and what's their relationship with the dataset and the learned representation? In this paper, we set out to answer these questions by studying multiple phase transitions in the IB…
This work analyzes how transformers learn common linear regression tasks.
problem Understanding how in-context learning operates in real-world applications with common task structures.
method Analyzing a linear attention model trained on low-rank regression tasks.
result Statistical fluctuations in finite pre-training data induce an implicit regularization, leading to a sharp phase transition in generalization error.
Proves spectra equivalence for Riemannian manifolds.
problem Equivalence of Almgren-Pitts and phase-transition half-volume spectra.
method Proof of spectra equivalence for Riemannian manifolds.
result Confirms conjecture about spectra equivalence.
Deep networks learn features suddenly, akin to a phase transition.
problem Understanding sudden feature learning in deep networks.
method Adaptive kernel approach applied to two teacher-student models.
result Feature learning and Grokking are analogous to a first-order phase transition.
Scaling properties in financial fluctuations are reviewed from the standpoint of statistical physics. We firstly show theoretically that the balance of demand and supply enhances fluctuations due to the underlying phase transition mechanism. By analyzing tick data of yen-dollar exchange rates we confirm two fractal pro…
Analyzes bias-variance in overparameterized linear models using random features.
problem Understanding bias-variance trade-off in overparameterized models.
method Zero-temperature cavity method and random matrix theory.
result Three phase transitions in the linear random features model.
Adaptive algorithm identifies best arm with abstention, showing phase transition from polynomial to exponential error probability.
problem Bayesian best-arm identification with abstention to reduce undetected error.
method Adaptive algorithm PGWS that optimally uses abstention budget.
result Introducing any positive abstention budget induces an exponential decay in undetected error probability.
Geometry of hypersurfaces defined by the relation which generalizes classical formula for free energy in terms of microstates is studied. Induced metric, Riemann curvature tensor, Gauss-Kronecker curvature and associated entropy are calculated. Special class of ideal statistical hypersurfaces is analyzed in details. No…
High-dimensional random geometry shows phase transitions in various problems.
problem Phase transitions in high-dimensional random geometry.
method Analysis of various financial, optimization, and ecological problems.
result Links between seemingly distant fields and further ramifications.
In this paper we study the phenomenon of phase transitions for the geodesic flow on some geometrically finite negatively curved manifolds. We define a class of potentials going slowly to zero through the cusps of M for which the pressure map exhibits a phase transition. By a careful choice of the metric at the cusp w…
We derive the exact solution of a one-dimensional Markov functional model with log-normally distributed interest rates in discrete time. The model is shown to have two distinct limiting states, corresponding to small and asymptotically large volatilities, respectively. These volatility regimes are separated by a phase …
The classification of phase transitions is a central and challenging task in condensed matter physics. Typically, it relies on the identification of order parameters and the analysis of singularities in the free energy and its derivatives. Here, we propose an alternative framework to identify quantum phase transitions,…