This work analyzes neural scaling laws using power-law data spectra and derives analytical expressions for generalization error.
problem Understanding how neural network performance scales with key factors like data size and model complexity.
method Statistical mechanics techniques applied to one-pass stochastic gradient descent in a student-teacher framework.
result Derivation of analytical expressions for generalization error under power-law data spectra and identification of conditions for power-law scaling.
The paper proves geometric and spectral alignment for deep neural networks.
problem Understanding the singular spectra of deep neural network layers.
method Proves deterministic quotient-geometric estimates for singular spectra of Frobenius-normalized layer factors.
result Exact power-law spectra form a trace-normalized Cartan orbit under Frobenius normalization.
Neural networks learn simpler features first, then more complex ones; Fourier analysis reveals this pattern.
problem Understanding the learning dynamics of neural networks, especially with natural image data.
method Fourier analysis of translation-invariant and power-law spectra to study feature learning.
result Simple neural networks first rely on amplitude information, then phase information, and power-law spectra can accelerate learning phase information.
Robust CD method for real-world time series with power-law distributions.
problem Challenges in causal discovery due to noise sensitivity.
method Power-law spectral feature extraction for robust CD.
result Consistently outperforms state-of-the-art alternatives on real-world datasets.
The log-periodic power law (LPPL) is a model of asset prices during endogenous bubbles. A major open issue is to verify the presence of LPPL in price sequences and to estimate the LPPL parameters. Estimation is complicated by the fact that daily LPPL returns are typically orders of magnitude smaller than measured price…
The study reveals the spectral structure of attention layers and its implications for generalization.
problem Understanding the spectral structure and generalization of trained attention layers.
method Empirical risk minimization in a single-head tied-attention layer, using random matrix theory, spin-glass theory, and approximate message passing.
result Exact high-dimensional characterization of training and test error, interpolation and recovery thresholds, and spectrum of the key and query matrices.
Random matrix theory is used to assess the significance of weak correlations and is well established for Gaussian statistics. However, many complex systems, with stock markets as a prominent example, exhibit statistics with power-law tails, that can be modelled with Levy stable distributions. We review comprehensively …
Paper examines the structure of stochastic gradients in deep learning.
problem Exploring the structure and heavy tails of stochastic gradients in deep learning.
method Conducted formal statistical tests on stochastic gradients and gradient noise.
result Stochastic gradients and gradient noise do not exhibit power-law heavy tails, but their covariance spectra do.
Signatures of universality are detected by comparing individual eigenvalue distributions and level spacings from financial covariance matrices to random matrix predictions. A chopping procedure is devised in order to produce a statistical ensemble of asset-price covariances from a single instance of financial data sets…
Stochastic momentum methods trade compute efficiency for serial runtime.
problem Stochastic momentum methods trade compute efficiency for serial runtime.
method Stochastic HB and ASGD for consistent linear regression with Gaussian covariates.
result HB preserves SGD-level CE over a larger batch-size window, allowing larger batches to reduce serial runtime until HB reaches its deterministic accelerated scale.
The study examines cryptocurrency market activity, revealing multifractal inter-transaction times and challenging traditional statistical models.
problem Analyzing long-range autocorrelations and multifractality in cryptocurrency market activity.
method Analysis of tick-by-tick data from multiple cryptocurrency trading platforms, focusing on inter-transaction times, transaction volumes, and volatility.
result Inter-transaction times exhibit multifractality, indicating periods of increased market activity are more complex than quiet periods.
Signals consisting of a sequence of pulses show that inherent origin of the 1/f noise is a Brownian fluctuation of the average interevent time between subsequent pulses of the pulse sequence. In this paper we generalize the model of interevent time to reproduce a variety of self-affine time series exhibiting power spec…
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
Investigates point spectra of vector fields and their properties.
problem Understanding the point spectra of vector fields.
method Define and study point spectra, prove properties under isometries, and analyze compactly supported fields.
result Point spectra are well-behaved under isometries and trivial for compactly supported fields.
Khovanov spectra are shown to be functorial under certain conditions.
problem Understanding functoriality of Khovanov spectra.
method Proving functoriality up to homotopy and sign for Khovanov spectra.
result Khovanov spectra are functorial under specific conditions.
We give a simple sufficient condition for Quinn's "bordism-type spectra" to be weakly equivalent to strictly associative ring spectra. We also show that Poincare bordism and symmetric L-theory are naturally weakly equivalent to monoidal functors. Part of the proof of these statements involves showing that Quinn's funct…
Method calculates spectra of Rarita-Schwinger operator on symmetric spaces.
problem Calculating spectra of the Rarita-Schwinger operator on compact symmetric spaces.
method Using Weitzenböck formulas, Laplace operator, Casimir operator, Freudenthal's formula, and branching rules.
result Obtained spectra on the sphere, complex projective space, and quaternionic projective space.
Proves spectra equivalence for Riemannian manifolds.
problem Equivalence of Almgren-Pitts and phase-transition half-volume spectra.
method Proof of spectra equivalence for Riemannian manifolds.
result Confirms conjecture about spectra equivalence.
Proposes a new complex Gaussian distribution for better modeling of complex-valued signals.
problem Limited ability of Gaussian distribution to represent diverse amplitude characteristics.
method Introduces a power-weighted noncentral complex Gaussian distribution on the complex plane.
result Consistently outperforms conventional distributions in log-likelihood for speech power spectra.
We compute the bridge spectra of cables of 2-bridge knots. We also give some results about bridge spectra and distance of Montesinos knots.
Paper resolves decades-old problem about L-spectra.
problem Identifying L-spectra local information with geometric data. method Proved equivalence of L-orientations and characteristic classes. result Levitt-Ranicki's theory equivalent to Brumfiel-Morgan's classes.
Study calculates spectra of minimal hypersurfaces in hyperbolic space.
problem Computing Laplacian spectra of minimal hypersurfaces.
method Analyzes hypersurfaces in hyperbolic space with specific asymptotic data.
result Obtains spectra and extremal properties of the bottom of the spectrum.
New metrics compare rational spectra using optimal transport.
problem Comparing rational spectra efficiently and accurately.
method Optimal transport and linear-systems theory.
result Established connection to Wasserstein distance.
Study how bottom of spectra changes with Riemannian coverings.
problem Behavior of bottom of spectra under Riemannian coverings.
method Analysis of scalar Schrödinger operators on Riemannian manifolds.
result Changes in the bottom of spectra observed under coverings.
Analyzes SGD dynamics on multi-class problems with exact expressions.
problem Analyzing SGD dynamics on multi-class problems.
method Developed a framework for analyzing training and learning rate dynamics using exact expressions.
result Exact expressions for risk and overlap with true signal in terms of ODEs.
In this paper, we numerically investigate the length spectra and the low-lying eigenvalue spectra of the Laplace-Beltrami operator for a large number of small compact(closed) hyperbolic (CH) 3-manifolds. The first non-zero eigenvalues have been successfully computed using the periodic orbit sum method, which are compar…
Tomova, along with results of Bachman and Schleimer, showed that any high distance knot has a stair-step bridge spectrum. In this paper, we compute the bridge spectra and distance of generalized Montesinos knots. In particular, we produce the first example of a class of knots which attain the stair-step bridge spectra …
Graphs approximate Laplacian spectra on manifolds.
problem Approximating Laplacian spectra on complex manifolds.
method Graph Laplacians on proximity graphs.
result Spectra of graph Laplacians approximate the Laplacian spectra of manifolds.
New ICA method for sources with mixed spectra.
problem Inaccurate separation of sources with temporal autocorrelations and mixed spectra.
method Estimates spectral density functions and line spectra using cubic splines and indicator functions, then maximizes the Whittle likelihood function.
result Outperforms existing ICA methods in simulations and EEG data applications.
Jones-Wenzl projectors lifted to Khovanov spectra, proving knot conjectures.
problem Understanding Jones-Wenzl projectors in Khovanov spectra.
method Constructing and studying lifted projectors via maps and polynomial actions.
result Complete computation of 3-colored Khovanov spectrum of the unknot, proving conjectures.
We introduce a framework, twisted parametrized stable homotopy theory, for describing semi-infinite homotopy types. A twisted parametrized spectrum is a section of a bundle whose fibre is the category of spectra. We define these bundles in terms of modules over a stack of parametrized spectra and in terms of diagrams o…
Functor decomposes Khovanov spectra for non-alternating diagrams.
problem Computing Khovanov spectra for diagrams without alternating pairs.
method Functor from cube to Burnside 2-category, decomposition into simplicial complexes.
result Homotopy type of almost-extreme Khovanov spectra computed.
Novel construction of Bauer--Furuta invariant using sheaves of spectra.
problem Constructing the Bauer--Furuta invariant without finite-dimensional approximations.
method Using sheaves of spectra and Borel--Moore homology, avoiding approximations.
result Defines the shriek functors and Thom spectra for index calculations.
Study on sine-cones' spectra and stability under Ricci-de Turck flow.
problem Analyzing stability and rigidity of sine-cones.
method Computed spectra of specific operators on sine-cones.
result Conditions for sine-cones' dynamic stability and rigidity.
We prove explicit upper and lower bounds for the L1-moment spectra for the Brownian motion exit time from extrinsic metric balls of submanifolds Pm in ambient Riemannian spaces Nn. We assume that P and N both have controlled radial curvatures (mean curvature and sectional curvature, respectively) as view…
The paper describes correlations of spectra for higher rank Anosov representations.
problem Understanding correlations of spectra for Anosov representations of higher rank groups.
method Relates correlation problem to counting projections in truncated hypertubes.
result Extends previous work on rank one representations to higher rank.
Formulas for spectra of higher spin operators on sphere subbundles.
problem Finding spectra of higher spin operators on specific subbundles of spinor-valued tensors.
method Explicit formulas derived for spectra in both even and odd dimensions.
result Spectra formulas for higher spin operators and their squares.
Study bottom of spectra on orbifolds via coverings.
problem Behavior of bottom of spectra under orbifold coverings.
method Analysis of scalar Schrödinger operators on orbifolds.
result Results apply to geometrically finite and conformally compact orbifolds.
Integrally splits L-spectra of integers into simpler components.
problem Understanding the homotopy type of L-spectra of integers.
method Using Anderson duality and splitting into simpler spectra.
result Splits L-spectra of integers into simpler components.
We view strict ring spectra as generalized rings. The study of their algebraic K-theory is motivated by its applications to the automorphism groups of compact manifolds. Partial calculations of algebraic K-theory for the sphere spectrum are available at regular primes, but we seek more conceptual answers in terms of lo…
Study uncovers scaling laws and spectral properties of shallow neural networks.
problem Understanding scaling laws and spectral properties of shallow neural networks.
method Leveraging connections with matrix compressed sensing and LASSO, derived a phase diagram for excess risk.
result Uncovered crossovers between scaling regimes and plateau behaviors, validated empirical observations.
Wavelet scattering spectra model non-Gaussian time-series, proving scale invariance for self-similar processes.
problem Modeling non-Gaussian time-series with stationary increments.
method Complex wavelet transform for scale variations, joint correlation matrix for scale dependencies, second wavelet transform for diagonalization, maximum entropy models conditioned by scattering spectra coefficients.
result Scattering spectra of self-similar processes are scale invariant, allowing statistical testing and generation of new time-series.
Manifold methods improve amino acid classification in LIBS spectra.
problem Improving classification accuracy of amino acids in LIBS spectra.
method Developed an information theoretic method for measuring LIBS energy spectra, implemented manifold methods for nonlinear dimensionality reduction.
result Nonlinear methods lead to increased classification accuracy in amino acid classification.
New SDE model from machine learning optimization with unique stationary distribution.
problem Stationary distribution of machine learning optimization models.
method Proved ergodicity and unique stationary distribution of power-law dynamic SDE.
result Power-law dynamic has a unique stationary distribution and is ergodic.
This paper describes a novel energy-based probabilistic distribution that represents complex-valued data and explains how to apply it to direct feature extraction from complex-valued spectra. The proposed model, the complex-valued restricted Boltzmann machine (CRBM), is designed to deal with complex-valued visible unit…
Intertrade duration of equities is an important financial measure characterizing the trading activities, which is defined as the waiting time between successive trades of an equity. Using the ultrahigh-frequency data of a liquid Chinese stock and its associated warrant, we perform a comparative investigation of the sta…
We take prior-to-crash market prices (NASDAQ, Dow Jones Industrial Average) as a signal, a function of time, we project these discrete values onto a vertical axis, thus obtaining a Cantordust. We study said cantordust with the tools of multifractal analysis, obtaining spectra by definition and by lagrangian coordinates…
We prove an analogue of Farb-Masur's theorem that the length-spectra metric on moduli space is "almost isometric" to a simple model V(S) which is induced by the cone metric over the complex of curves. As an application, we know that the Teichmüller metric and the length-spectra metric are "almost isometric…