Combines noncommutative geometry and spectral theory for new Weyl laws.
problem Developing new Weyl laws for noncommutative manifolds.
method Functional analysis, spectral theory, and Tauberian conditions.
result Generalizes and simplifies recent results on Weyl laws and integration formulas.
The study reveals the spectral structure of attention layers and its implications for generalization.
problem Understanding the spectral structure and generalization of trained attention layers.
method Empirical risk minimization in a single-head tied-attention layer, using random matrix theory, spin-glass theory, and approximate message passing.
result Exact high-dimensional characterization of training and test error, interpolation and recovery thresholds, and spectrum of the key and query matrices.
We introduce the stochastic multiplicative point process modelling trading activity of financial markets. Such a model system exhibits power-law spectral density S(f) ~ 1/f**beta, scaled as power of frequency for various values of beta between 0.5 and 2. Furthermore, we analyze the relation between the power-law autoco…
Auto-regressive conditionally heteroskedastic (ARCH) family models are still used, by practitioners in business and economic policy making, as a conditional volatility forecasting models. Furthermore ARCH models still are attracting an interest of the researchers. In this contribution we consider the well known GARCH(1…
Study spectral analysis on lens spaces, proving isospectral lens spaces with prime order fundamental groups.
problem Spectral analysis of the Kohn Laplacian on lens spaces.
method Analog of Weyl's law and isospectral lens spaces with prime order fundamental groups.
result Two 3D lens spaces with prime order fundamental groups are isospectral with respect to the Kohn Laplacian if and only if they are CR isometric.
Study uncovers scaling laws and spectral properties of shallow neural networks.
problem Understanding scaling laws and spectral properties of shallow neural networks.
method Leveraging connections with matrix compressed sensing and LASSO, derived a phase diagram for excess risk.
result Uncovered crossovers between scaling regimes and plateau behaviors, validated empirical observations.
Study spectral properties of sub-Riemannian Laplacians, proving quantum ergodicity and heat kernel asymptotics.
problem Spectral properties of sub-Riemannian Laplacians.
method Quantum ergodicity results, small-time asymptotics of sub-Riemannian heat kernels, Weyl law.
result Weyl law and spectral concentration on Lie brackets of length r-1.
Study spectral distribution of twisted Laplacian on high genus hyperbolic surfaces.
problem Estimating spectral distribution of twisted Laplacian on hyperbolic surfaces.
method Estimate spectral distribution by supremum norm of harmonic form; show small supremum norm for high genus surfaces; prove uniform Weyl law.
result Prove uniform Weyl law for real parts of spectrum on high genus hyperbolic surfaces.
Develops mixed quantization for graph vector bundles.
problem Solving asymptotic spectral problems on graph vector bundles.
method Mixed quantization technique for graph vector bundles.
result Applications to various spectral problems.
Sharp theory of neural network scaling laws for hierarchical targets.
problem Learning hierarchical multi-index models in neural networks.
method Sharp information-theoretic scaling laws derived for two-layer neural networks.
result Optimal rates achieved by a simple spectral estimator.
Muon spectral optimizer outperforms SGD in associative memory tasks.
problem Understanding the advantage of spectral optimizers in learning associative memory.
method Linear associative memory problem, Gaussian inputs and outputs, power law frequency distribution, thresholded gradient approximation.
result Muon significantly outperforms SGD in storage capacity and recovery rates.
The paper proves geometric and spectral alignment for deep neural networks.
problem Understanding the singular spectra of deep neural network layers.
method Proves deterministic quotient-geometric estimates for singular spectra of Frobenius-normalized layer factors.
result Exact power-law spectra form a trace-normalized Cartan orbit under Frobenius normalization.
We obtain asymptotic lower bounds for the spectral function of the Laplacian and for the remainder in local Weyl's law on manifolds. In the negatively curved case, thermodynamic formalism is applied to improve the estimates. Key ingredients of the proof include the wave equation parametrix, a pretrace formula and the D…
Optimizer choice affects neural scaling laws, changing the exponent α.
problem The exponent α in neural scaling laws L(N)∝N−α varies with the optimizer used. method Controlled random-feature regression experiments with five optimizer variants and six spectral conditions.
result Preconditioned optimizers yield steeper scaling (larger α), with the α-shift increasing across most of the tested spectral range. LLT transforms time series features based on linear laws.
problem Classifying univariate and multivariate time series.
method Time-delay embedding, spectral decomposition, and feature transformation.
result Transformed features improve classification accuracy.
Researchers prove Weyl laws for Schrödinger operators on noncompact manifolds.
problem Proving Weyl laws for Schrödinger operators on noncompact manifolds.
method Heat kernel asymptotics, Karamata-Hardy-Littlewood Tauberian theorem, and semiclassical analysis.
result Established both classical and semiclassical Weyl laws for Schrödinger operators on noncompact manifolds.
Study shows how anisotropic data affects learning dynamics in phase retrieval.
problem Understanding learning dynamics in phase retrieval with anisotropic Gaussian inputs.
method Developed a tractable reduction to reveal a three-phase trajectory and derived scaling laws.
result Found that anisotropy leads to a three-phase trajectory: fast escape, slow convergence, and spectral-tail learning.
Power-law spectrum of random feature model is preserved in neural networks.
problem Preserving power-law spectrum in neural networks through random feature model.
method Characterized eigenvalues of population random-feature covariance using dyadic head-tail decomposition and Wick chaos expansions.
result Power-law exponent α is inherited from input covariance, modified by a logarithmic correction. New models explain heavy-tailed behavior in neural networks.
problem Heavy-tailed spectral densities in neural networks.
method High-temperature Marchenko-Pastur (HTMP) ensemble models.
result Heavy-tailed behavior arises from three factors: data structure, training temperature, and eigenvector entropy.
Ridge regression linked to Poisson resetting in statistical physics.
problem Understanding and extending ridge regularization in machine learning.
method Connecting stochastic resetting from statistical physics with ridge regularization in machine learning, using renewal processes.
result Exact filter identities for ridge regularization in various reset laws, including exponential and non-exponential.
Let (M, g) be a compact smooth Riemannian manifold. We obtain new off-diagonal estimates as λ tend to infinity for the remainder in the pointwise Weyl Law for the kernel of the spectral projector of the Laplacian onto functions with frequency at most λ. A corollary is that, when rescaled around a non self-focal point, …
Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.
problem Understanding the scaling behavior of non-Markovian point processes.
method Analyzed a confined fractional Brownian motion-driven point process and compared it to a nonlinear Markovian process.
result A nonlinear Markovian process can reproduce the power-law scaling behavior of a non-Markovian point process.
Study how generalization scales with model size and data in quadratic neural networks.
problem Understanding how generalization scales with model size and data in quadratic neural networks.
method Analyzed ℓ2-regularized empirical test error minimization in a quadratic two-layer network with finite-sample setting and structured data. result Revealed a phase diagram with distinct scaling regimes as the number of parameters varies, showing data-dependent power laws controlled by spectral structure of the target.
New findings show neural network training loss follows a power law over time.
problem Understanding the optimization process of neural networks during training.
method Spectral analysis of the integral operator representing the linearized evolution of a large network.
result The loss function in neural network training follows a power law behavior, L(t)∼t−ξ, with exponent ξ determined by network parameters and data characteristics. Signals consisting of a sequence of pulses show that inherent origin of the 1/f noise is a Brownian fluctuation of the average interevent time between subsequent pulses of the pulse sequence. In this paper we generalize the model of interevent time to reproduce a variety of self-affine time series exhibiting power spec…
The exterior differential system for constant mean curvature (CMC) surfaces in a 3-dimensional space form is an elliptic Monge-Ampere system defined on the unit tangent bundle. We determine the infinite sequence of higher-order symmetries and conservation laws via an enhanced prolongation modelled on a loop algebra val…
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
Study analyzes spectral properties on specific geometric spaces.
problem Investigates spectral analysis on standard locally homogeneous spaces.
method Uses branching laws and invariant differential operators on spherical homogeneous spaces.
result Proves essential self-adjointness and infinite point spectrum for certain spaces.
Researchers found a Weyl law for Liouville quantum gravity eigenvalues.
problem Understanding the spectral geometry of Liouville quantum gravity.
method Obtained a Weyl law for eigenvalues of Liouville Brownian motion.
result The n-th eigenvalue grows linearly with n, with a constant determined by the Liouville area and a specific cγ. Modeling financial market dynamics with 2D Levy flights.
problem Capturing the complex, scaling laws in financial market dynamics.
method 2D Lévy flight model applied to S\&P 500 index prices.
result Empirical spectral properties match model predictions.
Study heavy-tailed weights' impact on neural network's spectral distribution.
problem Analyzing spectral distribution of conjugate kernel matrices with heavy-tailed weights.
method Computed limiting eigenvalue distribution through moments, considering heavy-tailed distributions and nonlinear activation functions.
result Heavy-tailed weights induce strong correlations, leading to fundamentally different spectral behavior.
This paper considers magnitude, asymptotics and duration of drawdowns for some Lévy processes. First, we revisit some existing results on the magnitude of drawdowns for spectrally negative Lévy processes using an approximation approach. For any spectrally negative Lévy process whose scale functions are well-behaved at …
We perform a parallel analysis of the spectral density of (i) the logarithm of price and (ii) the daily number of trades of a set of stocks traded in the New York Stock Exchange. The stocks are selected to be representative of a wide range of stock capitalization. The observed spectral densities show a different power-…
This article concerns new off-diagonal estimates on the remainder and its derivatives in the pointwise Weyl law on a compact n-dimensional Riemannian manifold. As an application, we prove that near any non self-focal point, the scaling limit of the spectral projector of the Laplacian onto frequency windows of constant …
Analyzes how diffusion models learn, revealing a spectral bias in structure mastery.
problem Understanding the learning dynamics and bias in diffusion models.
method Developed an analytical framework using a Gaussian-equivalence principle to solve gradient-flow dynamics and integrate probability-flow ODEs.
result Exposes a universal inverse-variance spectral law: high-variance structure is mastered faster than low-variance detail.
Study on KRR with power-law data, showing better sample complexity.
problem High-dimensional kernel ridge regression with anisotropic power-law covariance.
method Explicit characterization of kernel spectrum and asymptotic analysis of excess risk.
result Sample complexity is governed by effective dimension, not ambient dimension.
Algorithms based on spectral graph cut objectives such as normalized cuts, ratio cuts and ratio association have become popular in recent years because they are widely applicable and simple to implement via standard eigenvector computations. Despite strong performance for a number of clustering tasks, spectral graph cu…
Large models follow power laws in performance with dataset size or parameters.
problem Understanding neural scaling laws in large language models.
method Joint generative data model and random feature model.
result Modeling and solving the dual limit reveals insights into scaling laws.
Robust CD method for real-world time series with power-law distributions.
problem Challenges in causal discovery due to noise sensitivity.
method Power-law spectral feature extraction for robust CD.
result Consistently outperforms state-of-the-art alternatives on real-world datasets.
Study on sub-Riemannian geometry in 4D, focusing on abnormal geodesics.
problem Understanding sub-Riemannian geometry and its spectrum.
method Wave trace expansion, Weyl laws, propagation of singularities, quantum ergodicity.
result First appearance of abnormal geodesics in sub-Riemannian spectral geometry.
A hierarchical model shows how scaling laws emerge from sequential feature recovery.
problem Emergence of scaling laws from feature learning in multi-layer networks.
method Layer-wise spectral algorithm adapted to compositional structure, sequential feature detection.
result Sequential detection of latent features, leading to explicit power-law decay of prediction error.
We propose an extension of the differential system for constant mean curvature (CMC) surfaces in a three dimensional space form to an associated hierarchy of evolution equations by the higher-order commuting symmetries. The infinite sequence of higher-order conservation laws of CMC surfaces admit the corresponding exte…
We study spectral properties of the Laplace-Beltrami operator on two relevant almost-Riemannian manifolds, namely the Grushin structures on the cylinder and on the sphere. This operator contains first order diverging terms caused by the divergence of the volume. We get explicit descriptions of the spectrum and the eige…
The study shows how geometric Weyl bulk-density exponent rigidifies spectral encodings in O-regularly varying classes.
problem Understanding spectral encodings under Weyl growth conditions.
method Analyzing geometric Weyl bulk-density exponent and proving spectral rigidity.
result The geometric Weyl bulk-density exponent (d−2)/2 rigidifies spectral encodings in the O-regularly varying class, leading to unique admissible exponents and scaling laws. Study reveals 1/f noise in signals made from nonoverlapping rectangular pulses.
problem Analyzing 1/f noise in signals composed of nonoverlapping pulses. method Derived a general formula for power spectral density, analyzed rectangular pulse case.
result Observed pure 1/f noise until very low frequencies with long pulse durations. Analysis of DPPs and k-DPPs via spectral decomposition reveals identifiable parameters and non-identifiability gaps.
problem Identifying parameters of DPPs and k-DPPs through spectral decomposition.
method Spectral decomposition of the covariance matrix, analysis of invariances, and counting arguments.
result Identifiability of parameters changes fundamentally for k-DPPs, with specific invariances and non-identifiability gaps.
The paper studies neural networks with wide layers and finds a deformed semicircle law.
problem Investigating spectral distributions of neural networks in the ultra-wide regime.
method Analyzes empirical kernel matrices, proves deformed semicircle law, provides nonlinear Hanson-Wright inequality.
result Emergence of a deformed semicircle law in the ultra-wide neural network regime.
We explore the geometry of the Nahm-Schmid equations, a version of Nahm's equations in split signature. Our discussion ties up different aspects of their integrable nature: dimensional reduction from the Yang--Mills anti-self-duality equations, explicit solutions, Lax-pair formulation, conservation laws and spectral cu…