arXiv research

Mamba struggles with long context lengths, but spectrum scaling improves performance.

Improved scaling laws in linear regression using data reuse.

We apply state-of-the-art tools in modern high-dimensional numerical linear algebra to approximate efficiently the spectrum of the Hessian of modern deepnets, with tens of millions of parameters, trained on real data. Our results corroborate previous findings, based on small-scale networks, that the Hessian exhibits "s…

Study reveals an equivalence principle for the spectrum of random inner-product kernel matrices in polynomial scaling.

This study investigates that a characteristic time scale on an exchange rate market (USD/JPY) is examined for the period of 1998 to 2000. Calculating power spectrum densities for the number of tick quotes per minute and averaging them over the year yield that the mean power spectrum density has a peak at high frequenci…

Power spectrum densities for the number of tick quotes per minute (market activity) on three currency markets (USD/JPY, EUR/USD, and JPY/EUR) for periods from January 1999 to December 2000 are analyzed. We find some peaks on the power spectrum densities at a few minutes. We develop the double-threshold agent model and …

The cross correlation matrix between equities comprises multiple interactions between traders with varying strategies and time horizons. In this paper, we use the Maximum Overlap Discrete Wavelet Transform to calculate correlation matrices over different timescales and then explore the eigenvalue spectrum over sliding …

Muon outperforms GD in associative memory learning by balancing frequency components.

Prove first-band large-diameter asymptotics for Dirichlet spectrum on horoconvex domains in real hyperbolic space.

Generative models improve for multiscale scientific data with new noise and interpolation techniques.

This paper proposes a new Nystrom-based clustering algorithm for large-scale data.

Scaling laws in linear regression explain model performance improvements with size and data.

We introduce a notion of "effective dimension" of a statistical model based on the number of cubes of size $1/\sqrt{n}$ needed to cover the model space when endowed with the Fisher Information Matrix as metric, $n$ being the number of observations. The number of observations fixes a natural scale or resolution. The eff…

We propose a new method of learning a sparse nonnegative-definite target matrix. Our primary example of the target matrix is the inverse of a population covariance or correlation matrix. The algorithm first estimates each column of the target matrix by the scaled Lasso and then adjusts the matrix estimator to be symmet…

Analyzes Hessian spectrum for neural networks near optimal learning.

Recently the statistical characterizations of financial markets based on physics concepts and methods attract considerable attentions. We used two possible procedures of analyzing multifractal properties of a time series. The first one uses the continuous wavelet transform and extracts scaling exponents from the wavele…

While the harmonic function solution performs well in many semi-supervised learning (SSL) tasks, it is known to scale poorly with the number of samples. Recent successful and scalable methods, such as the eigenfunction method focus on efficiently approximating the whole spectrum of the graph Laplacian constructed from …

Jets from boosted heavy particles have a typical angular scale which can be used to distinguish them from QCD jets. We introduce a machine learning strategy for jet substructure analysis using a spectral function on the angular scale. The angular spectrum allows us to scan energy deposits over the angle between a pair …

There is a large body of work, built on tools developed in mathematics and physics, demonstrating that financial market prices exhibit self-similarity at different scales. In this paper, we explore the use of analytical topology to characterize financial price series. While wavelet and Fourier transforms decompose a si…

Variant of mSSA improves time series prediction error.

Radio emitter recognition in dense multi-user environments is an important tool for optimizing spectrum utilization, identifying and minimizing interference, and enforcing spectrum policy. Radio data is readily available and easy to obtain from an antenna, but labeled and curated data is often scarce making supervised …

This study improves graph coarsening methods by preserving graph spectrum and distances.

The execution flow drives market dynamics, validated on real data.

A new debiasing method for high-dimensional regression with applications to PCR.

Weak lensing maps contain information beyond two-point statistics on small scales. Much recent work has tried to extract this information through a range of different observables or via nonlinear transformations of the lensing field. Here we train and apply a 2D convolutional neural network to simulated noiseless lensi…

Paper introduces models to discover complex structures in large hypergraphs.

To understand the dynamics of optimization in deep neural networks, we develop a tool to study the evolution of the entire Hessian spectrum throughout the optimization process. Using this, we study a number of hypotheses concerning smoothness, curvature, and sharpness in the deep learning literature. We then thoroughly…

The miltifractal properties and scaling behaviour of the exchange rate variations of the Iranian rial against the US dollar from a daily perspective is numerically investigated. For this purpose the multifractal detrended fluctuation analysis (MF-DFA) is used. Through multifractal analysis, the scaling exponents, gener…

Paper assesses risks of stablecoins, from lending to business-to-business.

In the framework of Multifractal Diffusion Entropy Analysis we propose a method for choosing an optimal bin-width in histograms generated from underlying probability distributions of interest. The method presented uses techniques of Rényi's entropy and the mean squared error analysis to discuss the conditions under whi…

We perform a systematic investigation on the components of the empirical multifractality of financial returns using the daily data of Dow Jones Industrial Average from 26 May 1896 to 27 April 2007 as an example. The temporal structure and fat-tailed distribution of the returns are considered as possible influence facto…

This work explains scaling laws as redundancy laws in deep learning.

Study uncovers scaling laws and spectral properties of shallow neural networks.

Gaussian process regression generally does not scale to beyond a few thousands data points without applying some sort of kernel approximation method. Most approximations focus on the high eigenvalue part of the spectrum of the kernel matrix, $K$, which leads to bad performance when the length scale of the kernel is sma…

We define a new spectrum for compact length spaces and Riemannian manifolds called the "covering spectrum" which roughly measures the size of the one dimensional holes in the space. More specifically, the covering spectrum is a set of real numbers $δ>0$ which identify the distinct $δ$ covers of the space. We investigat…

The subject of this paper is the relationship among the marked length spectrum, the length spectrum, the Laplace spectrum on functions, and the Laplace spectrum on forms on Riemannian nilmanifolds. In particular, we show that for a large class of three-step nilmanifolds, if a pair of nilmanifolds in this class has the …

The subject of this paper is the relationship among the marked length spectrum, the length spectrum, the Laplace spectrum on functions, and the Laplace spectrum on forms on Riemannian nilmanifolds. In particular, we show that for a large class of three-step nilmanifolds, if a pair of nilmanifolds in this class has the …

New learning rates derived for Tikhonov-regularized problems without kernel assumptions.

Study the energy spectrum of metrics on surfaces and its relation to simple length spectrum.

A large set of daily FOREX time series is analyzed. The corresponding correlation matrices (CM) are constructed for USD, EUR and PLZ used as the base currencies. The triangle rule is interpreted as constraints reducing the number of independent returns. The CM spectrum is computed and compared with the cases of shuffle…

Study shows spectrum properties for specific Hadamard manifolds.

Proofs high-dimensional spectrum convergence of weighted sample covariance.

In this article we analyse linear correlation and non-linear dependence of traded volume, $v$, of the 30 constituents of Dow Jones Industrial Average at different value scales. Specifically, we have raised $v$ to some real value $α$ or $β$, which introduces a bias for small ($α, β<0$) or large ($α, β>1$) values. Our r…

Iterative method 'Concent' corrects spectrum bias in covariance matrices.

Theory explains neural network scaling with dataset and model size.

Develops a new spectrum for annular links, recovering a transverse invariant at extreme gradings.

The spectrum of certain manifolds matches that of hyperbolic space if the bottom spectrum is maximal.

Lower bounds for Hodge-Laplacian spectrum on orbifolds.