arXiv research

Three training regimes found for scale-invariant neural networks on the sphere.

This paper proposes a multi-scale Markov-Switching GARCH model for EUR/USD volatility.

Study uncovers scaling laws and spectral properties of shallow neural networks.

We investigate multifractality in the Korean stock-market index KOSPI. The generalized $q$th order height-height correlation function shows multiscaling properties. There are two scaling regimes with a crossover time around $t_c =40$ min. We consider the original data sets and the modified data sets obtained by removin…

Theory explains neural network scaling with dataset and model size.

We discuss the approximation of the value function for infinite-horizon discounted Markov Reward Processes (MRP) with nonlinear functions trained with the Temporal-Difference (TD) learning algorithm. We first consider this problem under a certain scaling of the approximating function, leading to a regime called lazy tr…

We develop a second-order model for limit order books in a single scaling regime.

New scaling framework for MoE architectures ensures stability and optimal performance at scale.

Large learning rates work surprisingly well in standard parameterization, contrary to theory.

Data pruning algorithms struggle in high compression regimes, as shown by theoretical and empirical studies.

Study shows optimal model performance at critical level of feature learning.

The paper analyzes SGD in high-dimensional networks, revealing new scaling limits.

When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large scale data applications like genomics, connectomics, and eco-informatics the dataset…

Model shows feature learning can improve neural scaling laws for hard tasks.

Framework models multiscale dynamics with Bayesian learning for regime changes.

Empirical analysis of financial market trends and reversions across various time scales.

Study how generalization scales with model size and data in quadratic neural networks.

Two distinct limits for deep learning have been derived as the network width $h\rightarrow \infty$, depending on how the weights of the last layer scale with $h$. In the Neural Tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described by a frozen kernel $Θ$. By contrast, in the Mean-Field …

Study on how initialization scale affects neural network training regimes.

Sharp feature transitions revealed in extensive-width networks.

Model shows loss curve with two distinct exponents due to sparse activations.

Scaling ResNets requires careful consideration of the layer depth and output scaling factors.

The paper analyzes learning curves for kernel ridge regression with dot-product kernels.

Using a large database of 8 million institutional trades executed in the U.S. equity market, we establish a clear crossover between a linear market impact regime and a square-root regime as a function of the volume of the order. Our empirical results are remarkably well explained by a recently proposed dynamical theory…

The paper explains the richness scale of wide neural networks.

Develops a new framework to analyze gradient flow regimes and derive explicit solutions.

We analyze architectural features of Deep Neural Networks (DNNs) using the so-called Neural Tangent Kernel (NTK), which describes the training and generalization of DNNs in the infinite-width setting. In this setting, we show that for fully-connected DNNs, as the depth grows, two regimes appear: "order", where the (sca…

The study provides precise asymptotic theory for in-context learning by Transformers.

A new approach is presented to describe the change in the statistics of the log return distribution of financial data as a function of the timescale. To this purpose a measure is introduced, which quantifies the distance of a considered distribution to a reference distribution. The existence of a small timescale regime…

Bayesian neural networks reveal multimodal predictive distributions.

Decomposing market impact into diffusive components

New model captures asymmetric rough volatility with Zumbach effect.

Gaussian equivalence fails for simple polynomial embeddings in quadratic scaling RF models.

Study on SGD dynamics and scaling laws for training quadratic neural networks in high dimensions.

This work bridges two views of feature learning in neural networks.

This paper optimizes importance sampling for rare-event options pricing under the Heston model.

New dynamics for SGD in small learning rate regime.

Optimizes dividend control in a bankruptcy process using a special Levy process.

LINTEL improves INTEL's time series prediction by optimizing computation and accuracy.

Optimization of neural networks scales with γ, revealing unique loss curves and optimal learning rates.

Develops identifiability theory for multi-lag regime-switching models.

Log-normal continuous random cascades form a class of multifractal processes that has already been successfully used in various fields. Several statistical issues related to this model are studied. We first make a quick but extensive review of their main properties and show that most of these properties can be analytic…

Superstatistics is a widely employed tool of non-equilibrium statistical physics which plays an important role in analysis of hierarchical complex dynamical systems. Yet, its "canonical" formulation in terms of a single nuisance parameter is often too restrictive when applied to complex empirical data. Here we show tha…

Ensembles of random-feature models can't outperform a single large model.

We analyze the Hessian spectra of large models up to 100B parameters.

New algorithm learns switching dynamics from multiple neural signals.

Embedded ensembles improve neural network performance efficiently.

Study on deep multi-head self-attention dynamics, proving homogenized limits under specific scalings.