arXiv research

New principles needed for scaling large language models, challenging traditional regularization methods.

Study uncovers scaling laws and spectral properties of shallow neural networks.

Decomposing market impact into diffusive components

We study how the presence of correlations in physical variables contributes to the form of probability distributions. We investigate a process with correlations in the variance generated by (i) a Gaussian or (ii) a truncated Lévy distribution. For both (i) and (ii), we find that due to the correlations in the variance,…

We propose a simple stochastic volatility model which is analytically tractable, very easy to simulate and which captures some relevant stylized facts of financial assets, including scaling properties. In particular, the model displays a crossover in the log-return distribution from power-law tails (small time) to a Ga…

The statistical properties of the multipliers of the absolute returns are investigated using one-minute high-frequency data of financial time series. The multiplier distribution is found to be independent of the box size $s$ when $s$ is larger than some crossover scale, providing direct evidence of the existence of sca…

We investigate the waiting-time distribution of the absolute return in the Korean stock-market index KOSPI. We define the waiting time as a time interval during which the normalized absolute return remains continuously below a threshold $r_c$. Through an exponential bin plot, we observe that the waiting-time distributi…

A new law limits kurtosis contrast in balanced mixtures.

Study on model collapse in regression models, proposing a mitigation strategy.

Based on the tick-by-tick price changes of the companies from the U.S. and from the German stock markets over the period 1998-99 we reanalyse several characteristics established by the Boston Group for the U.S. market in the period 1994-95, which serves to verify their space and time-translational invariance. By increa…

Using a large database of 8 million institutional trades executed in the U.S. equity market, we establish a clear crossover between a linear market impact regime and a square-root regime as a function of the volume of the order. Our empirical results are remarkably well explained by a recently proposed dynamical theory…

The detrending moving average (DMA) algorithm is one of the best performing methods to quantify the long-term correlations in nonstationary time series. Many long-term correlated time series in real systems contain various trends. We investigate the effects of polynomial trends on the scaling behaviors and the performa…

Unified thermodynamic approach to Transformer attention dynamics.

We analyze the sequence of time intervals between consecutive stock trades of thirty companies representing eight sectors of the U. S. economy over a period of four years. For all companies we find that: (i) the probability density function of intertrade times may be fit by a Weibull distribution; (ii) when appropriate…

Dropout schedules can be optimized to significantly reduce model test loss.

The herd behavior of returns is investigated in Korean futures exchange market. It is obtained that the probability distribution of returns for three types of herding parameter scales as a power law $R^{-β}$ with the exponents $β=3.6$(KTB203) and 2.9(KTB209) in two kinds of Korean treasury bond. For our case since the…

We study the statistical properties of volatility---a measure of how much the market is likely to fluctuate. We estimate the volatility by the local average of the absolute price changes. We analyze (a) the S&P 500 stock index for the 13-year period Jan 1984 to Dec 1996 and (b) the market capitalizations of the largest…

We study the probability distribution of stock returns at mesoscopic time lags (return horizons) ranging from about an hour to about a month. While at shorter microscopic time lags the distribution has power-law tails, for mesoscopic times the bulk of the distribution (more than 99% of the probability) follows an expon…

Long-range correlation in financial time series reflects the complex dynamics of the stock markets driven by algorithms and human decisions. Our analysis exploits ultra-high frequency order book data from NASDAQ Nordic over a period of three years to numerically estimate the power-law scaling exponents using detrended …

An exclusion particle model is considered as a highly simplified model of a limit order market. Its price behavior reproduces the well known crossover from over-diffusion (Hurst exponent H>1/2) to diffusion (H=1/2) when the time horizon is increased, provided that orders are allowed to be canceled. For early times a ma…

Optimal income crossover found using particle swarm optimization.

Several models of stock trading [P. Bak et al, Physica A {\bf 246}, 430 (1997)] are analyzed in analogy with one-dimensional, two-species reaction-diffusion-branching processes. Using heuristic and scaling arguments, we show that the short-time market price variation is subdiffusive with a Hurst exponent $H=1/4$. Biase…

We investigate the temporal correlations and multifractal nature of trading volume of 22 liquid stocks traded on the Shenzhen Stock Exchange in 2003. We find that the trading volume exhibit size-dependent non-universal long memory and multifractal nature. No crossover in the power-law dependence of the detrended fluctu…

We study the tick dynamical behavior of the yen-dollar exchange rate using the rescaled range analysis in financial market. It is found that the multifractal Hurst exponents with the short and long-run memory effects can be obtained from the yen-dollar exchange rate. This exists one crossover for the Hurst exponents at…

The intraday pattern, long memory, and multifractal nature of the intertrade durations, which are defined as the waiting times between two consecutive transactions, are investigated based upon the limit order book data and order flows of 23 liquid Chinese stocks listed on the Shenzhen Stock Exchange in 2003. An inverse…

The study examines Kernel Ridge Regression error rates across noiseless and noisy conditions.

The paper analyzes return distribution of Chinese stock market indices over various time scales.

The diagonal effect of orders is well documented in different markets, which states that orders are more likely to be followed by orders of the same aggressiveness and implies the presence of short-term correlations in order flows. Based on the order flow data of 43 Chinese stocks, we investigate if there are long-rang…

We investigate the herd behavior of returns for the yen-dollar exchange rate in the Japanese financial market. It is obtained that the probability distribution $P(R)$ of returns $R$ satisfies the power-law behavior $P(R) \simeq R^{-β}$ with the exponents $β=3.11$(the time interval $τ=$ one minute) and 3.36($τ=$ one da…

This work explains scaling laws as redundancy laws in deep learning.

The correlation function of a financial index of the New York stock exchange, the S&P 500, is analyzed at 1 min intervals over the 13-year period, Jan 84 -- Dec 96. We quantify the correlations of the absolute values of the index increment. We find that these correlations can be described by two different power laws wi…

A new scaling law predicts optimal batch size for training models.

Large models follow power laws in performance with dataset size or parameters.

This work extends the scaling law to multiple and kernel regression, challenging traditional machine learning principles.

Study reveals neural scaling laws in random graphs and natural language models.

This work analyzes neural scaling laws using power-law data spectra and derives analytical expressions for generalization error.

Unified theory for neural scaling laws in hierarchically compositional data.

New scaling laws explain deep learning performance growth.

Scaling laws in linear regression explain model performance improvements with size and data.

Employing profits data of Japanese companies in 2002 and 2003, we confirm that Pareto's law and the Pareto index are derived from the law of detailed balance and Gibrat's law. The last two laws are observed beyond the region where Pareto's law holds. By classifying companies into job categories, we find that companies …

New pruning method breaks power law scaling, potentially reducing error to exponential.

The herd behaviors of returns for the won-dollar exchange rate and the KOSPI are analyzed in Korean financial markets. It is shown that the probability distribution $P(R)$ of price returns $R$ for three values of the herding parameter tends to a power-law behavior $P(R) \simeq R^{-β}$ with the exponents $β=2.2$(the wo…

We show that an economic system populated by multiple agents generates an equilibrium distribution in the form of multiple scaling laws of conditional PDFs, which are sufficient for characterizing the probability distribution. The existence of the double scaling law is demonstrated empirically for the sales and the lab…

The study explains transformer scaling laws using statistical and approximation theories.

Scaling laws govern predictive uncertainties in deep learning models.

New neural scaling law found for simple quadratic function.

Bayesian theory explains market impact of large trades.

Improved scaling laws in linear regression using data reuse.