Non-Markovian point process shows power-law scaling, similar to nonlinear Markovian process.
problem Understanding the scaling behavior of non-Markovian point processes.
method Analyzed a confined fractional Brownian motion-driven point process and compared it to a nonlinear Markovian process.
result A nonlinear Markovian process can reproduce the power-law scaling behavior of a non-Markovian point process.
This work analyzes neural scaling laws using power-law data spectra and derives analytical expressions for generalization error.
problem Understanding how neural network performance scales with key factors like data size and model complexity.
method Statistical mechanics techniques applied to one-pass stochastic gradient descent in a student-teacher framework.
result Derivation of analytical expressions for generalization error under power-law data spectra and identification of conditions for power-law scaling.
Large models follow power laws in performance with dataset size or parameters.
problem Understanding neural scaling laws in large language models.
method Joint generative data model and random feature model.
result Modeling and solving the dual limit reveals insights into scaling laws.
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.
LLMs learn peaked distributions slowly due to power-law losses.
problem Slow convergence of loss in training large language models.
method Systematic analysis of toy models and empirical evaluation of LLMs.
result Power-law time scaling with an exponent of 1/3 for learning peaked distributions.
New pruning method breaks power law scaling, potentially reducing error to exponential.
problem Improving neural network performance through scaling alone is costly.
method Developed a new data pruning metric to break power law scaling.
result Pruned datasets show better than power law scaling on various image datasets.
Develops a simple model to understand learning curves for arbitrary power laws.
problem Lack of theoretical understanding of scaling laws in machine learning.
method Analyzes a toy model to determine if learning curves are universal or depend on data distribution.
result Determines that learning curves can exhibit n−β for arbitrary power β>0. We extend our previous study of scaling range properties done for detrended fluctuation analysis (DFA) \cite{former_paper} to other techniques of fluctuation analysis (FA). The new technique called Modified Detrended Moving Average Analysis (MDMA) is introduced and its scaling range properties are examined and compared…
Power-law portfolios improve diversification by scaling weights sub-linearly.
problem Optimization methods struggle with unstable pair correlations and non-Gaussian risk measures.
method Construct portfolios with penalty proportional to arbitrary order moment of returns, leading to sub-linear weight scaling.
result Infinite order power-law portfolios are perfectly diversified, improving diversification over Kelly portfolios.
We study distributions which have both fractal and non-fractal scale regions by introducing a typical scale into a scale invariant system. As one of models in which distributions follow power law in the large scale region and deviate further from the power law in the smaller scale region, we employ 2-dim quantum gravit…
We study the growth dynamics of the size of manufacturing firms considering competition and normal distribution of competency. We start with the fact that all components of the system struggle with each other for growth as happened in real competitive bussiness world. The detailed quantitative agreement of the theory w…
We introduce the stochastic multiplicative point process modelling trading activity of financial markets. Such a model system exhibits power-law spectral density S(f) ~ 1/f**beta, scaled as power of frequency for various values of beta between 0.5 and 2. Furthermore, we analyze the relation between the power-law autoco…
Analyzed a generalized voter model with power-law herding intensity, revealing anomalous diffusion and long-range memory.
problem Anomalous diffusion and long-range memory in a generalized voter model.
method Derived analytical expressions for moments and first passage time distribution, confirmed numerically.
result The model exhibits long-range memory indicators despite being a Markov model.
Superposition accelerates training to a universal power-law exponent.
problem Training dynamics in neural networks.
method Teacher-student framework and analytic theory.
result Superposition leads to a universal power-law exponent of ~1, independent of data and channel statistics.
Deep learning (DL) creates impactful advances following a virtuous recipe: model architecture search, creating large training data sets, and scaling computation. It is widely believed that growing training sets and models should improve accuracy and result in better products. As DL application domains grow, we would li…
Scaling laws found for reinforcement learning performance with model size and compute.
problem Challenges in extending generative modeling scaling laws to reinforcement learning.
method Introduced intrinsic performance as a monotonic function of mean episode return.
result Intrinsic performance scales as a power law in model size and environment interactions.
We consider an ideal closed stock market, in which 100 traders have economic activities. The assets of the traders change through buying and selling stocks. We simulate the assets under conservation of both total currency and total number of stocks. If the traders are identical, then the assets are distributed as a sta…
New mechanism found for power laws including Zipf's law.
problem Understanding the ubiquity of power law distributions.
method Introduced nonlinear self-excited Hawkes processes with fast-accelerating intensities.
result Wide class of nonlinear Hawkes processes have power law intensity PDFs.
We analyze a database comprising quarterly sales of 55624 pharmaceutical products commercialized by 3939 pharmaceutical firms in the period 1992--2001. We study the probability density function (PDF) of growth in firms and product sales and find that the width of the PDF of growth decays with the sales as a power law w…
Our purpose is to relate the Fokker-Planck formalism proposed by [Friedrich et al., Phys. Rev. Lett. 84, 5224 (2000)] for the distribution of stock market returns to the empirically well-established power law distribution with an exponent in the range 3-5. We show how to use Friedrich et al.'s formalism to predict that…
We present new findings in regard to data analysis in very high dimensional spaces. We use dimensionalities up to around one million. A particular benefit of Correspondence Analysis is its suitability for carrying out an orthonormal mapping, or scaling, of power law distributed data. Power law distributed data are foun…
A hierarchical model shows how scaling laws emerge from sequential feature recovery.
problem Emergence of scaling laws from feature learning in multi-layer networks.
method Layer-wise spectral algorithm adapted to compositional structure, sequential feature detection.
result Sequential detection of latent features, leading to explicit power-law decay of prediction error.
Improved scaling laws in linear regression using data reuse.
problem Sustainability of neural scaling laws when running out of new data.
method Data reuse in multi-pass stochastic gradient descent (multi-pass SGD) for M-dimensional linear models trained on N data with sketched features. result Multi-pass SGD achieves a test error of Θ(M1−b+L(1−b)/a) with L>N, improving scaling laws in data-constrained regimes. We study the relaxation dynamics of a financial market just after the occurrence of a crash by investigating the number of times the absolute value of an index return is exceeding a given threshold value. We show that the empirical observation of a power law evolution of the number of events exceeding the selected thre…
Study shows how anisotropic data affects learning dynamics in phase retrieval.
problem Understanding learning dynamics in phase retrieval with anisotropic Gaussian inputs.
method Developed a tractable reduction to reveal a three-phase trajectory and derived scaling laws.
result Found that anisotropy leads to a three-phase trajectory: fast escape, slow convergence, and spectral-tail learning.
The paper examines the short-time implied volatility of additive processes and finds key parameters.
problem Characterizing the short-time implied volatility of equity markets.
method Examined pure jump exponential additive processes with power-law scaling parameters.
result The implied volatility is consistent with equity market characteristics if and only if β=1 and δ=-1/2.
Detection of power-law behavior and studies of scaling exponents uncover the characteristics of complexity in many real world phenomena. The complexity of financial markets has always presented challenging issues and provided interesting findings, such as the inverse cubic law in the tails of stock price fluctuation di…
Study reveals neural scaling laws in random graphs and natural language models.
problem Understanding the origin of neural scaling laws in complex systems.
method Examined scaling laws in transformers trained on random walks and simplified natural language models.
result Neural scaling laws emerge in the absence of power law structure in data correlations.
Modeling financial returns as conditionally independent random variables explains power-law tails.
problem Understanding the distribution of financial returns and their relation to volatility.
method Assuming returns are conditionally independent given volatility, which varies randomly over time.
result Returns distribution can be described by the sum of conditionally independent random variables, showing scaling and power-law tails.
We present a model for random simple graphs with a degree distribution that obeys a power law (i.e., is heavy-tailed). To attain this behavior, the edge probabilities in the graph are constructed from Bertoin-Fujita-Roynette-Yor (BFRY) random variables, which have been recently utilized in Bayesian statistics for the c…
We consider the scaling behaviors for fluctuations of the number of Korean firms bankrupted in the period from August 1 2002 to October 28 2003. We observe a power law for the distribution of the number of the bankrupted firms. The Pareto exponent is close to unity. We also consider the daily increments of the number o…
We study the statistical properties of volatility---a measure of how much the market is likely to fluctuate. We estimate the volatility by the local average of the absolute price changes. We analyze (a) the S&P 500 stock index for the 13-year period Jan 1984 to Dec 1996 and (b) the market capitalizations of the largest…
A power-law fit to the empirical inference-compute frontier in LOB prediction suggests a scaling-law-style frontier.
problem Limit order book prediction
method Using a suite of models ranging from small decision trees to neural LOB architectures
result A power-law fit to the low- and mid-compute non-MLPLOB frontier extrapolates across multiple orders of magnitude and attains R2=0.941 on the excluded high-compute MLPLOB target frontier. Self-organized criticality has been claimed to play an important role in many natural and social systems. In the present work we empirically investigate the relevance of this theory to stock-market dynamics. Avalanches in stock-market indices are identified using a multi-scale wavelet-filtering analysis designed to rem…
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
P2P lending activities have grown rapidly and have caused the huge and complex networks of debtor-creditor relationships. The aim of this study was to study the underlying structural characteristics of networks formed by debtor-creditor relationships. According attributes of P2P lending, this paper model the networks o…
We analyze the sequence of time intervals between consecutive stock trades of thirty companies representing eight sectors of the U. S. economy over a period of four years. For all companies we find that: (i) the probability density function of intertrade times may be fit by a Weibull distribution; (ii) when appropriate…
Study reveals a universal formula for knotting in random equilateral polygons.
problem Probability of knotting in equilateral random polygons.
method Extensive Monte Carlo simulations with improved algorithms and knot invariants.
result A universal scaling formula for knotting probability with number of edges, involving exponential and power law factors.
Based on empirical financial time-series, we show that the "silence-breaking" probability follows a super-universal power law: the probability of observing a large movement is inversely proportional to the length of the on-going low-variability period. Such a scaling law has been previously predicted theoretically [R. …
The importance of the power law has been well realized in econophysics over the last decade. For instance, the distribution of the rate of stock price variation and of personal assets show the power law. While these results reveal the striking scale invariance of financial markets, the behaviour of price in real econom…
Defines complexity measure for neural networks and feature representations, revealing scaling patterns.
problem Understanding the nonlinearity and dimensionality of neural network computations and feature representations.
method Introduces complexity and effective dimension measures, investigates their dynamics during training, and analyzes their scaling properties.
result Power law scaling of complexity and effective dimension during training, revealing hidden structure of datasets.
We study by theoretical analysis and by direct numerical simulation the dynamics of a wide class of asynchronous stochastic systems composed of many autocatalytic degrees of freedom. We describe the generic emergence of truncated power laws in the size distribution of their individual elements. The exponents α of the…
Many systems of different nature exhibit scale free behaviors. Economic systems with power law distribution in the wealth is one of the examples. To better understand the working behind the complexity, we undertook an empirical study measuring the interactions between market participants. A Web server was setup to admi…
It is now well established empirically that financial price changes are distributed according to a power law, with cubic exponent. This is a fascinating regularity, as it holds for various classes of securities, on various markets, and on various time scales. The universality of this law suggests that there must be som…
We uncover scaling laws and statistical structure in complex datasets.
problem Understanding universal traits in complex datasets.
method Analogizing data to physical systems, using statistical physics and RMT.
result Real-world datasets and Gaussian data with long-range correlations share the same RMT universality class.
Proposes a new tail risk measure based on the most probable maximum risk event size.
problem Current risk measures like VaR and ES are limited in their applicability and require specifying a confidence level.
method Develops a new risk measure called MPMR that does not require a confidence level and scales with the length of the time interval.
result The new risk measure, MPMR, scales with the number of observations by a power law, allowing for reliable estimations of long-term risks based on short-term estimations.
New findings show neural network training loss follows a power law over time.
problem Understanding the optimization process of neural networks during training.
method Spectral analysis of the integral operator representing the linearized evolution of a large network.
result The loss function in neural network training follows a power law behavior, L(t)∼t−ξ, with exponent ξ determined by network parameters and data characteristics. This study reveals statistical patterns in ERC20 token transactions on Ethereum blockchain.
problem Understanding transactional dynamics in decentralized systems.
method Examined over 44 million ERC20 token transfers, categorized by address type (EOA or SC), and analyzed using scaling laws.
result EOA-driven transactions exhibit consistent statistical behavior, while SC-driven activity displays sublinear scaling and bursty activity.