Sparse deep neural networks follow a power law in their connectivity.
problem Understanding the connectivity patterns in sparse deep neural networks.
method Experimentally tested multilayer perceptrons and convolutional neural networks, proposed an internal preferential attachment model.
result Sparse deep neural networks exhibit a power law in their connectivity, similar to biological neural networks.
This study examines evolving networks of P2P lending relationships, revealing scale-free characteristics and the impact of interest rate and term.
problem Understanding the structural characteristics of evolving networks of debtor-creditor relationships in P2P lending.
method Modeling P2P lending networks as evolving networks with addition and deletion of nodes, analyzing attributes and factors affecting the scale-free exponent.
result P2P lending networks are scale-free with significant influence from interest rate and term on the exponent of power-law.
This work investigates power laws in deep neural network ensembles and predicts their performance.
problem Understanding the performance of deep neural network ensembles and their optimal structure.
method Investigated the behavior of negative log-likelihood (CNLL) of a deep ensemble as a function of ensemble size and member network size, identifying power law dependencies.
result One large network may perform worse than an ensemble of several medium-size networks, known as a memory split.
Paper models power laws in sparse graphs using completely random measures.
problem Modeling power laws in sparse network data.
method General framework using completely random measures, focusing on sparsity and various types of power laws.
result The model exhibits desirable asymptotic power-law behavior in simulations.
Superposition accelerates training to a universal power-law exponent.
problem Training dynamics in neural networks.
method Teacher-student framework and analytic theory.
result Superposition leads to a universal power-law exponent of ~1, independent of data and channel statistics.
A new model corrects SBM's bias for power-law degree networks.
problem SBM's incapability to handle power-law degree distributions.
method Introducing degree decay variables to encode varying degree distributions.
result PLD-SBM approximately preserves the scale-free feature in real networks and corrects SBM's bias.
This work analyzes neural scaling laws using power-law data spectra and derives analytical expressions for generalization error.
problem Understanding how neural network performance scales with key factors like data size and model complexity.
method Statistical mechanics techniques applied to one-pass stochastic gradient descent in a student-teacher framework.
result Derivation of analytical expressions for generalization error under power-law data spectra and identification of conditions for power-law scaling.
We provide an empirical investigation aimed at uncovering the statistical properties of intricate stock trading networks based on the order flow data of a highly liquid stock (Shenzhen Development Bank) listed on Shenzhen Stock Exchange during the whole year of 2003. By reconstructing the limit order book, we can extra…
New models generate power law exponents from 0.5 to 2.5, enabling inference.
problem Inference difficulty in models generating power laws with exponents > 2.
method Design and implement inference algorithms for a new class of models.
result Inference possible for models generating power laws with exponents from 0.5 to 2.5.
Power-law spectrum of random feature model is preserved in neural networks.
problem Preserving power-law spectrum in neural networks through random feature model.
method Characterized eigenvalues of population random-feature covariance using dyadic head-tail decomposition and Wick chaos expansions.
result Power-law exponent α is inherited from input covariance, modified by a logarithmic correction. New model captures power-law networks with block structure.
problem Power-law networks and block structure in network data.
method Introduced a new notion of exchangeability and derived a simple likelihood expression.
result Model can infer block structure and edge inhomogeneity.
Develops a simple model to understand learning curves for arbitrary power laws.
problem Lack of theoretical understanding of scaling laws in machine learning.
method Analyzes a toy model to determine if learning curves are universal or depend on data distribution.
result Determines that learning curves can exhibit n−β for arbitrary power β>0. Bayesian inference models power-law graphs with efficient algorithms.
problem Modeling networks with heavy-tailed degree distributions.
method Constructs graphs using BFRY random variables and applies variational Bayesian inference.
result Automatic selection of power law behavior from data.
Develops a new classical network ensemble framework.
problem Lack of information-theoretic frameworks for complex networks.
method Optimal trade-off between compressed representation and actual network ensemble.
result Power-law degree distribution is optimal for networks with only expected degrees as constraints.
Many systems of different nature exhibit scale free behaviors. Economic systems with power law distribution in the wealth is one of the examples. To better understand the working behind the complexity, we undertook an empirical study measuring the interactions between market participants. A Web server was setup to admi…
New findings show neural network training loss follows a power law over time.
problem Understanding the optimization process of neural networks during training.
method Spectral analysis of the integral operator representing the linearized evolution of a large network.
result The loss function in neural network training follows a power law behavior, L(t)∼t−ξ, with exponent ξ determined by network parameters and data characteristics. We generalize the scale-free network model of Barabàsi and Albert [Science 286, 509 (1999)] by proposing a class of stochastic models for scale-free interdependent networks in which interdependent nodes are not randomly connected but rather are connected via preferential attachment (PA). Each network grows through the …
The paper proves geometric and spectral alignment for deep neural networks.
problem Understanding the singular spectra of deep neural network layers.
method Proves deterministic quotient-geometric estimates for singular spectra of Frobenius-normalized layer factors.
result Exact power-law spectra form a trace-normalized Cartan orbit under Frobenius normalization.
New statistical models capture double power-law behavior in data.
problem Capturing two-regime power-law behavior in datasets.
method Introducing completely random measures with double power-law behavior.
result Proposed models provide a better fit than Pitman-Yor process.
This paper reformulates systemic risk measures and finds new properties and estimators.
problem Understanding and measuring systemic risk in financial networks.
method Representation of systemic risk measures in terms of univariate risk measures and quantiles determined by copulas. Empirical properties and estimators derived.
result MES is not suitable for measuring extreme risks. ES-based measures are more sensitive to power-law tails and large losses.
We study a model of wealth dynamics [Bouchaud and Mézard 2000, \emph{Physica A} \textbf{282}, 536] which mimics transactions among economic agents. The outcomes of the model are shown to depend strongly on the topological properties of the underlying transaction network. The extreme cases of a fully connected and a ful…
Unified model explains international trade patterns using reinforced urns.
problem Understanding the complex patterns of international trade networks.
method A unified modelling framework using reinforced urns and the Reinforced Urn Process.
result The model predicts power law behavior and accounts for various network properties.
Large models follow power laws in performance with dataset size or parameters.
problem Understanding neural scaling laws in large language models.
method Joint generative data model and random feature model.
result Modeling and solving the dual limit reveals insights into scaling laws.
We conduct a market experiment with human agents in order to explore the structure of transaction networks and to study the dynamics of wealth accumulation. The experiment is carried out on our platform for 97 days with 2,095 effective participants and 16,936 times of transactions. From these data, the hybrid distribut…
Defines complexity measure for neural networks and feature representations, revealing scaling patterns.
problem Understanding the nonlinearity and dimensionality of neural network computations and feature representations.
method Introduces complexity and effective dimension measures, investigates their dynamics during training, and analyzes their scaling properties.
result Power law scaling of complexity and effective dimension during training, revealing hidden structure of datasets.
Model analyzes how heterogeneity in bank and asset distributions affects financial contagion.
problem Effect of power-law distributions on financial contagion stability.
method Modeling financial contagion in a bipartite network with heterogeneous degrees and balance-sheet sizes.
result Power-law degree distributions in banks decrease system stability, while in assets increase it.
Adversarial examples arise from neural networks' uncertainty, leading to a power-law scaling in error.
problem Vulnerability of machine learning classifiers to adversarial examples.
method Analyzed uncertainty in neural network predictions and its relation to adversarial errors.
result Adversarial error scales as a power-law with perturbation size, independent of architecture, dataset, and training protocol.
We study the crash dynamics of the Warsaw Stock Exchange (WSE) by using the Minimal Spanning Tree (MST) networks. We find the transition of the complex network during its evolution from a (hierarchical) power law MST network, representing the stable state of WSE before the recent worldwide financial crash, to a superst…
New methods predict neural network quality without access to training data.
problem Predicting neural network quality without access to training or testing data.
method Meta-analysis of pretrained models using norm and power law based metrics.
result Power law based metrics can better distinguish well-trained from poorly-trained models.
We introduce a stochastic model to explain a double power-law distribution which exhibits two different Paretian behaviors in the upper and the lower tail and widely exists in social and economic systems. The model incorporates fitness consideration and noise fluctuation. We find that if the number of variables (e.g. t…
Recently, the visibility graph has been introduced as a novel view for analyzing time series, which maps it to a complex network. In this paper, we introduce new algorithm of visibility, "cross-visibility", which reveals the conjugation of two coupled time series. The correspondence between the two time series is mappe…
New SDE model from machine learning optimization with unique stationary distribution.
problem Stationary distribution of machine learning optimization models.
method Proved ergodicity and unique stationary distribution of power-law dynamic SDE.
result Power-law dynamic has a unique stationary distribution and is ergodic.
New causal models for growing networks avoid node deletion constraints.
problem Statistical models based on node exchangeability are not suitable for growing networks.
method Enumerated and partitioned causal directed acyclic graph (DAG) models over pairs of nodes.
result Simple model exhibits flexible power-law degree distributions and emergent phase transitions.
The study analyzes deep linear networks from random initialization, capturing dynamics and hyperparameter effects.
problem Understanding training dynamics in deep linear networks from random initialization.
method Theoretical analysis of gradient descent dynamics in deep linear networks with random initialization and large data.
result Captures the 'wider is better' effect and hyperparameter transfer effects, contrasting with neural-tangent parameterization.
Power laws detected in financial data, modeled with random multipliers.
problem Detecting power laws in financial data.
method Investigated data from financial instruments, proposed a model based on sums of Maxwell-Boltzmann distributions with random multipliers.
result Detected power laws with various exponents in financial data, proposed a universal model.
Neural networks learn simpler features first, then more complex ones; Fourier analysis reveals this pattern.
problem Understanding the learning dynamics of neural networks, especially with natural image data.
method Fourier analysis of translation-invariant and power-law spectra to study feature learning.
result Simple neural networks first rely on amplitude information, then phase information, and power-law spectra can accelerate learning phase information.
Manipulation is an important issue for both developed and emerging stock markets. For the study of manipulation, it is critical to analyze investor behavior in the stock market. In this paper, an analysis of the full transaction records of over a hundred stocks in a one-year period is conducted. For each stock, a tradi…
Study on KRR with power-law data, showing better sample complexity.
problem High-dimensional kernel ridge regression with anisotropic power-law covariance.
method Explicit characterization of kernel spectrum and asymptotic analysis of excess risk.
result Sample complexity is governed by effective dimension, not ambient dimension.
We use data on wealth of the richest persons taken from the "rich lists" provided by business magazines like Forbes to verify if upper tails of wealth distributions follow, as often claimed, a power-law behaviour. The data sets used cover the world's richest persons over 1996-2012, the richest Americans over 1988-2012,…
We use daily data on bilateral interbank exposures and monthly bank balance sheets to study network characteristics of the Russian interbank market over Aug 1998 - Oct 2004. Specifically, we examine the distributions of (un)directed (un)weighted degree, nodal attributes (bank assets, capital and capital-to-assets ratio…
Model market shows self-organized behavior with price adjustments.
problem Understanding collective behavior in market economics.
method Simple model of market economics with extremal dynamics.
result Market self-organizes through price adjustments in critical states.
A hierarchical model shows how scaling laws emerge from sequential feature recovery.
problem Emergence of scaling laws from feature learning in multi-layer networks.
method Layer-wise spectral algorithm adapted to compositional structure, sequential feature detection.
result Sequential detection of latent features, leading to explicit power-law decay of prediction error.
It is generally recognized that economical systems, and more in general complex systems, are characterized by power law distributions. Sometime, these distributions show a changing of the slope in the tail so that, more appropriately, they show a multi-power law behavior. We present a method to derive analytically a tw…
Study non-integer power-law potentials for Schrödinger operators using Lie-Rinehart algebras.
problem Analyzing Schrödinger operators with non-integer power-law potentials.
method Using Lie-Rinehart algebras and microlocal analysis.
result Microlocal analysis can be applied to Schrödinger operators with non-integer power-law potentials.
The role of kernels is central to machine learning. Motivated by the importance of power-law distributions in statistical modeling, in this paper, we propose the notion of power-law kernels to investigate power-laws in learning problem. We propose two power-law kernels by generalizing Gaussian and Laplacian kernels. Th…
I consider the problem of the optimal limit order price of a financial asset in the framework of the maximization of the utility function of the investor. The analytical solution of the problem gives insight on the origin of the recently empirically observed power law distribution of limit order prices. In the framewor…
We introduce the stochastic multiplicative point process modelling trading activity of financial markets. Such a model system exhibits power-law spectral density S(f) ~ 1/f**beta, scaled as power of frequency for various values of beta between 0.5 and 2. Furthermore, we analyze the relation between the power-law autoco…
The paper introduces new methods to measure cross-correlations between time series using power-law coherency.
problem Studying power-law cross-correlations between time series.
method Three estimators of the power-law coherency parameter H_ρ based on DCCA, DMCA, and HXA.
result DMCA-based method is the safest choice, HXA method is reasonable for long series, and DCCA-based method has unfavorable properties.