Machine learning finds new natural laws from noisy data.
problem Discovery of natural laws relies on human inspiration.
method Recursive-LASSO-based symbolic regression (RLS) method.
result Data-driven formulation of natural laws from noisy data.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “natural laws”
Study reveals neural scaling laws in random graphs and natural language models.
problem Understanding the origin of neural scaling laws in complex systems.
method Examined scaling laws in transformers trained on random walks and simplified natural language models.
result Neural scaling laws emerge in the absence of power law structure in data correlations.
When a gauge-natural invariant variational principle is assigned, to determine {\em canonical} covariant conservation laws, the vertical part of gauge-natural lifts of infinitesimal principal automorphisms -- defining infinitesimal variations of sections of gauge-natural bundles -- must satisfy generalized Jacobi equat…
The Gestalt laws of perceptual organization, which describe how visual elements in an image are grouped and interpreted, have traditionally been thought of as innate despite their ecological validity. We use deep-learning methods to investigate whether natural scene statistics might be sufficient to derive the Gestalt …
Theory predicts neural scaling exponents from language statistics.
problem No existing theory could quantitatively predict neural scaling exponents.
method Isolated two key statistical properties of language.
result Derives a simple formula predicting neural scaling exponents.
New framework reveals thermodynamic principles for LLM training.
problem Understanding the training dynamics of large language models.
method Introducing Neural Thermodynamic Laws (NTL) under river-valley loss landscape assumptions.
result Key thermodynamic quantities and principles naturally emerge in LLM training.
On the origin of power law tails in price fluctuationscond-mat.stat-mech
In a recent Nature paper, Gabaix et al. \cite{Gabaix03} presented a theory to explain the power law tail of price fluctuations. The main points of their theory are that volume fluctuations, which have a power law tail with exponent roughly -1.5, are modulated by the average market impact function, which describes the r…
The conservation laws of the third order quasilinear scalar evolution equations are considered via differential system and characteristic cohomology. We find a subspace of 2 forms in the infinite prolonged space in which every conservation law has a unique representative. The structure of this subspace naturally gives …
Counterexamples show failure of uniform laws of large numbers for subdifferentials.
problem Failure of uniform laws of large numbers for subdifferentials under natural assumptions.
method Univariate and bivariate random Lipschitz and convex functions with smooth pieces.
result Counterexamples demonstrate failure of uniform laws of large numbers for subdifferentials.
In this paper we will present Lagrangian and Hamiltonian -symplectic formalisms, we will recall the notions of symmetry and conservation law and we will define the notion of pseudosymmetry as a natural extension of symmetry. Using symmetries and pseudosymmetries, without the help of a Noether type theorem, we will o…
The paper studies symmetries and conservation laws of non-diagonalisable hydrodynamic systems.
problem Integrating non-diagonalisable hydrodynamic systems of partial differential equations.
method Analysis of gl-regular Nijenhuis operators, splitting Theorem for symmetries and conservation laws, relationship between symmetries and conservation laws.
result The system of partial differential equations is integrable in quadratures.
Bayesian nonparametric CMS improves frequency estimation for power-law data.
problem Estimating frequencies of low-frequency tokens in power-law data streams.
method Developed a learning-augmented count-min sketch using a normalized inverse Gaussian process prior.
result The approach achieves remarkable performance in estimating low-frequency tokens.
Large models follow power laws in performance with dataset size or parameters.
problem Understanding neural scaling laws in large language models.
method Joint generative data model and random feature model.
result Modeling and solving the dual limit reveals insights into scaling laws.
Dynamic risk measures follow law invariance principles over time.
problem Tackles dynamic risk measurement principles.
method Shows equivalence between adapted law invariance and recursive one-step conditional-law representation for time-consistent risk measures.
result Identifies adapted law invariance as the dynamic counterpart of ordinary law invariance.
We develop a simple test for deviations from power law tails, which is based on the asymptotic properties of the empirical distribution function. We use this test to answer the question whether great natural disasters, financial crashes or electricity price spikes should be classified as dragon kings or 'only' as black…
This paper solves the problem of optimal dynamic consumption, investment, and healthcare spending with isoelastic utility, when natural mortality grows exponentially to reflect Gompertz' law and investment opportunities are constant. Healthcare slows the natural growth of mortality, indirectly increasing utility from c…
Analyzes how BPE tokenisation affects corpus statistics and model entropy in transformer models.
problem Understanding how natural language properties relate to tokenisation schemes in transformer models.
method Analyzes Shannon entropy of corpora under Zipfian distribution, investigates BPE transformations, trains language models, and uses attention diagnostics.
result Transformer models trained on BPE-tokenised corpora increasingly agree with Zipfian predictions as BPE depth increases, indicating reduced local token dependencies.
The power law has been observed in the degree distributions of many biological neural networks. Sparse deep neural networks, which learn an economical representation from the data, resemble biological neural networks in many ways. In this paper, we study if these artificial networks also exhibit properties of the power…
In the classical Lagrangian approach to conservation laws of gauge-natural field theories a suitable (vector) density is known to generate the so--called {\em conserved Noether currents}. It turns out that along any section of the relevant gauge--natural bundle this density is the divergence of a skew--symmetric (tenso…
Ubiquitous power law scaling in nonlinear self-excited Hawkes processescond-mat.stat-mech
New mechanism found for power laws including Zipf's law.
problem Understanding the ubiquity of power law distributions.
method Introduced nonlinear self-excited Hawkes processes with fast-accelerating intensities.
result Wide class of nonlinear Hawkes processes have power law intensity PDFs.
Law explains how deep networks separate data for classification.
problem Black-box nature of deep learning limits architecture design and interpretation.
method Studied how deep neural networks process data in intermediate layers.
result Law of geometric data separation emerges in various architectures and datasets.
Bayesian nonparametric approaches, in particular the Pitman-Yor process and the associated two-parameter Chinese Restaurant process, have been successfully used in applications where the data exhibit a power-law behavior. Examples include natural language processing, natural images or networks. There is also growing em…
We present a model for random simple graphs with a degree distribution that obeys a power law (i.e., is heavy-tailed). To attain this behavior, the edge probabilities in the graph are constructed from Bertoin-Fujita-Roynette-Yor (BFRY) random variables, which have been recently utilized in Bayesian statistics for the c…
This is a short commentary piece that discusses how the methods used in the natural sciences can apply to economics in general and financial markets specifically.
The purpose of this paper is to extend the Green-Naghdi-Rivlin balance of energy method to continua with microstructure. The key idea is to replace the group of Galilean transformations with the group of diffeomorphisms of the ambient space. A key advantage is that one obtains in a natural way all the needed balance la…
The study explains transformer scaling laws using statistical and approximation theories.
problem Understanding why transformer scaling laws exist for large models trained on low-dimensional data.
method Established statistical estimation and mathematical approximation theories for transformers on low-dimensional manifolds.
result Predicted a power law between generalization error and model and data sizes, with power depending on intrinsic data dimension.
Optimizer choice affects neural scaling laws, changing the exponent .
problem The exponent in neural scaling laws varies with the optimizer used.
method Controlled random-feature regression experiments with five optimizer variants and six spectral conditions.
result Preconditioned optimizers yield steeper scaling (larger ), with the -shift increasing across most of the tested spectral range.
Benford's law predicted digit distribution of aggregated income taxes: the surprising conformity of Italian cities and regionsphysics.soc-ph
The yearly aggregated tax income data of all, more than 8000, Italian municipalities are analyzed for a period of five years, from 2007 to 2011, to search for conformity or not with Benford's law, a counter-intuitive phenomenon observed in large tabulated data where the occurrence of numbers having smaller initial digi…
Researchers use quantum chaos and RMT to analyze turbulence, revealing unique scaling laws.
problem Understanding the statistical structure and scaling laws of turbulence.
method Applied tools from quantum chaos and Random Matrix Theory to analyze turbulence datasets.
result Turbulence Gram matrices exhibit power-law scalings distinct from classical chaos and random data.
This is a second paper in a series devoted to the minimal unitary representation of O(p,q). By explicit methods from conformal geometry of pseudo-Riemannian manifolds, we find the branching law corresponding to restricting the minimal unitary representation to natural symmetric subgroups. In the case of purely discrete…
The Strong, Weak and Benign Goodhart's law. An independence-free and paradigm-agnostic formalisationstat.ML
Study on Goodhart's law without independence assumptions, finds new patterns in optimisation.
problem Understanding when and how optimisation of a proxy metric leads to over-optimisation of the intended goal.
method Formalized Goodhart's law without independence and paradigm assumptions, studied different cases of goal and discrepancy tailness.
result Dependence between proxy metric and goal does not change Goodhart's effect for light-tailed cases, but over-optimisation occurs in heavy-tailed discrepancy cases.
A one-to-one correspondence is drawn between law invariant risk measures and divergences, which we define as functionals of pairs of probability measures on arbitrary standard Borel spaces satisfying a few natural properties. Divergences include many classical information divergence measures, such as relative entropy a…
Financial Market Dynamicscond-mat.stat-mech
Distributions derived from non-extensive Tsallis statistics are closely connected with dynamics described by a nonlinear Fokker-Planck equation. The combination shows promise in describing stochastic processes with power-law distributions and superdiffusive dynamics. We investigate intra-day price changes in the S&P500…
In this article we present a natural generalization of Newton's Second Law valid in field theory, i.e., when the parameterized curves are replaced by parameterized submanifolds of higher dimension. For it we introduce what we have called the geodesic -vector field, analogous to the ordinary geodesic field and which …
Law explains how LLMs learn to predict next tokens.
problem Understanding how LLMs process input data internally.
method Introduced a precise law governing token embeddings in LLMs.
result Each layer equally contributes to next-token prediction accuracy.
Study detects endogenous bubbles in meme stocks using CI.
problem Detecting endogenous bubbles in meme stocks.
method Used Log-Periodic Power Law (LPPL) Confidence Indicator (CI).
result CI detected numerous bubbles in meme stocks but struggled with predicting exogenous rallies.
The nature of fluctuations in the Indian financial market is analyzed in this paper. We have looked at the price returns of individual stocks, with tick-by-tick data from the National Stock Exchange (NSE) and daily closing price data from both NSE and the Bombay Stock Exchange (BSE), the two largest exchanges in India.…
New model shows natural language exhibits phase transition similar to physics.
problem Understanding critical properties in natural language models.
method Created a context-sensitive random language model.
result Demonstrated a Berezinskii--Kosterlitz--Thouless phase transition.
Automates discovery of interpretable Lagrangians from data.
problem Lack of interpretable Lagrangian discovery methods.
method Data-driven machine learning algorithm to derive interpretable Lagrangians.
result Automated discovery of interpretable Lagrangians and conservation laws.
We consider the second variational derivative of a given gauge-natural invariant Lagrangian taken with respect to (prolongations of) vertical parts of gauge-natural lifts of infinitesimal principal automorphisms. By requiring such a second variational derivative to vanish, {\em via} the Second Noether Theorem we find t…
Mathematical framework for field theories on Finsler spacetimes.
problem Developing a consistent calculus for field theories on Finsler spacetimes.
method Constructing configuration bundles and applying coordinate-free calculus of variations.
result Averaged energy-momentum conservation law for Finsler field theories.
Taylor's law of temporal fluctuation scaling, variance mean, is ubiquitous in natural and social sciences. We report for the first time convincing evidence of a solid temporal fluctuation scaling law in stock illiquidity by investigating the mean-variance relationship of the high-frequency illiquidity o…
Copulas and time series with long-ranged dependencesphysics.data-an
We review ideas on temporal dependences and recurrences in discrete time series from several areas of natural and social sciences. We revisit existing studies and redefine the relevant observables in the language of copulas (joint laws of the ranks). We propose that copulas provide an appropriate mathematical framework…
Relevant Stylized Facts About Bitcoin: Fluctuations, First Return Probability, and Natural Phenomenaq-fin.ST
Bitcoin is a digital financial asset that is devoid of a central authority. This makes it distinct from traditional financial assets in a number of ways. For instance, the total number of tokens is limited and it has not explicit use value. Nonetheless, little is know whether it obeys the same stylized facts found in t…
Zipfian Whiteningcs.CL
PCA whitening weighted by Zipfian word frequencies improves task performance.
problem Skewed word embedding spaces in neural models.
method PCA whitening weighted by empirical word frequencies following Zipf's law.
result Significantly improves task performance, surpassing baselines.
A theory of exceptional extreme events, characterized by their abnormal sizes compared with the rest of the distribution, is presented. Such outliers, called "dragon-kings", have been reported in the distribution of financial drawdowns, city-size distributions (e.g., Paris in France and London in the UK), in material f…
We provide a variety of results for (quasi)convex, law-invariant functionals defined on a general Orlicz space, which extend well-known results in the setting of bounded random variables. First, we show that Delbaen's representation of convex functionals with the Fatou property, which fails in a general Orlicz space, c…
Tensor networks help learn complex physical laws from data.
problem Identifying non-linear dynamical laws from complex physical systems.
method Tensor network parameterizations and rank-adaptive optimization.
result Optimal tensor network models can be learned from data.