The paper investigates Goodhart's law and its impact on goal alignment.
problem The adverse effects of optimizing a measure when it diverges from the true goal.
method Formal analysis of Goodhart's law, focusing on the tail distribution of discrepancies.
result Goodhart's law depends on the tail distribution of discrepancies between the true goal and the optimized measure.
Study on Goodhart's law without independence assumptions, finds new patterns in optimisation.
problem Understanding when and how optimisation of a proxy metric leads to over-optimisation of the intended goal.
method Formalized Goodhart's law without independence and paradigm assumptions, studied different cases of goal and discrepancy tailness.
result Dependence between proxy metric and goal does not change Goodhart's effect for light-tailed cases, but over-optimisation occurs in heavy-tailed discrepancy cases.
There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an extent that further optimization is ineffective or harmful, and is sometimes termed Goodhart's Law. This class of failure is often poorly unders…
Study on RL on volatility surfaces, proving no free lunch for law-seeking methods.
problem Aligning RL agents with no-arbitrage laws in volatile markets.
method Built a law manifold, defined penalties, and used a Goodhart decomposition.
result No free lunch theorem: Law-seeking RL cannot outperform baselines.
Study scaling laws of reward model overoptimization in reinforcement learning.
problem Reward model overoptimization hinders true performance in reinforcement learning.
method Synthetic setup with fixed gold reward model; optimization using RL or best-of-n sampling; analysis of scaling laws. result Scaling laws of reward model overoptimization differ based on optimization method and scale smoothly with model parameters.
This review of the book "The Challenge of Financial Stability: A New Model and its Applications" by Goodhart C.A.E. and Tsomocos D.P. highlights the potential of the framework of strategic partial default of banks with credit chain on the interbank market for further theoretical and applied research on financial stabil…
Signed compression progress on a sealed audit is goodhart-resistant.
problem Intrinsic motivation for agents to improve their world models by compressing experience.
method Rewarding agents for the signed decrease of a fixed sealed-audit loss.
result Cumulative reward telescopes exactly to endpoint audit improvement, preventing infinite reward push while true audit performance stagnates.
This paper investigates teacher hacking during language model distillation and proposes methods to mitigate it.
problem Teacher hacking during language model distillation, leading to suboptimal performance.
method A controlled experimental setup involving an oracle LM, teacher LM, and student LM, using fixed offline or online data generation techniques.
result Data diversity is the key factor in preventing teacher hacking during distillation.
A new metric assesses latent variable models using data and model moments.
problem Difficulty in assessing the quality of unsupervised learning models.
method A moment-matching metric using matrix norms to compare data and model moments.
result The proposed metric is faster and has less variance than alternative methods.
When society maintains a competitive system to promote an abstract goal, competition by necessity relies on imperfect proxy measures. For instance profit is used to measure value to consumers, patient volumes to measure hospital performance, or the Journal Impact Factor to measure scientific value. Here we note that \t…
Higher conservative training increases reward-hacking in reasoning models.
problem Reward hacking during online adaptation in reasoning models.
method Conservative offline training with varying levels of conservatism (β) was applied to a Qwen3-14B policy, and online adaptation was measured against a reward ensemble.
result Higher conservatism (β) increases reward-hacking damage, measured by the Goodhart gap and AUGC.
This work explains scaling laws as redundancy laws in deep learning.
problem The mathematical origins of scaling laws in deep learning models remain unclear.
method Kernel regression and analysis of data covariance spectra.
result Scaling laws can be explained as redundancy laws, revealing the learning curve's slope depends on data redundancy.
A new scaling law predicts optimal batch size for training models.
problem Finding the optimal batch size for training models efficiently.
method Proposed a three-term scaling law that considers model size, training data, training steps, and batch size.
result The three-term law accurately recovers the optimal batch size and can be robustly fit with fewer training runs.
Space exploration technology advances exponentially, consistent with Moore's and Wright's laws.
problem Predicting the advancement of space exploration technology.
method Analysis of Moore's and Wright's laws applied to space exploration technology.
result Spacecraft technology advances exponentially, consistent with Moore's and Wright's laws.
This work analyzes neural scaling laws using power-law data spectra and derives analytical expressions for generalization error.
problem Understanding how neural network performance scales with key factors like data size and model complexity.
method Statistical mechanics techniques applied to one-pass stochastic gradient descent in a student-teacher framework.
result Derivation of analytical expressions for generalization error under power-law data spectra and identification of conditions for power-law scaling.
An Atlas model is a rank-based system of continuous semimartingales for which the steady-state values of the processes follow a power law, or Pareto distribution. For a power law, the log-log plot of these steady-state values versus rank is a straight line. Zipf's law is a power law for which the slope of this line is …
Large models follow power laws in performance with dataset size or parameters.
problem Understanding neural scaling laws in large language models.
method Joint generative data model and random feature model.
result Modeling and solving the dual limit reveals insights into scaling laws.
Conservation law for weakly harmonic mappings in high dimensions.
problem Conservation law for harmonic mappings in supercritical dimensions.
method Partial extension of Rivière's conservation law with Lorentz integrability condition.
result Conservation law for weakly harmonic mappings in supercritical dimensions.
Survey on conservation laws for geometric PDEs.
problem Modeling polyharmonic maps.
method Conservation law approach.
result Overview of conservation laws in geometric PDEs.
Dynamic risk measures follow law invariance principles over time.
problem Tackles dynamic risk measurement principles.
method Shows equivalence between adapted law invariance and recursive one-step conditional-law representation for time-consistent risk measures.
result Identifies adapted law invariance as the dynamic counterpart of ordinary law invariance.
Unified theory for neural scaling laws in hierarchically compositional data.
problem Understanding neural scaling laws in hierarchically compositional data.
method Probabilistic context-free grammars and power-law distributed production rules.
result Unified learning curve behavior for classification and next-token prediction tasks.
Defines formal vertex laws related to Lie conformal algebras.
problem No specific problem stated; focuses on definitions and proofs.
method Definitions and proofs of vertex/conformal versions of classical Lie theory results.
result Proves vertex/conformal versions of important Lie theory results.
We summarize a book under publication with his title written by the three present authors, on the theory of Zipf's law, and more generally of power laws, driven by the mechanism of proportional growth. The preprint is available upon request from the authors. For clarity, consistence of language and conciseness, we disc…
New concept of partial law invariance connects decision theory and financial risk management.
problem Connecting decision theory and financial risk management under uncertainty.
method Characterizing partially law-invariant coherent risk measures via a novel representation formula.
result Strong partial law invariance bridges the gap between existing risk measure representations.
This work extends the scaling law to multiple and kernel regression, challenging traditional machine learning principles.
problem Challenging traditional machine learning wisdom with scaling law in large practical models.
method Demonstrates the scaling law in multiple and kernel regression settings.
result The scaling law extends to multiple and kernel regression, providing deeper insights into LLMs.
Study on eigenvalue distribution of correlated time series deforming the semi-circle law.
problem Eigenvalue distribution of correlated time series differs from the semi-circle law.
method Analysis of Wigner random matrix with temporal correlation.
result Eigenvalue distribution converges to a deformed semi-circle law with longer tail and higher peak.
The article discusses conservation laws for polyharmonic maps and their applications.
problem Understanding conservation laws for polyharmonic maps.
method Recalling the stress-energy tensor and showing conservation laws with Killing vector fields.
result Conservation laws for polyharmonic maps and their applications.
Power laws detected in financial data, modeled with random multipliers.
problem Detecting power laws in financial data.
method Investigated data from financial instruments, proposed a model based on sums of Maxwell-Boltzmann distributions with random multipliers.
result Detected power laws with various exponents in financial data, proposed a universal model.
By using numerical simulation, we confirm that Takayasu--Sato--Takayasu (TST) model which leads Pareto's law satisfies the detailed balance under Gibrat's law. In the simulation, we take an exponential tent-shaped function as the growth rate distribution. We also numerically confirm the reflection law equivalent to the…
Empirical law predicts accuracy of Google Translate's translation chains.
problem Predicting accuracy in machine translation with multiple hops.
method Empirical testing of Google Translate's sequential translation.
result Accuracy decreases with the number of translating hops, following a power law.
Browsing and finding relevant information for Bangladeshi laws is a challenge faced by all law students and researchers in Bangladesh, and by citizens who want to learn about any legal procedure. Some law archives in Bangladesh are digitized, but lack proper tools to organize the data meaningfully. We present a text vi…
Machine learning finds new natural laws from noisy data.
problem Discovery of natural laws relies on human inspiration.
method Recursive-LASSO-based symbolic regression (RLS) method.
result Data-driven formulation of natural laws from noisy data.
I consider the existence and structure of conservation laws for the general class of evolutionary scalar second-order differential equations with parabolic symbol. First I calculate the linearized characteristic cohomology for such equations. This provides an auxiliary differential equation satisfied by the conservatio…
Employing profits data of Japanese companies in 2002 and 2003, we identify the non-Gibrat's law which holds in the middle profits region. From the law of detailed balance in all regions, Gibrat's law in the high region and the non-Gibrat's law in the middle region, we kinematically derive the profits distribution funct…
New SDE model from machine learning optimization with unique stationary distribution.
problem Stationary distribution of machine learning optimization models.
method Proved ergodicity and unique stationary distribution of power-law dynamic SDE.
result Power-law dynamic has a unique stationary distribution and is ergodic.
Study reveals neural scaling laws in random graphs and natural language models.
problem Understanding the origin of neural scaling laws in complex systems.
method Examined scaling laws in transformers trained on random walks and simplified natural language models.
result Neural scaling laws emerge in the absence of power law structure in data correlations.
New framework reveals thermodynamic principles for LLM training.
problem Understanding the training dynamics of large language models.
method Introducing Neural Thermodynamic Laws (NTL) under river-valley loss landscape assumptions.
result Key thermodynamic quantities and principles naturally emerge in LLM training.
Develops a simple model to understand learning curves for arbitrary power laws.
problem Lack of theoretical understanding of scaling laws in machine learning.
method Analyzes a toy model to determine if learning curves are universal or depend on data distribution.
result Determines that learning curves can exhibit n−β for arbitrary power β>0. Non-trivial conservation law found for a specific system.
problem Conservation law for a specific system with a vanishing characteristic.
method Analyzing overdetermined system with given characteristics.
result Non-trivial conservation law despite vanishing characteristic.
Study explains Zipf's law using geometric mechanisms from a finite alphabet.
problem Explains Zipf's law in language without relying on linguistic elements.
method Uses the Full Combinatorial Word Model (FCWM) to generate geometric distributions of word lengths.
result Supports predictions of power-law rank-frequency curves, matching various languages.
New scaling laws explain deep learning performance growth.
problem Understanding neural network performance growth.
method Analyzed entire training dynamics of various architectures.
result Identified two dynamical scaling laws.
Derives Weyl law for volume spectrum using parametric inequalities.
problem Deriving the Weyl law for the volume spectrum in compact Riemannian manifolds.
method Proves parametric generalizations of isoperimetric and coarea inequalities to derive the Weyl law.
result Derives the Weyl law for 1-cycles in 3-manifolds.
We study the relaxation dynamics of a financial market just after the occurrence of a crash by investigating the number of times the absolute value of an index return is exceeding a given threshold value. We show that the empirical observation of a power law evolution of the number of events exceeding the selected thre…
We show that there is a common mode of origin for the power laws observed in two different models: (i) the Pareto law for the distribution of money among the agents with random saving propensities in an ideal gas-like market model and (ii) the Gutenberg-Richter law for the distribution of overlaps in a fractal-overlap …
Superposition accelerates training to a universal power-law exponent.
problem Training dynamics in neural networks.
method Teacher-student framework and analytic theory.
result Superposition leads to a universal power-law exponent of ~1, independent of data and channel statistics.
The conservation laws of the third order quasilinear scalar evolution equations are considered via differential system and characteristic cohomology. We find a subspace of 2 forms in the infinite prolonged space in which every conservation law has a unique representative. The structure of this subspace naturally gives …
Formulates Hilbert reciprocity law on 3-manifolds.
problem Developing arithmetic topology on 3-manifolds.
method Formulated an analogue of Hilbert reciprocity law using intersection forms and Kummer extensions.
result Cyclic covers of links are analogues of Kummer extensions.
Novel groups exhibit contradictory behaviors with respect to Burnside laws.
problem Understanding probabilistic behaviors of groups under Burnside laws.
method Geometric analysis of relations, information-theoretic coding, combinatorial and probabilistic methods.
result Groups can satisfy Burnside laws with probability 1 for some generating sets and 0 for others.