In the presence of a layer of metaprobabilities (from uncertainty concerning the parameters), the asymptotic tail exponent corresponds to the lowest possible tail exponent regardless of its probability. The problem explains "Black Swan" effects, i.e., why measurements tend to chronically underestimate tail contribution…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
9 results for “powerlaw”
Predicts optimal training dataset sizes per class for machine learning models.
problem Optimizing training dataset sizes for class-specific machine learning models.
method Algorithm based on space-filling design of experiments, models like powerlaw curves and generalized linear models.
result The algorithm predicts optimal training dataset sizes per class for improved model performance.
We use a continuous-time random walk (CTRW) to model market fluctuation data from times when traders experience excessive losses or excessive profits. We analytically derive "superstatistics" that accurately model empirical market activity data (supplied by Bogachev, Ludescher, Tsallis, and Bunde)that exhibit transitio…
More features and data lead to better model performance in random feature regression.
problem Improving model performance in random feature regression.
method Theoretical analysis of random feature regression, demonstrating the benefits of overparameterization, overfitting, and more data.
result Infinite width RF architectures are preferable to those of any finite width, and training to near-zero training loss is obligatory for near-optimal performance.
Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Modelcond-mat.dis-nn
We find optimal learning rate schedules for a random feature model.
problem Choosing optimal learning rates for deep learning models.
method We analyze a powerlaw random feature model trained with SGD, considering optimal schedules as numerical and analytical problems.
result We discover two regimes: easy and hard phases, with different optimal learning rate schedules.
Ex ante forecast outcomes should be interpreted as counterfactuals (potential histories), with errors as the spread between outcomes. Reapplying measurements of uncertainty about the estimation errors of the estimation errors of an estimation leads to branching counterfactuals. Such recursions of epistemic uncertainty …
Heavy-tailed regularization improves deep neural network performance.
problem Improving generalization of deep neural networks.
method Introducing Heavy-Tailed Regularization, using differentiable penalty terms and Bayesian statistics.
result Heavy-tailed regularization outperforms conventional regularization techniques.
New insights into how depth and width affect in-context learning in deep models.
problem Understanding how various resources impact in-context learning in deep models.
method Analyzed linear regression in a deep linear self-attention model, varying resources like depth, width, context length, and training steps.
result Increasing depth improves in-context learning even at infinite context length, contrary to previous findings.
New study finds many neural networks are not benignly overfitting.
problem Understanding the behavior of overfitting in neural networks.
method Exploring kernel ridge regression and deep neural networks to identify overfitting behaviors.
result Many interpolating methods, including neural networks, exhibit tempered overfitting rather than benign or catastrophic.