AdamZ optimiser improves neural network training efficiency.
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Trend · papers per month
Gradient descent stagnates in low-precision, but unbiased rounding schemes improve convergence.
The aim of this paper is to compare statistical properties of stock price indices in periods of booms with those in periods of stagnations. We use the daily data of the four stock price indices in the major stock markets in the world: (i) the Nikkei 225 index (Nikkei 225) from January 4, 1975 to August 18, 2004, of (ii…
Data describing historical growth of income per capita [Gross Domestic Product per capita (GDP/cap)] for the world economic growth and for the growth in Western Europe, Eastern Europe, Asia, former USSR, Africa and Latin America are analysed. They follow closely the linearly-modulated hyperbolic distributions represent…
Historical economic growth in Asia (excluding Japan) is analysed. It is shown that Unified Growth Theory is contradicted by the data, which were used (but not analysed) during the formulation of this theory. Unified Growth Theory does not explain the mechanism of economic growth. It explains the mechanism of Malthusian…
New study shows deep networks generalize well due to loss surface geometry.
Historical economic growth in countries of the former USSR is analysed. It is shown that Unified Growth Theory is contradicted by the data, which were used, but not analysed, during the formulation of this theory. Unified Growth Theory does not explain the mechanism of economic growth. It explains the mechanism of Malt…
Data describing historical economic growth are analysed. They demonstrate convincingly that the takeoffs from stagnation to growth, claimed in the Unified Growth Theory, never happened. This theory is again contradicted by data, which were used, but never properly analysed, during its formulation. The absence of the cl…
This paper designs sensor arrays for estimating unsteady flows efficiently.
Data describing historical economic growth are analysed. Included in the analysis is the world and regional economic growth. The analysis demonstrates that historical economic growth had a natural tendency to follow hyperbolic distributions. Parameters describing hyperbolic distributions have been determined. A search …
Galor discovered many mysteries of the growth process. He lists them in his Unified Growth Theory and wonders how they can be explained. Close inspection of his mysteries reveals that they are of his own creation. They do not exist. He created them by his habitually distorted presentation of data. One of his self-creat…
A simple but useful method of reciprocal values is introduced, explained and illustrated. This method simplifies the analysis of hyperbolic distributions, which are causing serious problems in the demographic and economic research. It allows for a unique identification of hyperbolic distributions and for unravelling co…
Proposes r2SGLD for efficient constrained exploration in non-convex learning.
Machine learning improves RNA secondary structure prediction.
New sampler tackles complex discrete energy landscapes efficiently.
Gradient descent with biased rounding errors converges faster under certain conditions.
Fairness in machine learning has predominantly been studied in static classification settings without concern for how decisions change the underlying population over time. Conventional wisdom suggests that fairness criteria promote the long-term well-being of those groups they aim to protect. We study how static fairne…
Quantum control is valuable for various quantum technologies such as high-fidelity gates for universal quantum computing, adaptive quantum-enhanced metrology, and ultra-cold atom manipulation. Although supervised machine learning and reinforcement learning are widely used for optimizing control parameters in classical …
Study compares altcoins to Bitcoin, analyzing their features and market performance.
While first-order optimization methods such as stochastic gradient descent (SGD) are popular in machine learning (ML), they come with well-known deficiencies, including relatively-slow convergence, sensitivity to the settings of hyper-parameters such as learning rate, stagnation at high training errors, and difficulty …
Many optimization algorithms converge to stationary points. When the underlying problem is nonconvex, they may get trapped at local minimizers and occasionally stagnate near saddle points. We propose the Run-and-Inspect Method, which adds an "inspect" phase to existing algorithms that helps escape from non-global stati…
Real-world optimization problems often have expensive objective functions in terms of cost and time. It is desirable to find near-optimal solutions with very few function evaluations. Surrogate-assisted optimizers tend to reduce the required number of function evaluations by replacing the real function with an efficien…
Expressivity is one of the most significant issues in assessing neural networks. In this paper, we provide a quantitative analysis of the expressivity for the deep neural network (DNN) from its dynamic model, where the Hilbert space is employed to analyze the convergence and criticality. We study the feature mapping of…
The so-called great divergence in the income per capita is described in the Unified Growth Theory as the mind-boggling and unresolved mystery about the growth process. This mystery has now been solved: the great divergence never happened. It was created by the manipulation of data. Economic growth in various regions is…
As the performance of computer systems stagnates due to the end of Moore's Law, there is a need for new models that can understand and optimize the execution of general purpose code. While there is a growing body of work on using Graph Neural Networks (GNNs) to learn representations of source code, these representation…
Historical economic growth in Latin America is analysed using the data of Maddison. Unified Growth Theory is found to be contradicted by these data in the same way as it is contradicted by the economic growth in Africa, Asia, former USSR, Western Europe, Eastern Europe and by the world economic growth. Paradoxically, U…
This paper improves bandwidth selectors for SPBNs to enhance their performance.
Dual labor market model explains low inflation despite low unemployment.
New findings show Bregman proximal algorithms can get stuck near non-stationary points.
VINNAS uses variational inference to avoid mode collapse in neural architecture search.
Optimizes solving complex min-max problems with stochastic and nonconvex elements.
The paper analyzes RLVR's training dynamics, proving convergence depends on aligning update direction with Gradient Gap.
Kolmogorov-Arnold Networks promise scalable performance in high dimensions.
SAGE enhances reinforcement learning by injecting hints to prevent model stagnation.
PRUDEX-Compass evaluates FinRL methods on 6 axes for financial market investments.
Deep neural networks are usually trained with stochastic gradient descent (SGD), which minimizes objective function using very rough approximations of gradient, only averaging to the real gradient. Standard approaches like momentum or ADAM only consider a single direction, and do not try to model distance from extremum…
Satellite images predict U.S. county mortality rates.
Neural networks plateau during training, identified and quantified.
Signed compression progress on a sealed audit is goodhart-resistant.
Deep RL evaluation underestimates uncertainty, leading to misleading conclusions.