A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Study on massless Vlasov equation on Reissner-Nordström spacetimes, showing decay rates and non-decay phenomena.
problem Analyzing decay and non-decay rates of solutions to the massless Vlasov equation on Reissner-Nordström spacetimes.
method Quantitative analysis of geodesic flow and comparison to wave equation instability results.
result Exponential decay rates in subextremal cases and polynomial rates in extremal cases, with non-decay of transversal derivatives in extremal cases.
Minimax optimal convergence rates for classes of stochastic convex optimization problems are well characterized, where the majority of results utilize iterate averaged stochastic gradient descent (SGD) with polynomially decaying step sizes. In contrast, SGD's final iterate behavior has received much less attention desp…
We get decay rate of higher derivatives of nonlinear massless Dirac equations with a kind of "good" spin null form. The method we rely on is similar to that of Li and Zang. However, they only give the decay rate of solution itself to nonlinear massless Dirac system.
The paper studies harmonic map heat flow stability and decay rates.
problem Analyzing stability and decay rates of harmonic map heat flow solutions.
method Use of homogeneous Besov space B˙p,∞pd(Rd) for small initial data and self-similar decay assumption.
result Decay rates for solutions of the harmonic map flow of the form ∥ablau(t)∥L∞(Rd)≤Ct−21 and self-similar decay under stronger initial conditions.
Learning rate decay (lrDecay) is a \emph{de facto} technique for training modern neural networks. It starts with a large learning rate and then decays it multiple times. It is empirically observed to help both optimization and generalization. Common beliefs in how lrDecay works come from the optimization analysis of (S…
This paper concerns integral varifolds of arbitrary dimension in an open subset of Euclidean space satisfying integrability conditions on their first variation. Firstly, the study of pointwise power decay rates almost everywhere of the quadratic tilt-excess is completed by establishing the precise decay rate for two-di…
We consider any pseudo holomorphic integral 2-cycle in an arbitrary almost complex manifold and perform a blow up analysis at an arbitrary point. Building upon a pseudo algebraic blow up (previously introduced by the author) we prove a geometric rate of decay for the mass ratio towards the limiting density, with an exp…
Adaptive weights improve physics-informed neural networks and deep operator networks.
problem Training physics-informed neural networks and deep operator networks can be challenging, leading to unsatisfactory accuracy and efficiency.
method Proposes a pointwise adaptive weighting method that balances the residual decay rate across different training points.
result Our proposed approach of balanced residual decay rates offers advantages including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.
In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the d…
Stochastic (sub)gradient methods require step size schedule tuning to perform well in practice. Classical tuning strategies decay the step size polynomially and lead to optimal sublinear rates on (strongly) convex problems. An alternative schedule, popular in nonconvex optimization, is called \emph{geometric step decay…
We introduce a new weight-decay scaling rule to maintain sublayer gains across different widths in modern scale-invariant architectures.
problem In modern scale-invariant architectures, training quickly enters a steady state where normalization layers create backward scale sensitivity, degrading learning-rate transfer.
method We introduce a weight-decay scaling rule for AdamW that preserves sublayer gain across widths by equalizing the effective learning rate.
result Our empirical weight-decay scaling rule λ2∝d approximately keeps sublayer gains width invariant, enabling zero-shot transfer of learning rate and weight decay.
Momentum is a widely used technique for gradient-based optimizers in deep learning. In this paper, we propose a decaying momentum (\textsc{Demon}) rule. We conduct the first large-scale empirical analysis of momentum decay methods for modern neural network optimization, in addition to the most popular learning rate dec…
Regularization in the optimization of deep neural networks is often critical to avoid undesirable over-fitting leading to better generalization of model. One of the most popular regularization algorithms is to impose L-2 penalty on the model parameters resulting in the decay of parameters, called weight-decay, and the …
Study reveals dynamics of neural networks with normalization, weight decay, and SGD.
problem Understanding the equilibrium condition in Spherical Motion Dynamics (SMD).
method Investigates SMD by exploring the cause of equilibrium condition, introducing assumptions, proposing angular update, and verifying theoretical results.
result Proves weight norm and angular update can converge at linear rate under given assumptions.
We extend Eardley and Moncrief's L∞ estimates for the conformally invariant Yang-Mills-Higgs equations to the Einstein cylinder. Our method is to first work on Minkowski space and localise their estimates, and then carry them to the Einstein cylinder by a conformal transformation. By patching local estimates to…
We prove that the Ricci flow that contracts a hyperbolic cusp has curvature decay like one over time squared. In order to do this, we prove a new Li-Yau type differential Harnack inequality for Ricci flow on surfaces.
The paper presents a multi-power law for predicting loss curves across different learning rate schedules.
problem Understanding and optimizing the relationship between model performance and hyperparameters, especially learning rates.
method Proposes a multi-power law that combines power laws based on the sum of learning rates and additional laws for loss reduction due to decay.
result The multi-power law accurately predicts loss curves for unseen learning rate schedules and finds a schedule that outperforms cosine learning rate.
One knows that the large time heat decay exponent on a nilpotent group is given by half the growing rate of the volume of its large balls. This work deals with the similar problem of trying to interpret geometrically the heat decay on (one) forms. We will show how it is (partially) related to the depth of the relations…
The study introduces anytime learning schedules for large language models without fixed horizons.
problem Training large language models without knowing the total training horizon.
method Theoretical analysis and weight averaging to create anytime learning schedules.
result Theoretical and empirical evidence shows that weight averaging with simple step sizes can achieve comparable final loss to well-tuned cosine schedules.
We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kern…
Approximations to utility indifference prices are provided for a contingent claim in the large position size limit. Results are valid for general utility functions on the real line and semi-martingale models. It is shown that as the position size approaches infinity, the utility function's decay rate for large negative…
In this work, we consider solutions of the Maxwell equations on the Schwarzschild-de Sitter family of black hole spacetimes. We prove that, in the static region bounded by black hole and cosmological horizons, solutions of the Maxwell equations decay to stationary Coulomb solutions at a super-polynomial rate, with deca…
We consider a non-trapping n-dimensional Lorentzian manifold endowed with an end structure modeled on the radial compactification of Minkowski space. We find a full asymptotic expansion for tempered forward solutions of the wave equation in all asymptotic regimes. The rates of decay seen in the asymptotic expansion a…
Weight decay is one of the standard tricks in the neural network toolbox, but the reasons for its regularization effect are poorly understood, and recent results have cast doubt on the traditional interpretation in terms of L2 regularization. Literal weight decay has been shown to outperform L2 regularization for…