arXiv research

Geometric step decay schedules improve stochastic algorithms' convergence on sharp nonconvex problems.

Improved learning rate schedule for least squares regression.

Step decay schedules improve convergence in non-convex optimization.

Study of 4D Ricci solitons with symmetry, finding precise geometric asymptotics.

Last SGD iterate bounds for overparameterized linear regression.

Improved online prediction with guaranteed coverage.

A random walk $w_n$ on a separable, geodesic hyperbolic metric space $X$ converges to the boundary $\partial X$ with probability one when the step distribution supports two independent loxodromics. In particular, the random walk makes positive linear progress. Progress is known to be linear with exponential decay when …

We consider the least-squares regression problem and provide a detailed asymptotic analysis of the performance of averaged constant-step-size stochastic gradient descent (a.k.a. least-mean-squares). In the strongly-convex case, we provide an asymptotic expansion up to explicit exponentially decaying terms. Our analysis…

This is the second in a series of three papers in which we initiate the study of very rough solutions to the initial value problem for the Einstein vacuum equations expressed relative to wave coordinates. By very rough we mean solutions which cannot be constructed by the classical techniques of energy estimates and Sob…

Paper develops an online learning algorithm for functional data models.

Demon improves neural network training with a decaying momentum approach.

In this paper we investigate the life-span of classical solutions to the hyperbolic geometric flow in two space variables with slow decay initial data. By establishing some new estimates on the solutions of linear wave equations in two space variables, we give a lower bound of the life-span of classical solutions to th…

Adaptive LR improves neural network Lipschitz regularity without slowing convergence.

Paper tackles dynamic pricing in a geometrically decaying environment, achieving better occupancy with lower rates.

New method finds precise late-time behavior of wave equations.

Community detection in hypergraphs is explored. Under a generative hypergraph model called "d-wise hypergraph stochastic block model" (d-hSBM) which naturally extends the Stochastic Block Model from graphs to d-uniform hypergraphs, the asymptotic minimax mismatch ratio is characterized. For proving the achievability, w…

We consider any pseudo holomorphic integral 2-cycle in an arbitrary almost complex manifold and perform a blow up analysis at an arbitrary point. Building upon a pseudo algebraic blow up (previously introduced by the author) we prove a geometric rate of decay for the mass ratio towards the limiting density, with an exp…

One knows that the large time heat decay exponent on a nilpotent group is given by half the growing rate of the volume of its large balls. This work deals with the similar problem of trying to interpret geometrically the heat decay on (one) forms. We will show how it is (partially) related to the depth of the relations…

New inequality shows energy growth and decay in geometric problems.

New insights into how to inspect and learn from multi-stage processes and AI reasoning.

Study finds solutions to inequality decay to zero on warped cylinders.

We present the results of computer experiments suggesting that the probability that a random multiword in a free group is virtually geometric decays to zero exponentially quickly in the length of the multiword. We then prove this fact.

We present an explicit example of a fast decaying solution to the modified Novikov--Veselov equation with a one-point singularity in the space-time. It is constructed by using the geometrical interpretation of the Moutard transformation of solutions to this equation and the Enneper minimal surface.

Gradient descent converges linearly in finite-width networks with positive NTK and compatible conditions.

Study constructs scattering theory for massless Dirac field on Kerr spacetime.

New theory sharpens Q-learning with LDTZ rate, proving it's best of both worlds.

Survey on stability of Minkowski spacetime in relativity.

Regularizers change the geometric properties of loss functions in neural networks.

Paper develops a new geometric framework for Kerr stability.

SignSGD outperforms SGD in linear regression with optimal scaling laws under PLRF model.

Study geometric vector fields on 5D 2-step nilmanifolds.

The paper studies Teichmüller TQFT for hyperbolic knots, proving exponential decay of partition functions.

The energy in a square membrane $Ω$ subject to constant viscous damping on a subset $ω\subset Ω$ decays exponentially in time as soon as $ω$ satisfies a geometrical condition known as the "Bardos-Lebeau-Rauch" condition. The rate $τ(ω)$ of this decay satisfies $τ(ω)= 2 \min(-μ(ω), g(ω))$ (see Lebeau [Math. Phys. Stud. …

Study geometric step options with jumps, deriving pricing equations and characterizations.

Model proposes neural network for continuous time dynamics with inductive biases.

Improved analysis for fair federated learning reduces dependence on noise floor.

We introduce a novel algorithm that computes the $k$-sparse principal component of a positive semidefinite matrix $A$. Our algorithm is combinatorial and operates by examining a discrete set of special vectors lying in a low-dimensional eigen-subspace of $A$. We obtain provable approximation guarantees that depend on t…

New method uses entropy dissipation to prove isoperimetric inequalities.

Geometric focusing affects dispersive estimates for Schrödinger and wave equations.

Stagewise training strategy is widely used for learning neural networks, which runs a stochastic algorithm (e.g., SGD) starting with a relatively large step size (aka learning rate) and geometrically decreasing the step size after a number of iterations. It has been observed that the stagewise SGD has much faster conve…

The paper analyzes Teukolsky equations on Kerr backgrounds, proving boundedness and decay of solutions.

This paper is motivated by the non-linear stability problem for the expanding region of Kerr de Sitter cosmologies in the context of Einstein's equations with positive cosmological constant. We show that under dynamically realistic assumptions the conformal Weyl curvature of the spacetime decays towards future null inf…

Temporal Difference Learning analysis under non-i.i.d. data and nonlinear approximation.

In this paper, we study the online learning algorithm without explicit regularization terms. This algorithm is essentially a stochastic gradient descent scheme in a reproducing kernel Hilbert space (RKHS). The polynomially decaying step size in each iteration can play a role of regularization to ensure the generalizati…

AdamNX improves Adam's stability by adjusting its learning rate.

WSD schedule improves model training efficiency by adapting learning rates dynamically.

Gradient descent outperforms ridge regression under certain covariance matrix decay conditions.

Optimal learning rate schedules derived for various tasks.