arXiv research

Introduces gradient decay in Softmax for better generalization.

Analyzes dependencies in sequential datasets to improve deep neural architectures.

In this paper, we examine the dependence of standard gluing process for pseudoholomorphic curves under the change of the length $T$ of the neck-region with respect to the cylindrical metrics associated to the given analytic coordinates near the punctures in the setting of bordered open Riemann surface with boundary pun…

Three mechanisms of weight decay found for different optimizers and architectures.

Last SGD iterate bounds for overparameterized linear regression.

Weight decay stabilizes training dynamics by slowing progressive sharpening.

Study optimizes decay estimates for minimizing currents in submanifolds.

Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.

The paper proves scalar curvature decay for uniformly contractible manifolds with finite asymptotic dimension.

Step decay schedules improve convergence in non-convex optimization.

Paper tackles dynamic pricing in a geometrically decaying environment, achieving better occupancy with lower rates.

Optimal multistage method solves noisy minimax problems.

Under what conditions is an edge present in a social network at time t likely to decay or persist by some future time t + Delta(t)? Previous research addressing this issue suggests that the network range of the people involved in the edge, the extent to which the edge is embedded in a surrounding structure, and the age…

The paper explores how the probability of default estimation changes with temporal correlation decay.

Gradient descent outperforms ridge regression under certain covariance matrix decay conditions.

The paper improves transformer generalization bounds using rank-dependent covering number bounds.

Adaptive time decay functions improve financial product recommendation accuracy.

Paper addresses bias in kernel density estimation under minimal assumptions.

The statistical properties of the increments x(t+T) - x(t) of a financial time series depend on the time resolution T on which the increments are considered. A non-parametric approach is used to study the scale dependence of the empirical distribution of the price increments x(t+T) - x(t) of S&P Index futures, for time…

Paper questions RNN and LSTM's long-term memory and introduces a new definition.

WSD schedule improves model training efficiency by adapting learning rates dynamically.

In this paper we study the behaviour of the continuous spectrum of the Laplacian on a complete Riemannian manifold of bounded curvature under perturbations of the metric. The perturbations that we consider are such that its covariant derivatives up to some order decay with some rate in the geodesic distance from a fixe…

Improved analysis for fair federated learning reduces dependence on noise floor.

Regularizers change the geometric properties of loss functions in neural networks.

In recent work, we have proven uniform decay bounds for solutions of the wave equation $\Box_gφ=0$ on a Schwarzschild exterior, in particular, the uniform pointwise estimate $|φ|\le Cv_+^{-1}$, which holds throughout the domain of outer communications, where $v$ is an advanced Eddington-Finkelstein coordinate, $v_+=\ma…

Introduces recency bias to improve time-series forecasting.

Estimates kernel eigenvalues for compositional dot-product kernels.

Analyzes why neural networks generalize beyond training data.

New approach finds optimal hidden paths in large models, scaling to high dimensions.

A technique identifies memoryless algorithms approximating memory-dependent optimization methods.

We propose the time-dependent generalization of an `ordinary' autonomous human biomechanics, in which total mechanical + biochemical energy is not conserved. We introduce a general framework for time-dependent biomechanics in terms of jet manifolds derived from the extended musculo-skeletal configuration manifold. The …

New bandit algorithm for non-i.i.d. noise, improving standard rates.

A new model SMPS alleviates the exponential decay of correlations in MPS.

Adaptive LR improves neural network Lipschitz regularity without slowing convergence.

This paper deals with two related problems, namely distance-preserving binary embeddings and quantization for compressed sensing . First, we propose fast methods to replace points from a subset $\mathcal{X} \subset \mathbb{R}^n$, associated with the Euclidean metric, with points in the cube $\{\pm 1\}^m$ and we associa…

We consider the least-squares regression problem and provide a detailed asymptotic analysis of the performance of averaged constant-step-size stochastic gradient descent (a.k.a. least-mean-squares). In the strongly-convex case, we provide an asymptotic expansion up to explicit exponentially decaying terms. Our analysis…

We study the Cauchy problem for the wave equation on extreme Kerr backgrounds under axisymmetry. Specifically, we consider regular axisymmetric initial data prescribed on a Cauchy hypersurface S which connects the future event horizon with spacelike or null infinity, and we solve the linear wave equation on the domain …

We introduce a new weight-decay scaling rule to maintain sublayer gains across different widths in modern scale-invariant architectures.

The paper analyzes multivariate Hawkes processes and their induced population processes.

Using a proprietary dataset of meta-orders and prediction signals, and assuming a quasi-linear impact model, we deconvolve market impact from past correlated trades and a predictable return component to elicit the temporal dependence of the market impact of a single daily meta-order, over a ten day horizon in various e…

Study online learning in RKHS with dependent processes, focusing on \(β\)- and \(φ\)-mixing.

In this work, we first analyze the memory behavior in three recurrent neural networks (RNN) cells; namely, the simple RNN (SRN), the long short-term memory (LSTM) and the gated recurrent unit (GRU), where the memory is defined as a function that maps previous elements in a sequence to the current output. Our study show…

In this article we study the dependence degree of the traded volume of the Dow Jones 30 constituent equities by using a nonextensive generalised form of the Kullback-Leibler information measure. Our results show a slow decay of the dependence degree as a function of the lag. This feature is compatible with the existenc…

HyperAdam learns to optimize neural networks by combining Adam's updates with varying decay rates.

Stochastic momentum methods trade compute efficiency for serial runtime.

Paper develops an online learning algorithm for functional data models.

Weibull weight-scale parameter $λ$ evolves during AdamW training, with alignment, injection, and decay forces driving its growth and relaxation.

New height estimate for area minimizing currents, leading to unique tangent cones and decay properties.