Extends Adams' theorem to periodic cohomology.
problem Proving Adams' theorem for periodic cohomology.
method Adapting Adams' approach to periodic cohomology.
result Conjecture proven in a special case.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Adams theorem”
This paper establishes inequalities on quaternionic hyperbolic spaces and the Cayley hyperbolic plane.
problem Establishing higher order Poincaré-Sobolev and Hardy-Sobolev-Maz'ya inequalities on quaternionic hyperbolic spaces and the Cayley hyperbolic plane.
method Developing factorization theorems and introducing Geller's operators, combining with Helgason-Fourier analysis and kernel estimates.
result Established higher order Poincaré-Sobolev and Hardy-Sobolev-Maz'ya inequalities on quaternionic hyperbolic spaces and the Cayley hyperbolic plane.
Sharp inequalities on Siegel domains and complex hyperbolic spaces established.
problem Establishing inequalities on complex hyperbolic spaces and Siegel domains.
method Helgason-Fourier analysis, Kunze-Stein phenomenon, factorization theorem.
result Sharp Hardy-Adams and Adams type inequalities on Sobolev spaces of any positive fractional order on complex hyperbolic spaces.
Sharp inequalities for radial functions on hyperbolic spaces without boundary conditions.
problem Establishing inequalities for radial functions on hyperbolic spaces without zero boundary conditions.
method Novel approach considering both bounded and unbounded domains, focusing on weighted Sobolev and Adams-Trudinger-Moser embeddings.
result Theorems 1.2, 1.3, and 1.4 for weighted Sobolev embedding theorems, and Theorems 1.5 and 1.6 for Adams-Trudinger-Moser type embedding theorems.
Polyak-Ruppert CLT for SA-Adam with momentum and non-convergent adaptive preconditioning
problem Adaptive optimizers combining momentum and non-convergent preconditioning
method Proving positive drift stability and a non-autonomous Polyak-Ruppert CLT for SA-Adam
result The iterate-marginal covariance is exactly the plain stochastic gradient descent (SGD) sandwich
Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimizationstat.ML
Adam is a popular variant of stochastic gradient descent for finding a local minimizer of a function. In the constant stepsize regime, assuming that the objective function is differentiable and non-convex, we establish the convergence in the long run of the iterates to a stationary point under a stability condition. Th…
Proves a conjecture for a specific group using spectral sequences and homology.
problem Proves the Gromov-Lawson-Rosenberg Conjecture for the group Z/4xZ/4.
method Used the Adams spectral sequence and detection theorems to compute connective real k-homology.
result Determines differentials of the Adams spectral sequence and studies the cap structure of relevant sub-hopf algebras.
A curvature model (V,A) is a real vector space V which is equipped with a "curvature operator" A(x,y)z that A has the same symmetries as an affine curvature operator; A(x,y)z=-A(y,x)z and A(x,y)z+A(y,z)x+A(z,x)y=0. Such a model is called projective affine Osserman if the spectrum of the Jacobi operator J(y):x->A(x,y)y,…
Proves existence of manifolds with Kervaire invariant one in specific dimensions.
problem Existence of smooth framed manifolds with Kervaire invariant one.
method Adams spectral sequence and combination of theorems.
result Smooth framed manifolds with Kervaire invariant one exist in dimensions 2, 6, 14, 30, 62, and 126.
We describe pairs (p,n) such that n-dimensional affine space is fibered by pairwise skew p-dimensional affine subspaces. The problem is closely related with the theorem of Adams on vector fields on spheres and the Hurwitz-Radon theory of composition of quadratic forms.
Hill-ADAM optimizes loss landscapes by exploring state space deterministically.
problem Escaping local minima in loss landscapes.
method Hill-ADAM alternates between minimizing and maximizing error to explore the loss space.
result Hill-ADAM finds the global minimum state in loss landscapes.
EAdam optimizer improves performance by adjusting ε in Adam.
problem Improving performance of Adam optimizer.
method Proposes EAdam optimizer by adjusting the position of ε in Adam.
result EAdam outperforms Adam on various tasks and models.
Adam's generalization performance is improved by batch size and weight decay in neural networks.
problem Understanding how batch size and weight decay affect Adam's generalization in neural networks.
method Theoretical analysis of two-layer over-parameterized CNNs on image data.
result Adam's mini-batch variants can achieve near-zero test error, unlike full-batch Adam.
AdamS uses momentum as a denominator to optimize LLMs efficiently.
problem Optimizing large language models (LLMs) with efficient and effective methods.
method AdamS introduces a novel denominator based on the root of the weighted sum of squares of momentum and current gradient.
result AdamS achieves superior optimization performance with minimal memory and compute requirements.
Adam optimizes linear classifiers with separable data.
problem Understanding Adam's implicit bias in linear logistic regression.
method Study of Adam's behavior on linearly separable data.
result Adam converges to a linear classifier with maximum -margin.
Simpler, parameter-free AdaGrad and Adam variants with convergence guarantees.
problem Inefficiencies in ad-hoc learning rate tuning for optimization algorithms.
method Developed AdaGrad++ and Adam++ without predefined learning rates and proved their convergence.
result AdaGrad++ and Adam++ achieve comparable convergence rates to AdaGrad and Adam respectively.
A new memory-efficient Adam variant reduces second moments when feasible.
problem Memory constraints in training machine learning models.
method Signal-to-Noise Ratio (SNR) analysis to identify dimensions where second moments can be replaced by means.
result Memory-efficient Adam variant (SlimAdam) matches performance and stability of Adam while saving up to 98% of second moments.
Adam optimizer converges to zeros of a new vector field, not just gradient zeros.
problem Prove convergence rates for Adam optimizer in simple quadratic optimization problems.
method Introduced Adam vector field to analyze Adam optimizer's convergence.
result Established optimal convergence rates for Adam optimizer.
AdaX improves Adam by exponentially accumulating past gradients, leading to better performance in machine learning tasks.
problem Adam's fast convergence can lead to local minimums in non-convex problems.
method AdaX exponentially accumulates past gradients to adaptively tune the learning rate.
result AdaX outperforms Adam in various machine learning tasks, including computer vision and natural language processing.
AdaSGD combines SGD and Adam benefits, eliminating the need for transition.
problem Understanding when to transition from Adam to SGD for optimal performance.
method Adapting a single global learning rate for SGD (AdaSGD).
result AdaSGD combines the benefits of both SGD and Adam, improving convergence and generalization.
Adaptive optimization algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, recent studies show that they often lead to worse generalization performance than SGD, especially for training deep neural networks (DNNs). In this wo…
Improved Adam for time series forecasting with distributional drift.
problem Non-stationary data challenges Adam's effectiveness.
method Proposed TS_Adam, removing Adam's second-order bias correction.
result TS_Adam achieves 12.8% reduction in MSE and 5.7% in MAE on ETT datasets.
Combining Adam and its Inverse Counterpart to Enhance Generalization of Deep Learning Optimizerscs.LG
DualAdam improves generalization of Adam by integrating its update mechanisms.
problem Adam's tendency to converge to sharp minima leading to suboptimal generalization.
method DualAdam combines Adam and inverse Adam's update mechanisms to enhance generalization.
result DualAdam outperforms Adam and state-of-the-art variants in generalization performance.
Adam converges to stationary points under relaxed conditions.
problem Understanding and proving convergence of Adam under realistic assumptions.
method New proof of boundedness of gradients and variance-reduced Adam.
result Adam converges to ε-stationary points with O(ε⁻⁴) gradient complexity under realistic conditions.
The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't. Why? We interpret ADAM as a combination of two aspects: for each weight, the update direction is determined by the sign of stochastic gradients, whereas the update magnitude is determined by an esti…
Novel Adam-family method with decoupled weight decay for training neural networks.
problem Training nonsmooth neural networks with weight decay.
method Proposes a novel Adam-family method with decoupled weight decay, establishing convergence properties and demonstrating superior performance.
result Asymptotically approximates SGD and enhances generalization performance.
AdamCB optimizes neural network training by adaptively selecting samples.
problem Inefficient convergence due to unequal influence of different data samples.
method Integrates combinatorial bandit techniques into Adam to adaptively select samples.
result AdamCB achieves faster convergence and better performance than existing methods.
APMSqueeze improves Adam for faster training with less communication.
problem Adam's inefficiency in parallel training due to gradient compression limitations.
method Error-compensated gradient compression for Adam-preconditioned Momentum SGD.
result Significantly reduces communication time per epoch without sacrificing accuracy.
Paper studies Adam's convergence under relaxed assumptions, proving a rate of O(poly(log T)/sqrt(T)).
problem Understanding Adam's convergence in non-convex, stochastic optimization with unbounded gradients and noise.
method Introduced a comprehensive noise model and used it to prove Adam's convergence rate.
result Adam finds a stationary point with a rate of O(poly(log T)/sqrt(T)) in high probability.
New algorithms improve generalization of Adam and AdamW optimizers.
problem Adam and AdamW optimizers generalize worse than SGD.
method Algorithmic stability analysis and clever momentum-based SGD integration.
result HomeAdam(W) algorithms achieve better generalization and faster convergence.
Adam's convergence rate in nonsmooth nonconvex optimization is analyzed.
problem Understanding Adam's convergence behavior in nonsmooth nonconvex optimization.
method Finite-time analysis of Adam's classical form.
result Proven convergence rate of for nonsmooth nonconvex optimization.
Adam optimization algorithm can have non-zero average regret under certain conditions.
problem Non-zero average regret in Adam optimization algorithm.
method Used a three-periodic sequence of linear functions on [-1,1] with slopes c, -1, -1, and analyzed Adam variants.
result Adam optimization algorithm can have non-zero average regret under certain conditions.
This work analyzes Adam's preconditioning effect on quadratic functions and quantifies its impact on condition number.
problem Understanding and quantifying the preconditioning effect of Adam to alleviate ill-conditioning in gradient descent.
method Detailed analysis of Adam's preconditioning effect for quadratic functions, including empirical evidence.
result Adam can mitigate the condition number but at a dimension-dependent cost, with specific bounds for different types of Hessians.
GRU models with Adam optimizer outperform other combinations in stock market forecasting.
problem Comparing optimization techniques for time series forecasting in LSTM and GRU networks.
method Examined Adam and Nesterov Accelerated Gradient (NAG) on LSTM and GRU models for stock market forecasting.
result GRU models with Adam optimizer produced the lowest RMSE and outperformed other combinations.
Adam avoids simplicity bias in neural networks, leading to better generalization.
problem Simplicity bias in neural networks trained with SGD.
method Comparison of Adam and GD on binary classification tasks with Gaussian data.
result Adam leads to richer and more diverse features, improving generalization.
Adam converges with high probability under unconstrained non-convex smooth stochastic optimizations.
problem Theoretical limitations of Adam's convergence under unconstrained non-convex smooth stochastic optimizations.
method Deep analysis of Adam's convergence rate under affine variance noise, without bounded gradient assumptions.
result Adam converges to the stationary point with a high probability rate of $\mathcal{O}\left({
m poly}(\log T)/\sqrt{T}
ight)$.
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularizationcs.LG
Adam can lead to worse test errors than GD in deep learning, especially with over-parameterized networks.
problem Adam's inferior generalization performance compared to GD in deep learning optimization.
method Theoretical analysis of Adam and gradient descent in over-parameterized two-layer convolutional neural networks.
result Adam and GD can converge to different global solutions with different generalization errors in nonconvex optimization landscapes.
The adaptive optimizer for training neural networks has continually evolved to overcome the limitations of the previously proposed adaptive methods. Recent studies have found the rare counterexamples that Adam cannot converge to the optimal point. Those counterexamples reveal the distortion of Adam due to a small secon…
MaxVA improves Adam's step sizes by maximizing gradient variance.
problem Unstable or extreme adaptive learning rates in Adam.
method Maximizing the variance of gradient coordinates in Adam's running mean of squared gradients.
result Faster adaptation and more desirable convergence behaviors than Adam.
Adam's bias shifts from full-batch to max-margin of different norms for separable data.
problem Understanding Adam's implicit bias in the incremental batch setting.
method Analyzing incremental Adam on linearly separable data, constructing datasets, and using a proxy algorithm.
result Incremental Adam can converge to different max-margin classifiers depending on the dataset and batching scheme.
First-order optimization algorithms have been proven prominent in deep learning. In particular, algorithms such as RMSProp and Adam are extremely popular. However, recent works have pointed out the lack of ``long-term memory" in Adam-like algorithms, which could hamper their performance and lead to divergence. In our s…
This research explains why SGD generalizes better than ADAM in deep learning.
problem Understanding the generalization gap between SGD and ADAM in deep learning.
method Analyzing local convergence behaviors through Levy-driven stochastic differential equations (SDEs).
result SGD is more locally unstable and better escapes from sharp minima to flatter ones, leading to better generalization.
We discuss an universal bordism invariant obtained from the Atiyah-Patodi-Singer eta-invariant from the analytic and homotopy theoretic point of view. Classical invariants like the Adams e-invariant, -invariants and -bordism invariants are derived as special cases. The main results are a secondary index theo…
Adam is shown not being able to converge to the optimal solution in certain cases. Researchers recently propose several algorithms to avoid the issue of non-convergence of Adam, but their efficiency turns out to be unsatisfactory in practice. In this paper, we provide new insight into the non-convergence issue of Adam …
Adam and RMSProp are two of the most influential adaptive stochastic algorithms for training deep neural networks, which have been pointed out to be divergent even in the convex setting via a few simple counterexamples. Many attempts, such as decreasing an adaptive learning rate, adopting a big batch size, incorporatin…
The paper analyzes Adam and SGD in nonstationary optimization, revealing tradeoffs between noise and drift.
problem Analyzing Adam and SGD in nonstationary optimization problems.
method Theoretical analysis of Adam and SGD under non-stationary stochastic objectives, separating two regimes.
result Characterizes the tradeoff between noise and drift in Adam and SGD, revealing when adaptive step-sizing is beneficial or harmful.
Adambs adapts Adam to prioritize important training examples.
problem Training deep learning models efficiently with varying example importance.
method Adapt Adam using a bandit sampling approach to prioritize examples based on their importance.
result Adambs improves convergence rate compared to Adam in some cases.
We provide a simple convergence proof for Adam and Adagrad.
problem Smooth objective functions with bounded gradients.
method Simple proof covering Adam and Adagrad.
result Explicit upper-bound on the squared norm of the objective gradient.