Efficiently optimizes CNN and RNN parameters on Stiefel manifold.
problem Computational expense in optimizing orthonormal matrices on Stiefel manifold.
method Cayley transform for efficient retraction and vector transport on Stiefel manifold.
result Cayley SGD and ADAM achieve faster convergence and less training time.
This paper establishes inequalities on quaternionic hyperbolic spaces and the Cayley hyperbolic plane.
problem Establishing higher order Poincaré-Sobolev and Hardy-Sobolev-Maz'ya inequalities on quaternionic hyperbolic spaces and the Cayley hyperbolic plane.
method Developing factorization theorems and introducing Geller's operators, combining with Helgason-Fourier analysis and kernel estimates.
result Established higher order Poincaré-Sobolev and Hardy-Sobolev-Maz'ya inequalities on quaternionic hyperbolic spaces and the Cayley hyperbolic plane.
This work improves disentanglement in latent space models without sacrificing generation quality.
problem Trade-off between disentanglement and generation quality in latent space models.
method Manifold optimization with a sum of autoencoder and PCA reconstruction errors, on the Stiefel manifold.
result Improves disentanglement without sacrificing generation quality.
Deformations of singular Cayley submanifolds studied.
problem Constructing fibrations of compact Spin(7) manifolds.
method Deformation theory of conically singular and asymptotically conical Cayley submanifolds.
result Detailed description of the deformation theory.
In this article we study the deformation theory of conically singular Cayley submanifolds. In particular, we prove a result on the expected dimension of a moduli space of Cayley deformations of a conically singular Cayley submanifold. Moreover, when the Cayley submanifold is a two-dimensional complex submanifold of a C…
Desingularizes conically singular Cayley submanifolds.
problem Constructing fibrations of compact Spin(7) manifolds by Cayley submanifolds.
method Desingularization through gluing rescaled asymptotically conical submanifolds.
result Conically singular Cayley submanifolds can be desingularized.
Third in a series, this paper constructs non-trivial Cayley fibrations with conical singularities.
problem Constructing non-trivial Cayley fibrations with conical singularities.
method Using gluing methods and stability results for weak and usual fibrations.
result Construction of examples of Cayley fibrations on twisted connected sum G2 manifolds. The study examines discrete curvature notions on Cayley graphs of certain groups.
problem Understanding curvature in discrete settings for various groups.
method Introduced Right Angled Artin-Coxeter Hybrids (RAACHs) and derived curvatures of Cayley graphs.
result Addition of relators does not decrease weighted curvatures of Cayley graphs.
Study Cayley fibrations on Bryant-Salamon Spin(7) manifolds.
problem Investigate Cayley fibrations on specific Spin(7) manifolds. method Analyze invariant Cayley fibrations for each SU(2) action. result Explicitly describe the fibres of Cayley fibrations.
Given a Kaehler manifold of complex dimension 4, we consider submanifolds of (real) dimension 4, whose Kaehler angles coincide. We call these submanifolds Cayley. We investigate some of their basic properties, and prove that (a) if the ambient manifold is a Calabi-Yau, the minimal Cayley submanifolds are just the Cayle…
Eastwood and Ezhov generalized the Cayley surface to the Cayley hypersurface in each dimension, proved some characteristic properties of the Cayley hypersurface and conjectured that a homogeneous hypersurface in affine space satisfying these properties must be the Cayley hypersurface. We will prove this conjecture when…
The study introduces Cayley--Abels--Rosendal graphs for Polish groups.
problem Understanding the structure of Polish groups through graph theory.
method Developing Cayley--Abels--Rosendal graphs and applying them to Polish groups.
result Groups with Cayley--Abels--Rosendal graphs are topological analogues of finitely generated groups.
Quasi-transitive graphs quasi-isometric to planar graphs can be upgraded to Cayley graphs.
problem Quasi-transitive graphs quasi-isometric to planar graphs need to be upgraded to Cayley graphs.
method Upgrading a planar graph to a Cayley graph.
result Quasi-transitive graphs quasi-isometric to planar graphs can be upgraded to Cayley graphs.
We define Cayley structures as a field of Cayley's ruled cubic surfaces over a four dimensional manifold and motivate their study by showing their similarity to indefinite conformal structures and their link to differential equations. In particular, for Cayley structures an extension of certain notions defined for inde…
Cayley cones in the octonions O that are ruled by oriented 2-planes are equivalent to pseudoholomorphic curves in the Grassmannian of oriented 2-planes G(2,8). The well known twistor fibration G(2,8)−>S6 is used to prove the existence of immersed higher-genus pseudoholomorphic curves in $\gro$. Equivale…
The Cayley graph of quandles reveals structural properties and is studied for various classes.
problem Investigating structural properties of Cayley graphs of quandles.
method Analyzing Cayley graphs for different quandle classes and proving properties.
result Connected components of Cayley graphs of Alexander quandles correspond to cosets of specific subgroups.
Using octonions and the triality property of Spin(8), we find explicit formulae for the Lie brackets of the exceptional simple real Lie algebras f4 and f4∗, i.e. the Lie algebras of the isometry groups of the Cayley projective plane and the Cayley hyperbolic plane. As an application, we cla…
In this paper we study almost complex and almost para-complex Cayley structures on six-dimensional pseudo-Riemannian spheres in the space of purely imaginary octaves of the split Cayley algebra Ca′. It is shown that the Cayley structures are non-integrable, their basic geometric characteristics are calculate…
Introduces Lorentzian Cayley form solving geometric puzzle.
problem Solving geometric puzzle in 4D Lorentzian geometry.
method Introduces complex Cayley forms and Lorentzian Cayley form.
result Lorentzian Cayley form solves geometric puzzle.
Study shows only two topological configurations for Spin(7)-manifold fibrations, ruling out smooth Cayley fibrations.
problem Understanding smooth fibrations of compact Spin(7)-manifolds by Cayley submanifolds.
method Geometric and topological constraints from Spin(7)-structure, spinnability criterion, gauge-theoretic input.
result Rules out smooth Cayley fibrations on all known compact torsion-free Spin(7)-manifolds.
Functions of several octonion variables are investigated and integral representation theorems for them are proved. With the help of them solutions of the ∂~-equations are studied. More generally functions of several Cayley-Dickson variables are considered. Integral formulas of the Martinelli-Bochner,…
We exhibit a family of homogeneous hypersurfaces in affine space, one in each dimension, generalising the Cayley surface.
The paper studies properties of Artin monoid Cayley graphs and their quasi-isometry to Deligne complexes.
problem Investigate properties of Artin monoid Cayley graphs.
method Show quasi-isometry to modified Deligne complex, address infinite diameter conjecture.
result Prove conjecture about infinite diameter for Artin groups containing specific subgroups.
Following an idea of Dadok, Harvey and Lawson, we apply the triality property of SO(8) to study the comass of certain self-dual 4-forms on R^8. In particular, we prove that the Cayley 4-form has comass 1 and that any self-dual 4-form realizing the maximal Wirtinger ratio is SO(8)-conjugate to the Cayley 4-form. We also…
Hill-ADAM optimizes loss landscapes by exploring state space deterministically.
problem Escaping local minima in loss landscapes.
method Hill-ADAM alternates between minimizing and maximizing error to explore the loss space.
result Hill-ADAM finds the global minimum state in loss landscapes.
We prove an existence theorem for Spin(7)-instantons, which are highly concentrated near a Cayley submanifold; thus giving a partial converse to Tian's foundational compactness theorem. As an application, we show how to construct Spin(7)-instantons on Spin(7)-manifolds with suitable local K3 Cayley fibrations. This rec…
The Cayley hyperbolic space minimizes volume entropy among finite-volume metrics.
problem Volume entropy rigidity in Cayley hyperbolic spaces.
method Repairing a gap in the proof of volume entropy rigidity theorem.
result Cayley hyperbolic space minimizes volume entropy.
EAdam optimizer improves performance by adjusting ε in Adam.
problem Improving performance of Adam optimizer.
method Proposes EAdam optimizer by adjusting the position of ε in Adam.
result EAdam outperforms Adam on various tasks and models.
This paper develops graph theory for racks and quasigroups.
problem Characterizing and realizing right quasigroups and related structures.
method Study of graph markings, Schreier graphs, and Cayley graphs.
result All right quasigroups are realizable by specific types of graphs.
Groups can act on 3-manifolds if their Cayley complex can embed in specific types of 3-manifolds.
problem Understanding group actions on 3-manifolds.
method Proving groups admit actions on 3-manifolds if their Cayley complexes can embed in specific 3-manifolds.
result Groups with embeddable Cayley complexes can act on four specific types of 3-manifolds.
Adam's generalization performance is improved by batch size and weight decay in neural networks.
problem Understanding how batch size and weight decay affect Adam's generalization in neural networks.
method Theoretical analysis of two-layer over-parameterized CNNs on image data.
result Adam's mini-batch variants can achieve near-zero test error, unlike full-batch Adam.
AdamS uses momentum as a denominator to optimize LLMs efficiently.
problem Optimizing large language models (LLMs) with efficient and effective methods.
method AdamS introduces a novel denominator based on the root of the weighted sum of squares of momentum and current gradient.
result AdamS achieves superior optimization performance with minimal memory and compute requirements.
Adam optimizes linear classifiers with separable data.
problem Understanding Adam's implicit bias in linear logistic regression.
method Study of Adam's behavior on linearly separable data.
result Adam converges to a linear classifier with maximum ℓ∞-margin. Simpler, parameter-free AdaGrad and Adam variants with convergence guarantees.
problem Inefficiencies in ad-hoc learning rate tuning for optimization algorithms.
method Developed AdaGrad++ and Adam++ without predefined learning rates and proved their convergence.
result AdaGrad++ and Adam++ achieve comparable convergence rates to AdaGrad and Adam respectively.
A new memory-efficient Adam variant reduces second moments when feasible.
problem Memory constraints in training machine learning models.
method Signal-to-Noise Ratio (SNR) analysis to identify dimensions where second moments can be replaced by means.
result Memory-efficient Adam variant (SlimAdam) matches performance and stability of Adam while saving up to 98% of second moments.
Adam optimizer converges to zeros of a new vector field, not just gradient zeros.
problem Prove convergence rates for Adam optimizer in simple quadratic optimization problems.
method Introduced Adam vector field to analyze Adam optimizer's convergence.
result Established optimal convergence rates for Adam optimizer.
AdaX improves Adam by exponentially accumulating past gradients, leading to better performance in machine learning tasks.
problem Adam's fast convergence can lead to local minimums in non-convex problems.
method AdaX exponentially accumulates past gradients to adaptively tune the learning rate.
result AdaX outperforms Adam in various machine learning tasks, including computer vision and natural language processing.
AdaSGD combines SGD and Adam benefits, eliminating the need for transition.
problem Understanding when to transition from Adam to SGD for optimal performance.
method Adapting a single global learning rate for SGD (AdaSGD).
result AdaSGD combines the benefits of both SGD and Adam, improving convergence and generalization.
Adaptive optimization algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, recent studies show that they often lead to worse generalization performance than SGD, especially for training deep neural networks (DNNs). In this wo…
Improved Adam for time series forecasting with distributional drift.
problem Non-stationary data challenges Adam's effectiveness.
method Proposed TS_Adam, removing Adam's second-order bias correction.
result TS_Adam achieves 12.8% reduction in MSE and 5.7% in MAE on ETT datasets.
Extends Adams' theorem to periodic cohomology.
problem Proving Adams' theorem for periodic cohomology.
method Adapting Adams' approach to periodic cohomology.
result Conjecture proven in a special case.
In this article, we consider Cayley deformations of a compact complex surface in a Calabi--Yau four-fold. We will study complex deformations of compact complex submanifolds of Calabi--Yau manifolds with a view to explaining why complex and Cayley deformations of a compact complex surface are the same. We in fact prove …
DualAdam improves generalization of Adam by integrating its update mechanisms.
problem Adam's tendency to converge to sharp minima leading to suboptimal generalization.
method DualAdam combines Adam and inverse Adam's update mechanisms to enhance generalization.
result DualAdam outperforms Adam and state-of-the-art variants in generalization performance.
Adam converges to stationary points under relaxed conditions.
problem Understanding and proving convergence of Adam under realistic assumptions.
method New proof of boundedness of gradients and variance-reduced Adam.
result Adam converges to ε-stationary points with O(ε⁻⁴) gradient complexity under realistic conditions.
In this paper we introduce the area of 2-ruled 4-folds in R^n (n=7 or 8), that is, submanifolds M of R^n that admit a fibration over some 2-fold Sigma such that each fibre is an affine 2-plane in R^n. This is motivated by the paper math.DG/0012060 by Joyce on ruled special Lagrangian 3-folds in C^3 and the work of the …
We construct a Kaehler structure on the punctured cotangent bundle of the Cayley projective plane whose Kaehler form coincides with the natural symplectic form on the cotangent bundle and we show that the geodesic flow action is holomorphic and is expressed in a quite explicit form. We also give an embedding of the pun…
Laprop separates Adam's momentum and adaptivity to improve stability and speed.
problem Unnecessary coupling between Adam's momentum and adaptivity leads to instability and divergence.
method Proposes Laprop, a method that decouples momentum and adaptivity.
result Laprop consistently improves speed and stability over Adam on various tasks.
The ADAM optimizer is exceedingly popular in the deep learning community. Often it works very well, sometimes it doesn't. Why? We interpret ADAM as a combination of two aspects: for each weight, the update direction is determined by the sign of stochastic gradients, whereas the update magnitude is determined by an esti…