Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Riemannian acceleration”
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein spacemath.OC
New algorithm accelerates optimization on Riemannian manifolds, including Wasserstein space.
problem Accelerating optimization methods in Riemannian geometry.
method Dynamic stepsize algorithms on Riemannian manifolds with specific vector transport.
result First provable accelerated gradient method in Wasserstein space.
We propose the first global accelerated gradient method for Riemannian manifolds. Toward establishing our result we revisit Nesterov's estimate sequence technique and develop an alternative analysis for it that may also be of independent interest. Then, we extend this analysis to the Riemannian setting, localizing the …
Riemannian cubics are critical points for the norm of acceleration of curves in Riemannian manifolds . In the present paper the norm replaces the norm, and a less direct argument is used to derive necessary conditions analogous to those for Riemannian cubics. The necessary conditions are exami…
No accelerated gradient method for hyperbolic convex functions.
problem Existence of accelerated gradient methods for geodesically convex functions on hyperbolic spaces.
method Analysis of volume growth in negatively curved spaces.
result No-go theorem for accelerated gradient methods on hyperbolic plane.
Accelerated method finds critical points faster on manifolds.
problem Optimization on non-convex manifolds.
method Accelerated gradient methods on Riemannian manifolds.
result Find approximate first-order critical points faster than regular gradient descent.
New methods optimize functions on hyperbolic and spherical spaces, matching Euclidean rates up to logarithmic factors.
problem Optimizing functions on non-Euclidean spaces like hyperbolic and spherical geometries.
method Introduced accelerated global first-order methods for -smooth and geodesically convex functions on hyperbolic and spherical spaces.
result Achieved the same rates as accelerated gradient descent in Euclidean space, up to logarithmic factors.
A geometric framework for metrics of maximal acceleration which is applicable to large proper accelerations is discussed, including a theory of connections associated with the geometry of maximal acceleration. In such a framework it is shown that the uniform bound on the proper maximal acceleration implies an uniform b…
Sub-Riemannian cubics are a generalisation of Riemannian cubics to a sub-Riemannian manifold. Cubics are curves which minimise the integral of the norm squared of the covariant acceleration. Sub-Riemannian cubics are cubics which are restricted to move in a horizontal subspace of the tangent space. When the sub-Riemann…
In this work we propose a differential geometric motivation for Nesterov's accelerated gradient method (AGM) for strongly-convex problems. By considering the optimization procedure as occurring on a Riemannian manifold with a natural structure, The AGM method can be seen as the proximal point method applied in this cur…
New study shows acceleration in hyperbolic spaces is impossible for strongly geodesically convex functions.
problem Acceleration in hyperbolic spaces for strongly geodesically convex functions is impossible.
method Perturbing hard functions with sums of bump functions chosen by a resisting oracle.
result Acceleration is unachievable for any deterministic algorithm in hyperbolic spaces for strongly geodesically convex functions.
This paper analyzes two Lie group momentum optimization algorithms and their convergence rates.
problem Optimizing functions on Lie groups using momentum-based dynamics.
method Investigates Lie Heavy-Ball and Lie NAG-SC algorithms, quantifying their convergence rates under smoothness and convexity assumptions.
result Lie NAG-SC accelerates optimization over the momentumless case, while Lie Heavy-Ball does not.
The Variational Principle for the Uniform Acceleration and Quasi-Spin in Two Dimensional Space-Timemath-ph
The variational principle and the corresponding differential equation for geodesic circles in two dimensional (pseudo)-Riemannian space are being discovered. The relationship with the physical notion of uniformly accelerated relativistic particle is emphasized. The known form of spin-curvature interaction emerges due t…
A variational model for data fitting on manifolds by minimizing the acceleration of a Bézier curvemath.NA
We derive a variational model to fit a composite Bézier curve to a set of data points on a Riemannian manifold. The resulting curve is obtained in such a way that its mean squared acceleration is minimal in addition to remaining close the data points. We approximate the acceleration by discretizing the squared second o…
Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamicsstat.ME
Novel geometry-informed irreversible perturbation accelerates Langevin dynamics convergence.
problem Accelerating convergence of Langevin dynamics for Bayesian computation.
method Geometry-informed irreversible perturbation of Riemannian manifold Langevin dynamics.
result Improves estimation performance over irreversible perturbations that ignore geometry.
New method synthesizes data on curved spaces for better interpolation.
problem Synthesizing data on curved spaces for better interpolation.
method Riemannian Diffusion Schrödinger Bridge
result Generalizes Diffusion Schrödinger Bridge to curved spaces for better interpolation.
A simple model for unbalanced optimal transport captures key features.
problem Capturing the main features of unbalanced optimal transport.
method Introducing a metric on the conical extension of diffeomorphisms and studying its properties.
result Total mass evolves with constant acceleration along geodesics.
Several first order stochastic optimization methods commonly used in the Euclidean domain such as stochastic gradient descent (SGD), accelerated gradient descent or variance reduced methods have already been adapted to certain Riemannian settings. However, some of the most popular of these optimization tools - namely A…
RieCUR improves Robust PCA by combining Riemannian optimization and CUR decompositions.
problem Robust Principal Component Analysis (PCA) to recover low-rank and sparse matrices from their sum.
method Riemannian CUR (RieCUR) algorithm that combines Riemannian optimization and robust CUR decompositions.
result RieCUR achieves state-of-the-art performance in Robust PCA with improved robustness to outliers and comparable computational complexity.
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
Reduces necessary conditions for collision avoidance on curved spaces.
problem Finding non-intersecting trajectories for multiple agents on curved spaces.
method Reduction by Lie group symmetries of variational collision avoidance problems.
result Derives necessary conditions for reduced extremals.
New quasi-geodesics for Stiefel manifold simplify complex computations.
problem Efficiently solving geodesic endpoint problem on Stiefel manifold.
method Derived new representations of quasi-geodesics for large-scale computations.
result New quasi-geodesics are closer to Riemannian geodesics.
We consider the minimization of a function defined on a Riemannian manifold accessible only through unbiased estimates of its gradients. We develop a geometric framework to transform a sequence of slowly converging iterates generated from stochastic gradient descent (SGD) on to an averaged i…
A Riemannian Optimization Perspective of the Gauss-Newton Method for Feedforward Neural Networksmath.OC
The Gauss-Newton method is analyzed for neural networks using Riemannian optimization techniques.
problem Training neural networks with smooth activations and convergence rates.
method Riemannian optimization perspective, analyzing the Gauss-Newton method in both underparameterized and overparameterized regimes.
result Geometric convergence rates independent of conditioning and eigenvalues, demonstrating accelerated convergence.
New method generates equilibrium glass configurations efficiently.
problem Sampling equilibrium configurations of amorphous materials is slow and difficult.
method Riemannian stochastic interpolation framework combining Riemannian stochastic interpolant and equivariant flow matching.
result Enforcing geometric and symmetry constraints significantly improves generative performance.
The paper analyzes fixed step-size SA schemes on Riemannian manifolds.
problem Developing efficient algorithms for optimization on curved spaces.
method Fixed step-size stochastic approximation schemes in a Riemannian framework.
result The schemes converge to the solution as the step-size approaches zero.
Optimization on manifolds: A symplectic approachcond-mat.stat-mech
New method solves optimization problems on manifolds using symplectic integrators.
problem Optimization tasks on manifolds with nonlinear constraints.
method Dissipative extension of Dirac's theory of constrained Hamiltonian systems and geometric/symplectic numerical integrators.
result Developed algorithms achieve optimal convergence rates locally.
Recently, classical results on completeness of trajectories of Hamiltonian systems obtained at the beginning of the seventies, have been revisited, improved and applied to Lorentzian Geometry. Our aim here is threefold: to give explicit proofs of some technicalities in the background of the specialists, to show that th…
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
We propose an L-BFGS optimization algorithm on Riemannian manifolds using minibatched stochastic variance reduction techniques for fast convergence with constant step sizes, without resorting to linesearch methods designed to satisfy Wolfe conditions. We provide a new convergence proof for strongly convex functions wit…
Formulates mechanics for probability distributions on statistical manifold.
problem Formulating mechanics for probability distributions on statistical manifold.
method Information-geometric formulation of Classical Mechanics on statistical manifold, using dually-flat connection and Hilbert bundle structure.
result Provides coherent formalism for Lagrangian and Hamiltonian mechanics on statistical bundle.
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
problem Locally accelerated CG requires knowledge of smoothness and strong convexity parameters.
method Parameter-Free Locally Accelerated CG (PF-LaCG) algorithm.
result PF-LaCG achieves local acceleration without requiring knowledge of smoothness and strong convexity parameters.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossipmath.OC
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelera…
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
FedAc accelerates Federated Averaging for distributed optimization.
problem Efficiently optimizing distributed machine learning models.
method Federated Accelerated Stochastic Gradient Descent (FedAc) using a potential-based perturbed iterate analysis.
result FedAc achieves faster convergence and lower communication costs than previous methods.
This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in distance for log-strongly-concave targets.
This paper studies accelerations in Q-learning algorithms. We propose an accelerated target update scheme by incorporating the historical iterates of Q functions. The idea is conceptually inspired by the momentum-based accelerated methods in the optimization theory. Conditions under which the proposed accelerated algor…
In this study, the concept of dual Lorentzian homotetic exponential motions in is discussed and their velocities, accelerations obtained. Also, some geometric results between velocity and acceleration vectors of a point in a spatial motion are obtained. Finally, the theorems related to acceleration and acceleration cen…
Improved Analysis of the Accelerated Noisy Power Method with Applications to Decentralized PCAstat.ML
Improved analysis of accelerated noisy power method for PCA.
problem Inexact matrix-vector products in PCA settings.
method Improved analysis of Accelerated Noisy Power Method under milder perturbation conditions.
result Worst-case optimal convergence rate with relaxed noise conditions.
Variance reduction is a simple and effective technique that accelerates convex (or non-convex) stochastic optimization. Among existing variance reduction methods, SVRG and SAGA adopt unbiased gradient estimators and are the most popular variance reduction methods in recent years. Although various accelerated variants o…
Two Kaehler metrics on one complex manifold are said to be c-projectively equivalent if their J-planar curves, i.e., curves defined by the property that their acceleration is complex proportional to their velocity, coincide. The degree of mobility of a Kaehler metric is the dimension of the space of metrics that are c-…
A novel online framework for analyzing multidimensional functional data.
problem Analysis of multidimensional functional data streams poses significant challenges.
method Online functional principal component analysis using tensor product splines on a Stiefel manifold with Riemannian stochastic gradient descent.
result Efficient and scalable modeling of multidimensional functional data.
Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
problem Accelerating convex optimization
method Hamiltonian dynamics
result Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
We consider gradient descent with `momentum', a widely used method for loss function minimization in machine learning. This method is often used with `Nesterov acceleration', meaning that the gradient is evaluated not at the current position in parameter space, but at the estimated position after one step. In this work…