Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “Acceleration Methods”
The paper explores how accelerated methods in optimization can be understood through a continuous-time perspective.
problem Understanding the natural scope of acceleration in optimization methods.
method A variational perspective using a Bregman Lagrangian to study accelerated methods in optimization.
result Accelerated methods can be seen as traveling the same curve in spacetime at different speeds, providing a unified view.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
New RNA scheme accelerates gradient methods online and improves convergence.
problem Improving convergence rates of gradient methods.
method Adapting Regularized Nonlinear Acceleration (RNA) to handle faster multistep algorithms.
result Optimal complexity bounds and asymptotically optimal rates for convex minimization problems.
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
Improved Analysis of the Accelerated Noisy Power Method with Applications to Decentralized PCAstat.ML
Improved analysis of accelerated noisy power method for PCA.
problem Inexact matrix-vector products in PCA settings.
method Improved analysis of Accelerated Noisy Power Method under milder perturbation conditions.
result Worst-case optimal convergence rate with relaxed noise conditions.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
Paper accelerates SAGA using sampled negative momentum.
problem Directly accelerating SAGA for optimization problems.
method Proposes SSNM for direct acceleration of SAGA.
result Achieves best known oracle complexity for strongly convex problems.
New method uses centripetal acceleration to improve GAN training.
problem Cyclic behaviors in GAN training.
method Simultaneous Centripetal Acceleration (SCA) and Alternating Centripetal Acceleration (ACA) methods.
result Gradient descent methods with SCA or ACA are linearly convergent for bilinear games.
Two accelerated methods for linearly constrained convex programming are proposed, improving convergence rates.
problem Efficiently solving structured linearly constrained convex programming problems.
method Two accelerated methods: LALM and LADMM, for composite convex objectives.
result Accelerated methods achieve faster convergence rates compared to non-accelerated methods.
Many applications require that we learn the parameters of a model from data. EM is a method used to learn the parameters of probabilistic models for which the data for some of the variables in the models is either missing or hidden. There are instances in which this method is slow to converge. Therefore, several accele…
Improved training of large-scale neural networks with reduced variance noise.
problem Training large-scale neural networks with high variance noise.
method Stochastic variance reduced Nesterov's Accelerated Quasi-Newton method (SVR-NAQ).
result Improved performance compared to conventional methods on benchmark problems.
A new method accelerates K-Means clustering by reducing the number of iterations.
problem Reducing the number of iterations required for K-Means clustering convergence.
method Applying Anderson acceleration to the assignment and update steps of Lloyd's algorithm, dynamically adjusting the number of previous iterates used.
result Achieves robust and consistent speedups across different problem instances, outperforming other algorithms in 106 out of 120 test cases.
Catalyst accelerates convex optimization methods with practical guidelines.
problem Improving convergence of gradient-based optimization methods.
method Catalyst scheme based on inexact accelerated proximal point algorithm.
result Faster convergence rates for various convex optimization methods.
Locally Accelerated Conditional Gradients improve convergence rates for smooth convex optimization problems.
problem Achieving optimal convergence rates for smooth convex optimization problems over polytopes.
method Locally Accelerated Conditional Gradients, coupling accelerated steps with conditional gradient steps.
result Achieves optimal accelerated local convergence for smooth strongly convex problems.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
Accelerated gradient method's stability deteriorates exponentially with steps.
problem Algorithmic stability of Nesterov's accelerated gradient method.
method Analysis of two notions of algorithmic stability for Nesterov's accelerated gradient method.
result Stability of Nesterov's accelerated method deteriorates exponentially with the number of gradient steps.
ASVRG accelerates stochastic variance reduction methods with simplicity and efficiency.
problem Efficiently solving convex and non-convex optimization problems.
method Accelerated proximal stochastic variance reduced gradient (ASVRG) method with momentum acceleration.
result ASVRG achieves best known oracle complexities for strongly and non-strongly convex objectives.
New method accelerates steepest descent for convex optimization.
problem Achieving acceleration for general smooth functions.
method Primal-dual iterate sequences with differing norms, implicitly determined interpolation parameter.
result Improves iteration complexity to for norm smooth problems.
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
problem Locally accelerated CG requires knowledge of smoothness and strong convexity parameters.
method Parameter-Free Locally Accelerated CG (PF-LaCG) algorithm.
result PF-LaCG achieves local acceleration without requiring knowledge of smoothness and strong convexity parameters.
Accelerated optimization on curved geometry analyzed.
problem Understanding the accelerated gradient method in curved spaces.
method Geometric interpretation of AGM on Riemannian manifold, continuous time ODE analysis.
result Convergence rate analysis of ODE for quadratic objectives.
Super-acceleration of gradient descent with momentum improves loss function minimization.
problem Minimizing loss functions in machine learning.
method Extending Nesterov acceleration by using gradients at multiple steps ahead.
result Super-acceleration of the momentum algorithm is beneficial for various loss landscapes and tasks.
Two new differentially private optimization algorithms derived from accelerated methods.
problem Improving privacy in optimization algorithms while maintaining convergence rates.
method Polyak's heavy ball method and Nesterov's accelerated gradient method with differential privacy.
result The proposed algorithms outperform existing differentially private optimization methods.
Improved quasi-Newton method for convex optimization with linear and accelerated convergence.
problem Efficiently solving composite optimization problems with strong convexity.
method Proximal quasi-Newton algorithm with accelerated variant.
result Acceleration may not improve convergence in quasi-Newton setting.
Symplectic discretization accelerates optimization of smooth convex functions.
problem Optimizing smooth convex functions efficiently.
method Discretizing Nesterov's and Polyak's methods using Euler and symplectic schemes.
result Symplectic discretization achieves accelerated optimization for smooth convex functions.
New method accelerates gradient descent on curved spaces.
problem Optimizing functions on curved Riemannian manifolds.
method Developed a novel geometric inequality to control metric distortion, enabling a Riemannian accelerated gradient method.
result Proposed the first global accelerated gradient method for Riemannian manifolds.
A new method speeds up optimization for sums, simpler than others.
problem Optimizing finite sums, especially in non-smooth cases.
method A novel accelerated method based on SAGA, with one parameter.
result Achieves accelerated convergence on strongly convex smooth problems.
This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in distance for log-strongly-concave targets.
Accelerates Q-learning by updating targets more efficiently.
problem Improving convergence speed in Q-learning algorithms.
method Proposes an accelerated target update scheme based on momentum methods.
result The proposed algorithms converge faster than vanilla Q-learning.
Novel method improves training RNNs by accelerating gradient descent.
problem Vanishing and exploding gradient problems in RNNs training.
method Adaptive stochastic Nesterov accelerated quasi-Newton method.
result Improved performance in training RNNs with low per-iteration cost.
This work refutes the conventional wisdom and shows acceleration can be made robust for least squares regression.
problem The challenge of using fast gradient methods for stochastic optimization due to instability and error accumulation.
method Introduced an accelerated stochastic gradient method for least squares regression.
result Proves accelerated stochastic gradient descent achieves minimax optimal statistical risk faster than SGD.
Interpolatron accelerates deep neural network optimization faster than existing methods.
problem Accelerating nonconvex optimization for deep neural networks.
method Proposes Interpolatron, a new interpolation scheme to accelerate nonconvex optimization.
result Interpolatron converges much faster than state-of-the-art methods on DNNs of great depths.
This paper reviews and analyzes methods to accelerate CNNs.
problem Computational intensity of CNNs limits their further implementation.
method Taxonomy of acceleration methods at structure, algorithm, and implementation levels.
result Large exploration space remains for each level of acceleration methods.
Improved Anderson acceleration speeds up nonlinear optimization.
problem Optimizing nonlinear functions efficiently.
method Combining Anderson acceleration with Chebyshev polynomials.
result Achieves optimal convergence rate for nonlinear problems.
Study accelerates gradient methods in machine learning, revealing risk and stability connections.
problem Understanding the statistical risk of accelerated gradient methods in machine learning.
method Continuous-time analysis of Nesterov's accelerated gradient method and Polyak's heavy ball method for least squares regression.
result Connections between early stopping, stability, and curvature of loss function are revealed.
Accelerates MMLE using SVGD with Nesterov acceleration.
problem Maximum Marginal Likelihood Estimation optimization.
method Stein variational gradient descent with Nesterov acceleration.
result Consistently accelerates convergence across various tasks.
Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein spacemath.OC
New algorithm accelerates optimization on Riemannian manifolds, including Wasserstein space.
problem Accelerating optimization methods in Riemannian geometry.
method Dynamic stepsize algorithms on Riemannian manifolds with specific vector transport.
result First provable accelerated gradient method in Wasserstein space.
This paper improves NAQ method for faster convergence on Tensorflow.
problem Non-convex optimization problems.
method Modified Nesterov's Accelerated Quasi-Newton (NAQ) method on Tensorflow.
result mNAQ converges better and faster than first and second order optimizers.
New methods accelerate gradient descent for convex and strongly convex functions.
problem Improving convergence rates of gradient-based optimization methods.
method Formulated two classes of first-order algorithms with Lyapunov analyses and Hamiltonian assisted gradient method.
result Achieved accelerated convergence rates matching Nesterov's methods in strongly and general convex settings.
AGNES accelerates gradient descent with noisy gradients.
problem Minimizing smooth convex and strongly convex functions with noisy gradients.
method Generalization of Nesterov's accelerated gradient descent algorithm for noisy conditions.
result AGNES achieves acceleration for noisy gradients with a constant of proportionality up to 1.
Unified analysis of conjugate gradients and accelerated methods using duality gap.
problem Minimizing convex quadratic functions efficiently.
method Approximate Duality Gap Technique to unify conjugate gradients and accelerated methods.
result Unified and self-contained proof of conjugate gradients without relying on Chebyshev polynomials.
Paper accelerates NMF algorithms using extrapolation.
problem Efficiently solving nonnegative matrix factorization problems.
method Inspired by convex optimization, uses extrapolation for two-block exact coordinate descent algorithms.
result Significantly accelerates NMF algorithms on various data sets.
Accelerates machine learning algorithms for sparse data.
problem Efficiently solving composite convex minimization problems.
method Accelerated dual-averaging primal-dual method for composite convex minimization.
result Demonstrates advantages in handling sparse data both theoretically and empirically.
New method accelerates gradient optimization using curvature information.
problem Solving large-scale learning optimization problems efficiently.
method Curvature-aided gradient tracking step for incremental gradient optimization.
result Proves linear convergence rates for new methods.
Sinh-acceleration speeds up B-spline option pricing.
problem Improving efficiency in option pricing calculations.
method Using sinh-acceleration on B-spline probability density projection.
result SINH acceleration technique improves error control and reduces CPU time.
New adaptive and accelerated SGD methods achieve optimal convergence rates.
problem Optimizing convergence rates of stochastic gradient descent methods.
method Integrates diagonal scaling and momentum into accelerated SGD.
result Achieves optimal sampling and iteration complexity for smooth stochastic optimization.