Locally Accelerated Conditional Gradients improve convergence rates for smooth convex optimization problems.
problem Achieving optimal convergence rates for smooth convex optimization problems over polytopes.
method Locally Accelerated Conditional Gradients, coupling accelerated steps with conditional gradient steps.
result Achieves optimal accelerated local convergence for smooth strongly convex problems.
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
problem Locally accelerated CG requires knowledge of smoothness and strong convexity parameters.
method Parameter-Free Locally Accelerated CG (PF-LaCG) algorithm.
result PF-LaCG achieves local acceleration without requiring knowledge of smoothness and strong convexity parameters.
Non-convex gradient descent accelerates convergence in matrix factorization models.
problem Non-convex optimization in matrix factorization models.
method Factored gradient descent with acceleration.
result Acceleration leads to linear convergence rate in non-convex settings.
New method accelerates diffusion models for broader target distributions.
problem Current diffusion models have limited acceleration for certain target distributions.
method Developed a novel accelerated stochastic DDPM sampler.
result Achieved accelerated performance for three broad distribution classes.
Unified framework for understanding and optimizing training acceleration.
problem Challenges in optimizing training with regularization and acceleration techniques.
method Explains how AdaGrad, RMSProp, and Adam accelerate training, and derives a generalization for L 1 L_1 L 1 -regularization. result Derives a unified mathematical framework for understanding and optimizing training acceleration.
Improved SGD algorithm with faster convergence.
problem Optimization of machine learning models.
method Conditional accelerated lazy stochastic gradient descent.
result Convergence rate of $O\left(\frac{1}{\varepsilon^2}
ight)$ , faster than previous methods.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
Improved analysis of accelerated noisy power method for PCA.
problem Inexact matrix-vector products in PCA settings.
method Improved analysis of Accelerated Noisy Power Method under milder perturbation conditions.
result Worst-case optimal convergence rate with relaxed noise conditions.
Accelerates Q-learning by updating targets more efficiently.
problem Improving convergence speed in Q-learning algorithms.
method Proposes an accelerated target update scheme based on momentum methods.
result The proposed algorithms converge faster than vanilla Q-learning.
New methods accelerate gradient descent for convex and strongly convex functions.
problem Improving convergence rates of gradient-based optimization methods.
method Formulated two classes of first-order algorithms with Lyapunov analyses and Hamiltonian assisted gradient method.
result Achieved accelerated convergence rates matching Nesterov's methods in strongly and general convex settings.
Study on Nesterov's method in stochastic settings, revealing divergence under certain conditions.
problem Understanding Nesterov's method in stochastic settings, especially finite-sum.
method Analysis of Nesterov's accelerated gradient method in stochastic and finite-sum settings.
result Nesterov's method may diverge in finite-sum settings without additional conditions.
Direct discretization of ODE achieves gradient-based acceleration.
problem Achieving acceleration in gradient-based optimization.
method Direct discretization of a related ODE using Runge-Kutta integrators.
result Convergence to optimal solution at rate O ( N − 2 s s + 1 ) \mathcal{O}({N^{-2\frac{s}{s+1}}}) O ( N − 2 s + 1 s ) . New method uses centripetal acceleration to improve GAN training.
problem Cyclic behaviors in GAN training.
method Simultaneous Centripetal Acceleration (SCA) and Alternating Centripetal Acceleration (ACA) methods.
result Gradient descent methods with SCA or ACA are linearly convergent for bilinear games.
SGD converges fast for over-parameterized models.
problem Training highly expressive models that fit data perfectly.
method Constant step-size SGD with Nesterov acceleration.
result Matches deterministic accelerated method's convergence rate.
Riemannian cubics are critical points for the L 2 L^2 L 2 norm of acceleration of curves in Riemannian manifolds M M M . In the present paper the L ∞ L^\infty L ∞ norm replaces the L 2 L^2 L 2 norm, and a less direct argument is used to derive necessary conditions analogous to those for Riemannian cubics. The necessary conditions are exami…
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
Generative model synthesizes earthquake acceleration data.
problem Robust estimation of ground motions for engineering applications.
method Wasserstein GAN formulation for conditioning on physical variables.
result Trained model synthesizes realistic 3-component accelerograms.
Catalyst accelerates convex optimization methods with practical guidelines.
problem Improving convergence of gradient-based optimization methods.
method Catalyst scheme based on inexact accelerated proximal point algorithm.
result Faster convergence rates for various convex optimization methods.
A new algorithm improves convergence rates for convex optimization problems.
problem Convex optimization problems with finite-sum structure.
method Nesterov Accelerated Shuffling Gradient (NASG) integrating Nesterov's acceleration with different shuffling schemes.
result Improved convergence rate of O(1/T) for unified shuffling schemes.
Polyak's momentum accelerates training of neural networks.
problem Understanding and explaining the acceleration effect of Polyak's momentum in neural network training.
method Modular analysis of Polyak's momentum for training wide ReLU networks and deep linear networks.
result Polyak's momentum achieves an accelerated linear rate of ( 1 − Θ ( 1 κ ′ ) ) t (1-Θ(\frac{1}{\sqrt{κ'}}))^t ( 1 − Θ ( κ ′ 1 ) ) t for training wide ReLU networks and deep linear networks. New algorithm accelerates optimization on Riemannian manifolds, including Wasserstein space.
problem Accelerating optimization methods in Riemannian geometry.
method Dynamic stepsize algorithms on Riemannian manifolds with specific vector transport.
result First provable accelerated gradient method in Wasserstein space.
This work accelerates gradient descent with anytime convergence guarantees.
problem Improving the convergence rate of gradient descent methods.
method Proposes a stepsize schedule for gradient descent that achieves anytime convergence rates.
result Gradient descent can achieve convergence rates of O ( T − 1.119 ) O(T^{-1.119}) O ( T − 1.119 ) for any stopping time T T T . Kaczmarz++ accelerates convergence for ill-conditioned systems.
problem Solving ill-conditioned linear systems efficiently.
method Adaptive momentum acceleration, Tikhonov-regularized projections, and memoization.
result Kaczmarz++ converges faster than Krylov methods on ill-conditioned systems.
New methods accelerate distributed optimization in noisy networks.
problem Optimizing distributed stochastic gradient methods for noisy, connected networks.
method Developed a framework for choosing stepsize and momentum parameters, proving acceleration and providing performance bounds.
result Distributed accelerated methods achieve acceleration with optimal complexity, reducing bias and variance.
New algorithms improve distributed optimization under specific conditions.
problem Distributed optimization problems with high communication costs.
method SVRS and AccSVRS algorithms combining gradient sliding and variance reduction.
result Achieved better communication complexity in distributed optimization.
In this paper, we propose a novel technique to implement stochastic gradient methods, which are beneficial for learning from large datasets, through accelerated stochastic dynamics. A stochastic gradient method is based on mini-batch learning for reducing the computational cost when the amount of data is large. The sto…
Accelerated SGD method converges linearly to local minima of nonconvex problems.
problem Nonconvex nonsmooth optimization problems.
method Combining variance reduction and Nesterov's extrapolation for accelerated SGD.
result Linear convergence to a stationary point of the nonconvex optimization problem.
FMCIT accelerates CI tests for causal discovery, maintaining power and efficiency.
problem High computational complexity in CI tests limits practical applicability of causal discovery methods.
method Flow Matching-based Conditional Independence Test (FMCIT) that leverages flow matching for fast CI tests.
result FMCIT effectively controls type-I error and maintains high testing power under the alternative hypothesis.
A new algorithm reduces the complexity of solving optimal transport problems.
problem Optimal transport problem with linear constraints.
method Primal-dual accelerated stochastic gradient descent with variance reduction (PDASGD).
result Achieves the best-known computational complexity of O ~ ( n 2 / ε ) \widetilde{\mathcal{O}}(n^2/ε) O ( n 2 / ε ) for OT problems. Novel methods for accelerating optimization in complex bilevel and minimax problems.
problem Optimization challenges in bilevel and minimax problems, especially when strong convexity assumptions are not met.
method Accelerated fully first-order methods for Bilevel Optimization (BLO) and Minimax Optimization (NCSC).
result State-of-the-art complexity for finding approximate second-order stationary points in BLO and NCSC.
RL accelerates portfolio optimization and option pricing by dynamically adjusting preconditioner sizes.
problem Large linear systems in portfolio optimization and option pricing lead to slow convergence.
method Reinforcement Learning (RL) dynamically adjusts block-preconditioner sizes to accelerate convergence.
result RL-driven solver significantly reduces computational cost and accelerates convergence.
New method speeds up diffusion models without requiring complex assumptions.
problem Slow sampling in diffusion models due to high computational cost.
method Training-free acceleration scheme under minimal assumptions.
result Provable acceleration within O ~ ( d 5 / 4 / ε ) \widetilde{O}(d^{5/4}/\sqrt{\varepsilon}) O ( d 5/4 / ε ) iterations. Improved SGD for non-strongly-convex regression with faster convergence.
problem Non-strongly-convex least squares regression problems.
method Modified accelerated gradient descent.
result Achieves optimal prediction error rates of O ( d / t ) O(d/t) O ( d / t ) and forgets initial conditions faster to O ( d / t 2 ) O(d/t^2) O ( d / t 2 ) . WSD uses a deterministic model to accelerate diffusion-based sampling.
problem Slow refinement process in diffusion models.
method Warm-start model that predicts an informed prior conditioned on input context.
result Significantly reduces the number of diffusion steps required for realistic samples.
ScaledGD accelerates ill-conditioned low-rank estimation.
problem Slow convergence of gradient descent in ill-conditioned problems.
method Scaled gradient descent (ScaledGD) with preconditioning.
result Linear convergence rate independent of condition number.
Paper proposes an efficient online Newton method with Nesterov's acceleration for streaming data.
problem Efficient inference of online Newton methods with robustness to noise and ill-conditioning.
method Online Newton method with Hessian averaging and Nesterov's accelerated sketch-and-project solver.
result Global almost-sure convergence and asymptotic normality of the last iterate with non-asymptotic convergence guarantees.
GD and NAG accelerate matrix factorization and neural networks.
problem Optimizing rectangular matrix factorization and linear neural networks.
method Gradient descent and Nesterov's accelerated gradient with specific initialization.
result NAG achieves the best-known iteration complexity for these problems.
Bayesian DOE accelerates experimental design with improved efficiency.
problem Enhancing experimental design efficiency and reliability.
method Bayesian framework, conditional density estimation, informative data selection.
result Significantly improved computational efficiency of experimental design.
Paper proposes an accelerated algorithm for sparse subspace clustering.
problem Inefficient and inaccurate subspace clustering methods.
method Accelerated orthogonal least-squares for sparse subspace clustering.
result The proposed method is more accurate and efficient than existing methods.
New method accelerates gradient optimization using curvature information.
problem Solving large-scale learning optimization problems efficiently.
method Curvature-aided gradient tracking step for incremental gradient optimization.
result Proves linear convergence rates for new methods.
Paper analyzes why LISTA accelerates sparse coding.
problem Sparse coding optimization with generic methods.
method Matrix factorization of Gram kernel for acceleration.
result Improved convergence for LISTA compared to non-adaptive methods.
ANADDH uses deep learning to improve volatility risk management.
problem Traditional Vega hedging strategies are inadequate for rapidly changing markets.
method Combines distributional reinforcement learning with adaptive Nesterov acceleration.
result Significant performance gains over existing hedging techniques.
New methods optimize functions on hyperbolic and spherical spaces, matching Euclidean rates up to logarithmic factors.
problem Optimizing functions on non-Euclidean spaces like hyperbolic and spherical geometries.
method Introduced accelerated global first-order methods for L L L -smooth and geodesically convex functions on hyperbolic and spherical spaces. result Achieved the same rates as accelerated gradient descent in Euclidean space, up to logarithmic factors.
New algorithm reduces feature count and accelerates error convergence.
problem Exponential error convergence in data classification with optimized random features.
method Optimized random features accelerated by quantum machine learning.
result Achieves exponential error convergence under low-noise condition.
Develops accelerated fixed-point methods with delayed oracles for scientific computing.
problem Approximating fixed points of nonexpansive operators.
method Combines Nesterov's acceleration and KM iteration with delayed inexact oracles.
result Establishes improved convergence rates for fixed-point approximation.
Study shows accelerated convergence of stochastic momentum methods in Wasserstein distances.
problem Performance sensitivity of momentum methods to noise in gradients.
method Stochastic momentum methods under a first-order stochastic oracle model.
result Linear convergence rates for AG and HB methods in Wasserstein metrics, robust to noise.
Two accelerated extragradient methods converge at O ( 1 / k ) O(1/k) O ( 1/ k ) rate for co-hypomonotone inclusions.
problem Solving co-hypomonotone inclusions with sum of Lipschitz and multivalued operators.
method Developed two Nesterov's accelerated extragradient methods for co-hypomonotone inclusions.
result Achieve O ( 1 / k ) \mathcal{O}(1/k) O ( 1/ k ) last-iterate convergence rates on the residual norm. Improved sampling guarantees for underdamped Langevin Monte Carlo without restrictive assumptions.
problem Sampling from unnormalized densities with improved guarantees and acceleration.
method Novel analysis relaxing assumptions on log-Sobolev inequality and Hessian smoothness, using Rényi discretization bounds.
result First KL divergence guarantees for ULMC without Hessian smoothness under strong log-concavity.