New method uses centripetal acceleration to improve GAN training.
problem Cyclic behaviors in GAN training.
method Simultaneous Centripetal Acceleration (SCA) and Alternating Centripetal Acceleration (ACA) methods.
result Gradient descent methods with SCA or ACA are linearly convergent for bilinear games.
Centripetal SGD prunes deep CNNs by making filters collapse.
problem Pruning deep CNNs with complex structures.
method Centripetal SGD, a novel optimization method.
result Pruning deep CNNs without performance loss.
Improves training GANs by escaping limit cycles.
problem Limit cycling behavior in training GANs.
method Predictive Centripetal Acceleration Algorithm (PCAA) combined with Adam.
result PCAA improves convergence rates and effectively trains GANs.
HitNet uses a Hit-or-Miss layer to enhance feature interpretability in capsule networks.
problem Difficulty in interpreting complex neural network architectures.
method Replacing the last layer with a Hit-or-Miss layer that trains capsules to hit or miss a target capsule using centripetal loss.
result HitNet achieves better performance than initial CapsNet on various datasets and provides interpretable feature representations.
Understanding how spatial configurations of economic activity emerge is important when formulating spatial planning and economic policy. A simple model was proposed by Simon, who assumed that firms grow at a rate proportional to their size, and that new divisions of firms with certain probabilities relocate to other fi…
Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
Maximal acceleration metrics limit spacetime curvature.
problem Bounding spacetime curvature under maximal acceleration.
method Developed a geometric framework for maximal acceleration metrics and associated connections, proving curvature bounds.
result Uniform bounds on curvature components follow from uniform bounds on maximal acceleration.
Accelerates Q-learning by updating targets more efficiently.
problem Improving convergence speed in Q-learning algorithms.
method Proposes an accelerated target update scheme based on momentum methods.
result The proposed algorithms converge faster than vanilla Q-learning.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
Super-acceleration of gradient descent with momentum improves loss function minimization.
problem Minimizing loss functions in machine learning.
method Extending Nesterov acceleration by using gradients at multiple steps ahead.
result Super-acceleration of the momentum algorithm is beneficial for various loss landscapes and tasks.
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
problem Locally accelerated CG requires knowledge of smoothness and strong convexity parameters.
method Parameter-Free Locally Accelerated CG (PF-LaCG) algorithm.
result PF-LaCG achieves local acceleration without requiring knowledge of smoothness and strong convexity parameters.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelera…
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.
Accelerates sampling from Gibbs distributions using ARWP method.
problem Sampling from Gibbs distributions efficiently.
method ARWP method, combining Nesterov acceleration and regularized Wasserstein proximal.
result ARWP exhibits higher contraction rate and faster tail exploration.
FedAc accelerates Federated Averaging for distributed optimization.
problem Efficiently optimizing distributed machine learning models.
method Federated Accelerated Stochastic Gradient Descent (FedAc) using a potential-based perturbed iterate analysis.
result FedAc achieves faster convergence and lower communication costs than previous methods.
This research accelerates sampling methods using Nesterov's Acceleration.
problem Improving sampling efficiency in MCMC methods.
method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.
result Quantified acceleration beyond underdamped Langevin in W2 distance for log-strongly-concave targets. Locally Accelerated Conditional Gradients improve convergence rates for smooth convex optimization problems.
problem Achieving optimal convergence rates for smooth convex optimization problems over polytopes.
method Locally Accelerated Conditional Gradients, coupling accelerated steps with conditional gradient steps.
result Achieves optimal accelerated local convergence for smooth strongly convex problems.
In this study, the concept of dual Lorentzian homotetic exponential motions in is discussed and their velocities, accelerations obtained. Also, some geometric results between velocity and acceleration vectors of a point in a spatial motion are obtained. Finally, the theorems related to acceleration and acceleration cen…
Improved analysis of accelerated noisy power method for PCA.
problem Inexact matrix-vector products in PCA settings.
method Improved analysis of Accelerated Noisy Power Method under milder perturbation conditions.
result Worst-case optimal convergence rate with relaxed noise conditions.
Variance reduction is a simple and effective technique that accelerates convex (or non-convex) stochastic optimization. Among existing variance reduction methods, SVRG and SAGA adopt unbiased gradient estimators and are the most popular variance reduction methods in recent years. Although various accelerated variants o…
New dynamical system framework explains Nesterov acceleration.
problem Understanding Nesterov's accelerated gradient method.
method Dynamical system derivation without vanishing step size.
result Acceleration arises from discretizing an ODE with semi-implicit Euler.
Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
problem Accelerating convex optimization
method Hamiltonian dynamics
result Hamiltonian dynamics-based algorithms achieve deterministic and accelerated convergence for convex optimization.
AGBM accelerates GBM with theoretical guarantees.
problem Accumulation of errors in GBM's momentum term.
method Incorporates Nesterov's acceleration techniques and a corrected pseudo residual.
result First GBM type with theoretically-justified accelerated convergence rate.
Accelerates MMLE using SVGD with Nesterov acceleration.
problem Maximum Marginal Likelihood Estimation optimization.
method Stein variational gradient descent with Nesterov acceleration.
result Consistently accelerates convergence across various tasks.
Many applications require that we learn the parameters of a model from data. EM is a method used to learn the parameters of probabilistic models for which the data for some of the variables in the models is either missing or hidden. There are instances in which this method is slow to converge. Therefore, several accele…
We propose a novel method to accelerate Lloyd's algorithm for K-Means clustering. Unlike previous acceleration approaches that reduce computational cost per iterations or improve initialization, our approach is focused on reducing the number of iterations required for convergence. This is achieved by treating the assig…
Symplectic discretization accelerates optimization of smooth convex functions.
problem Optimizing smooth convex functions efficiently.
method Discretizing Nesterov's and Polyak's methods using Euler and symplectic schemes.
result Symplectic discretization achieves accelerated optimization for smooth convex functions.
HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.
problem Optimizing functions efficiently and accelerating convergence rates.
method Randomized Hamiltonian flow (RHF) with accelerated convergence rates.
result RHGD achieves accelerated convergence rates similar to Nesterov's AGD.
AGNES accelerates gradient descent with noisy gradients.
problem Minimizing smooth convex and strongly convex functions with noisy gradients.
method Generalization of Nesterov's accelerated gradient descent algorithm for noisy conditions.
result AGNES achieves acceleration for noisy gradients with a constant of proportionality up to 1.
Accelerated Langevin algorithm improves MCMC sampling efficiency.
problem Improving the efficiency of MCMC sampling methods.
method Formulated gradient-based MCMC as optimization on probability measures, showing underdamped Langevin performs accelerated gradient descent.
result Accelerated rates can be achieved for nonconvex functions using the Langevin algorithm.
Accelerates deep RL algorithms with regularized Anderson acceleration.
problem Slow convergence and sample inefficiency in model-free, off-policy deep RL.
method Proposes a general acceleration method for deep RL by extending regularized Anderson acceleration.
result Improves both learning speed and final performance of deep RL algorithms.
Accelerated gradient method's stability deteriorates exponentially with steps.
problem Algorithmic stability of Nesterov's accelerated gradient method.
method Analysis of two notions of algorithmic stability for Nesterov's accelerated gradient method.
result Stability of Nesterov's accelerated method deteriorates exponentially with the number of gradient steps.
New method accelerates steepest descent for convex optimization.
problem Achieving acceleration for general ℓp smooth functions. method Primal-dual iterate sequences with differing norms, implicitly determined interpolation parameter.
result Improves iteration complexity to O(d1−p2) for ℓp norm smooth problems. New algorithm accelerates optimization on Riemannian manifolds, including Wasserstein space.
problem Accelerating optimization methods in Riemannian geometry.
method Dynamic stepsize algorithms on Riemannian manifolds with specific vector transport.
result First provable accelerated gradient method in Wasserstein space.
New method accelerates gradient descent on curved spaces.
problem Optimizing functions on curved Riemannian manifolds.
method Developed a novel geometric inequality to control metric distortion, enabling a Riemannian accelerated gradient method.
result Proposed the first global accelerated gradient method for Riemannian manifolds.
We present an accelerated algorithm for hierarchical density based clustering. Our new algorithm improves upon HDBSCAN*, which itself provided a significant qualitative improvement over the popular DBSCAN algorithm. The accelerated HDBSCAN* algorithm provides comparable performance to DBSCAN, while supporting variable …
DANCE optimizes neural network and accelerator design for faster, more efficient DNN execution.
problem Challenges in optimizing neural network and accelerator design for efficient DNN execution.
method Differentiable approach to co-exploration of accelerator and network architecture design.
result Significantly shorter time to achieve superior accuracy and hardware cost metrics.
Sinh-acceleration speeds up B-spline option pricing.
problem Improving efficiency in option pricing calculations.
method Using sinh-acceleration on B-spline probability density projection.
result SINH acceleration technique improves error control and reduces CPU time.
Unified framework for understanding and optimizing training acceleration.
problem Challenges in optimizing training with regularization and acceleration techniques.
method Explains how AdaGrad, RMSProp, and Adam accelerate training, and derives a generalization for L1-regularization. result Derives a unified mathematical framework for understanding and optimizing training acceleration.
We present theoretical results on the convergence of \emph{non-convex} accelerated gradient descent in matrix factorization models with ℓ2-norm loss. The purpose of this work is to study the effects of acceleration in non-convex settings, where provable convergence with acceleration should not be considered a \em…
Paper discovers structural dynamics equations from only acceleration data.
problem Discovering equations from only acceleration measurements in structural dynamics.
method Library-based approach with Approximate Bayesian Computation (ABC) prioritizing parsimonious models.
result Efficacy demonstrated in four structural dynamics examples, including linear and nonlinear systems.
Improved training of large-scale neural networks with reduced variance noise.
problem Training large-scale neural networks with high variance noise.
method Stochastic variance reduced Nesterov's Accelerated Quasi-Newton method (SVR-NAQ).
result Improved performance compared to conventional methods on benchmark problems.
Octagon map accelerates diagonal changes algorithm.
problem Improving the efficiency of diagonal changes algorithm.
method Octagon Farey map as an acceleration.
result Octagon map accelerates diagonal changes algorithm.
Regularized nonlinear acceleration (RNA) estimates the minimum of a function by post-processing iterates from an algorithm such as the gradient method. It can be seen as a regularized version of Anderson acceleration, a classical acceleration scheme from numerical analysis. The new scheme provably improves the rate of …