Paper accelerates SAGA using sampled negative momentum.
problem Directly accelerating SAGA for optimization problems.
method Proposes SSNM for direct acceleration of SAGA.
result Achieves best known oracle complexity for strongly convex problems.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “accelerated variant”
ADASS selects adaptive subsets for SGD training acceleration.
problem Fixed sample size in SGD limits training efficiency.
method ADASS selects adaptive subsets based on Lipschitz constants.
result ADASS achieves comparable accuracy with full training set.
Paper analyzes and accelerates Langevin Monte Carlo methods using large deviations theory.
problem High-dimensional sampling problems in machine learning.
method Unified approach using large deviations theory to study and accelerate Langevin dynamics variants.
result Efficiency of Langevin dynamics variants demonstrated through numerical experiments.
A Continuized View on Nesterov Acceleration for Stochastic Gradient Descent and Randomized Gossipmath.OC
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
Extragradient-Type Methods with Last-Iterate Convergence Rates for Co-Hypomonotone Inclusionsmath.OC
Two accelerated extragradient methods converge at rate for co-hypomonotone inclusions.
problem Solving co-hypomonotone inclusions with sum of Lipschitz and multivalued operators.
method Developed two Nesterov's accelerated extragradient methods for co-hypomonotone inclusions.
result Achieve last-iterate convergence rates on the residual norm.
Improved quasi-Newton method for convex optimization with linear and accelerated convergence.
problem Efficiently solving composite optimization problems with strong convexity.
method Proximal quasi-Newton algorithm with accelerated variant.
result Acceleration may not improve convergence in quasi-Newton setting.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
New algorithms solve monotone inclusions and convex-concave minimax problems.
problem Solving maximally monotone equations and inclusions.
method Developed new accelerated algorithms based on Halpern-type fixed-point iteration and Popov's past extra-gradient method.
result Achieved convergence rates for various problems.
Accelerated Block Coordinate Proximal Gradients with Applications in High Dimensional Statisticsmath.OC
New method accelerates optimization for nonconvex problems in statistics.
problem Nonconvex optimization problems in statistics and machine learning.
method Adaptive momentum and block coordinate proximal gradient method with specific update rules.
result The method achieves provable local linear convergence in sparse linear regression problems.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
New AMSGrad variant accelerates nonconvex optimization.
problem Training deep neural networks efficiently.
method Adds prior knowledge to AMSGrad, leveraging sequential predictability and optimistic online learning.
result Accelerates convergence and increases sample efficiency.
New algorithms find near-stationary points in convex optimization.
problem Finding near-stationary points in convex optimization.
method Memory-saving variant of OGM-G, accelerated SVRG, adaptively regularized accelerated SVRG.
result Schemes achieve fast rates for minimizing gradient norm and function value.
New methods accelerate distributed optimization in noisy networks.
problem Optimizing distributed stochastic gradient methods for noisy, connected networks.
method Developed a framework for choosing stepsize and momentum parameters, proving acceleration and providing performance bounds.
result Distributed accelerated methods achieve acceleration with optimal complexity, reducing bias and variance.
A new simple algorithm reduces variance for fast convergence.
problem Improving convergence rates for stochastic variance reduced algorithms.
method Introducing a simple stochastic variance reduced algorithm (MiG) with fast convergence rates.
result MiG achieves best-known convergence rates for both strongly and non-strongly convex problems.
Improved algorithms for optimal transport with better complexity bounds.
problem Efficiency of algorithms for solving optimal transport problems.
method Greenkhorn, APDAMD, and accelerated Sinkhorn algorithms with improved complexity bounds.
result Improved complexity bounds for Greenkhorn, APDAMD, and accelerated Sinkhorn algorithms.
Improved IHT with momentum accelerates convex optimization with non-convex constraints.
problem Optimizing convex criteria with non-convex constraints.
method Modified iterative hard thresholding with momentum.
result Acceleration leads to significant improvements over state-of-the-art methods.
Two new differentially private optimization algorithms derived from accelerated methods.
problem Improving privacy in optimization algorithms while maintaining convergence rates.
method Polyak's heavy ball method and Nesterov's accelerated gradient method with differential privacy.
result The proposed algorithms outperform existing differentially private optimization methods.
Catalyst accelerates convex optimization methods with practical guidelines.
problem Improving convergence of gradient-based optimization methods.
method Catalyst scheme based on inexact accelerated proximal point algorithm.
result Faster convergence rates for various convex optimization methods.
Survey of Graph Neural Networks for efficient computation.
problem Efficient processing of Graph Neural Networks (GNNs) is challenging.
method Review of GNN algorithms, software and hardware acceleration analysis.
result Distilled hardware-software, graph-aware, and communication-centric vision for GNN accelerators.
Momentum accelerates Frank Wolfe algorithms on certain problems.
problem Improving convergence rate of Frank Wolfe algorithms.
method Introducing momentum into Frank Wolfe algorithms and proving faster convergence rate.
result Accelerated Frank Wolfe (AFW) converges with a faster rate of .
ASBART accelerates Soft BART for faster Bayesian regression.
problem Slow computation in Soft BART.
method Proposed ASBART, a variant of Soft BART.
result ASBART is about 10 times faster than Soft BART with similar accuracy.
New SARAH variant MB-SARAH-RBB accelerates mini-batch optimization.
problem Improving the performance of SARAH in mini-batch settings.
method Introduced MB-SARAH-RBB, a variant of SARAH using RBB step size calculation.
result MB-SARAH-RBB converges linearly for strongly convex objectives and outperforms existing methods.
Study of accelerated dynamics for convex function minimization with noisy gradients.
problem Minimizing smooth convex functions with noisy gradients.
method Formulate and study continuous-time stochastic dynamics, prove convergence rates.
result Derive estimates of convergence rates for function values, both persistent and asymptotic.
New algorithms accelerate SVGD convergence using deep unfolding.
problem Improving the speed of SVGD convergence.
method Integrating deep unfolding into SVGD for parameter learning.
result Proposed algorithms achieve faster convergence in various tasks.
A Class of Accelerated Fixed-Point-Based Methods with Delayed Inexact Oracles and Its Applicationsmath.OC
Develops accelerated fixed-point methods with delayed oracles for scientific computing.
problem Approximating fixed points of nonexpansive operators.
method Combines Nesterov's acceleration and KM iteration with delayed inexact oracles.
result Establishes improved convergence rates for fixed-point approximation.
Survey on extragradient methods for solving nonlinear equations and inclusions.
problem Approximating solutions of nonlinear equations and inclusions.
method Unified convergence analysis of extragradient and its variants.
result Sublinear convergence rates for different classes of algorithms.
A budgeted experiment design method for causal structure learning with improved efficiency.
problem Learning causal structure with limited experiments.
method Formulated as an optimization problem, solved using a greedy algorithm with submodularity and accelerated variants.
result Achieves $(1- rac{1}{e})$-approximation of optimal value and significantly reduces the number of interventions.
The paper connects dynamical systems to ADMM for solving nonsmooth constrained problems.
problem Solving nonsmooth and constrained optimization problems.
method Developed differential inclusions for accelerated ADMM variants and analyzed their convergence rates.
result Derived rates of convergence for the dynamical systems under different settings, showing tradeoffs between damping strategies.
Paper proposes an accelerated algorithm for sparse subspace clustering.
problem Inefficient and inaccurate subspace clustering methods.
method Accelerated orthogonal least-squares for sparse subspace clustering.
result The proposed method is more accurate and efficient than existing methods.
New algorithms accelerate MAP inference in Markov fields with faster convergence.
problem Finding the most likely configuration in discrete-valued Markov random fields.
method Entropy-regularized linear programming with accelerated gradient methods.
result Accelerated algorithms find optimal solutions faster, especially when the LP is tight.
A new accelerated method with simpler momentum update rules.
problem Optimizing parameters in machine learning models.
method Proposes a novel accelerated stochastic gradient method with simpler momentum update rules.
result The method outperforms Sgdm and Adam in practical problems.
Accelerated optimization methods improve robustness and privacy in estimation.
problem Improving robustness and privacy in estimation methods.
method Accelerated gradient methods based on Frank-Wolfe and projected gradient descent, with tailored learning rates and Nesterov's momentum.
result Reduction in iteration complexity, leading to stronger statistical guarantees.
Enhances optimization algorithms to achieve faster convergence rates.
problem Minimizing smooth convex functions efficiently.
method Extends optimistic learning to achieve faster convergence rates.
result Achieves a rate of for optimization.
AIDE improves distributed optimization efficiency and matches communication lower bounds.
problem Efficient distributed optimization with reduced communication costs.
method AIDE: an accelerated variant of an inexact DANE algorithm.
result AIDE matches communication complexity lower bounds and outperforms other methods.
Accelerates deep neural network training with a generalized BN approach.
problem Conventional Batch Normalization (BN) struggles with convergence speed and error rate.
method Introduces Generalized Batch Normalization (GBN) using alternative deviation measures and statistics.
result GBN accelerates training and often improves error rate compared to conventional BN.
A fast method for training linear classifiers maximizes margins.
problem Training linear classifiers with maximum margins.
method Momentum-based gradient method derived from convex dual with Nesterov acceleration.
result Exponentially faster convergence rate compared to standard methods.
Unified view of stochastic optimization methods with improved convergence and robustness.
problem Stochastic convex composite optimization with noise.
method Estimate sequence approach, accelerated algorithms, robust strategies.
result Optimal complexity accelerated SVRG algorithm robust to noise.
Unified analysis of Federated Averaging and Nesterov FedAvg for linear speedup.
problem Understanding convergence of FL algorithms under non-i.i.d. data and partial participation.
method Systematic study of convergence guarantees for FedAvg and Nesterov FedAvg under different conditions.
result Unified analysis of linear speedup for FedAvg and Nesterov FedAvg in various settings.
The Dynamic ECME Algorithmstat.CO
The ECME algorithm has proven to be an effective way of accelerating the EM algorithm for many problems. Recognising the limitation of using prefixed acceleration subspace in ECME, we propose the new Dynamic ECME (DECME) algorithm which allows the acceleration subspace to be chosen dynamically. Our investigation of an …
Explains gradient descent methods and their convergence, focusing on simple analysis.
problem Understanding and analyzing gradient descent methods and their variants.
method Elementary mathematical analysis focusing on structures and assumptions of objective functions.
result Unified convergence analysis of various gradient descent methods and variants.
Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problemsmath.OC
Paper develops accelerated APCD for nonconvex nonsmooth problems with performance guarantees.
problem Efficient methods for nonconvex nonsmooth optimization problems with performance guarantees.
method Asynchronous Accelerated Proximal Coordinate Descent (AAPCD) for nonsmooth and nonconvex problems.
result AAPCD ensures that every limit point is a critical point and achieves linear and sublinear convergence rates.
A new optimizer combines Newton and ADMM for faster classification.
problem Slower convergence of first-order methods in distributed learning.
method Integrates GPU-accelerated Newton solver with ADMM for multiclass classification.
result Significantly reduces the time to solution and generalization error.
Improved k-means algorithm runs up to 3 times faster.
problem Efficiently clustering data points in k-means algorithm.
method Novel accelerated exact k-means algorithm with improved distance bounds.
result 3 times faster performance in 18 of 22 experiments.
A simple power iteration with momentum achieves optimal PCA in stochastic settings.
problem Accelerating PCA in the stochastic setting with limited data.
method A simple variant of the power iteration with momentum.
result Achieves optimal sample and iteration complexity of .
Develops an accelerated algorithm for solving nonmonotone generalized equations.
problem Solving nonmonotone generalized equations with possibly non-accelerated schemes.
method Combines Nesterov's acceleration and variance-reduction techniques for a class of generalized equations.
result Achieves convergence rates, improving upon non-accelerated counterparts.
Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimizationmath.OC
Unified analysis of momentum methods for deep learning.
problem Convergence analysis of stochastic momentum methods for convex and non-convex optimization.
method Developed a convergence analysis for two stochastic momentum methods.
result Unified framework revealing similarities and differences between methods.
This paper advances extragradient methods for solving inclusions under co-hypomonotonicity.
problem Solving inclusions with non-Lipschitz mappings.
method Unified and generalized extragradient methods under co-hypomonotonicity.
result Achieved convergence rates for residual norms.
Faster WIND accelerates iterative BOND for LLM alignment.
problem Iterative BOND is inefficient in practice due to sample and computation inefficiency.
method Unified game-theoretic connection to self-play alignment, WIND framework with efficient algorithms.
result WIND variant achieves superior sample efficiency and faster computation.