In this paper, we present the Bennett-type generalization bounds of the learning process for i.i.d. samples, and then show that the generalization bounds have a faster rate of convergence than the traditional results. In particular, we first develop two types of Bennett-type deviation inequality for the i.i.d. learning…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “faster convergence rate”
Estimates the maximal rate of convergence for Ricci flow solutions.
problem Understanding the maximal rate of convergence of Ricci flow solutions.
method Estimates the rate from above for solutions converging to solitons.
result Solutions converging faster than any fixed exponential rate must be self-similar.
LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
Estimates the rate of convergence of mean curvature flow solutions.
problem Understanding the convergence rate of mean curvature flow solutions.
method Estimates the upper bound of convergence rate to a limit self-similar solution.
result Solutions converging faster than any fixed exponential rate must be shrinkers themselves.
New algorithm speeds up Lasso computation by proving faster convergence.
problem Lasso estimator's slow convergence rate due to penalty.
method Homotopic approach using surrogate functions.
result Proves convergence rate for Lasso computation.
MSGD outperforms SGD in overparametrized settings with faster convergence rates.
problem Optimization of non-convex functions with momentum.
method Momentum Stochastic Gradient Descent (MSGD) with rigorous analysis.
result MSGD converges exponentially faster than SGD in overparametrized settings.
Riemannian stochastic gradient descent converges faster with increasing batch size.
problem Improving convergence rate of Riemannian stochastic gradient descent.
method Theoretical analysis and numerical investigation of increasing batch size effects.
result Riemannian stochastic gradient descent converges faster with increasing batch size.
Sampling without replacement speeds up optimization in minimax problems.
problem Optimizing minimax problems with faster convergence rates.
method Analysis of gradient descent ascent and proximal point method with two sampling strategies.
result Sampling without replacement leads to faster convergence rates in minimax optimization.
There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012], who showed that a random-coordinate selection rule achieves the same convergence rate as the Gauss-Southwell selection rule. This result su…
We extend the Fourier cosine method to discrete probability distributions, achieving faster convergence rates.
problem Extending Fourier cosine method to discrete probability distributions.
method Spectral filters and convergence rates analysis.
result Spectral filters achieve one order faster convergence rates than previously recognized.
We investigate the learning rate of multiple kernel learning (MKL) with and elastic-net regularizations. The elastic-net regularization is a composition of an -regularizer for inducing the sparsity and an -regularizer for controlling the smoothness. We focus on a sparse setting where the total …
Improves stochastic gradient methods for faster convergence.
problem Low asymptotic convergence of stochastic gradient methods in nonconvex optimization.
method Predictive Local Smoothness (PLS) method to adaptively adjust learning rates based on local smoothness predictions.
result New variants of SGD, AccSGD, and AMSGrad achieve faster linear convergence.
Faster algorithms for solving multichain MDPs under average-reward criterion.
problem Navigating towards the best connected component in multichain MDPs.
method Developed algorithms to better solve the navigational subproblem, achieving faster convergence rates.
result Improved rates of convergence and sharper complexity measures for multichain MDPs.
Paper proposes DSBA, a stochastic algorithm for decentralized learning that converges faster and uses sparse communication.
problem Efficient decentralized learning with sparse communication for complex problems.
method Generalizes decentralized optimization to monotone operator root finding, proposes DSBA algorithm.
result DSBA converges geometrically with a rate linearly depending on problem condition number and uses sparse communication.
We propose the stochastic average gradient (SAG) method for optimizing the sum of a finite number of smooth convex functions. Like stochastic gradient (SG) methods, the SAG method's iteration cost is independent of the number of terms in the sum. However, by incorporating a memory of previous gradient values the SAG me…
Paper analyzes faster convergence rates for reinforcement learning from offline data.
problem Analyzing faster convergence rates for reinforcement learning from offline data.
method Fine analysis of reinforcement learning from offline data, providing fast rates for regret convergence.
result The paper provides fast rates for the regret convergence, showing that the level of exponentiation depends on the noise in the decision-making problem.
Super-convergence allows neural nets to train faster with large learning rates.
problem Training neural networks too slowly.
method Training with large learning rates and one learning rate cycle.
result Neural networks can be trained an order of magnitude faster.
Improved SGD algorithm with faster convergence.
problem Optimization of machine learning models.
method Conditional accelerated lazy stochastic gradient descent.
result Convergence rate of $O\left(\frac{1}{\varepsilon^2}
ight)$, faster than previous methods.
New proof shows faster convergence rate for robust estimation with Lasso in adversarially contaminated outputs.
problem Robust estimation of parameters in the presence of adversarial output contamination.
method Extended Lasso with Huber loss function and penalty, focusing on specific properties of the Huber function.
result Same convergence rate as Dalalyan and Thompson (2019), but with a different proof.
Improved non-smooth optimization methods achieve faster convergence rates.
problem Non-smooth optimization problems, especially in and -SVM.
method Higher-order accelerated methods, leveraging recent advances in smooth convex optimization.
result Achieved iteration complexity for regression, breaking previous barriers.
VAV method optimizes learning rate for faster, stable SGD convergence.
problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable .
result VAV method achieves faster convergence and superior stability with larger learning rates.
New methods improve convergence rate of zeroth-order proximal stochastic algorithms.
problem Nonconvex nonsmooth optimization problems with infeasible gradients.
method ZO-ProxSVRG and ZO-ProxSAGA with variance reduction techniques.
result Convergence rate improved to from .
Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Ratesmath.OC
Improved SGD methods converge faster for nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Adaptive SGD with line-search and Polyak stepsizes.
result Unified convergence rates for various nonconvex functions.
This paper proves AdaGrad and Adam converge linearly under PL inequality.
problem Understanding the convergence of adaptive gradient methods.
method Unified approach proving AdaGrad and Adam converge linearly under PL inequality.
result AdaGrad and Adam converge linearly when the cost function is smooth and satisfies PL inequality.
Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errorsstat.ML
This paper shows faster convergence rates for stochastic gradient descent in binary classification.
problem Achieving faster convergence rates for stochastic gradient descent in binary classification.
method Stochastic gradient descent and averaging variant, focusing on exponential convergence rates under strong low-noise conditions.
result Exponential convergence of the expected classification error in the final phase of stochastic gradient descent and averaged stochastic gradient descent for differentiable convex loss functions.
pSCOPE improves convergence in distributed sparse learning.
problem Efficiently partitioning data for faster convergence in distributed sparse learning.
method pSCOPE is a cooperative local learning framework based on CALL.
result pSCOPE achieves linear convergence rate with good data partition.
SGDM accelerates faster than SGD with large batch sizes and permits broader learning rates.
problem Understanding the role of momentum in SGDM and its convergence rates.
method Analysis of SGDM convergence rates under strongly convex settings, including finite-sample rates and asymptotic normality of the averaged estimator.
result SGDM converges faster than SGD with large batch sizes and permits broader learning rates.
New SGDA method speeds up nonconvex minimax optimization.
problem Improving convergence of nonconvex minimax optimization.
method SGDA with random reshuffling for nonconvex-PŁ objectives.
result Convergence rates faster than with-replacement SGDA.
Paper connects loss functions and t-norms for faster deep learning convergence.
problem Improving convergence rates in deep learning models.
method Interprets loss functions through t-norms and generator functions.
result Derives a general relation between loss functions and t-norms leading to faster convergence.
We study Frank-Wolfe methods for nonconvex stochastic and finite-sum optimization problems. Frank-Wolfe methods (in the convex case) have gained tremendous recent interest in machine learning and optimization communities due to their projection-free property and their ability to exploit structured constraints. However,…
New algorithms solve convex-concave problems faster than previous methods.
problem Solving min-max problems without bilinear structure.
method Stochastic primal-dual algorithms with logarithmic dual updates.
result Faster convergence rates than for certain problems.
Proposes a neural density estimator that adapts to low-dimensional structures and integrates into generative models.
problem Challenges in implementing neural density estimators and lack of theoretical understanding.
method Structure-agnostic neural density estimator that is easy to implement and provably adaptive.
result Adapts to low-dimensional structures and achieves faster convergence rates.
The paper analyzes how larger minibatch sizes in SG-MCMC lead to faster convergence.
problem Theoretical analysis of impact of minibatch size on SG-MCMC convergence rate.
method Proposes a variance-reduction technique for SG-MCMC and proves its faster convergence rate.
result The proposed variance-reduction technique leads to a faster convergence rate than standard SG-MCMC.
Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Ratemath.OC
A new method for faster optimization in high dimensions.
problem Slow convergence in high-dimensional optimization problems.
method Subspace cubic regularized Newton method within Krylov subspace.
result Achieves a dimension-independent convergence rate of O(1/mk + 1/k^2).
SGD without replacement converges faster for smooth convex functions.
problem Improving convergence rate of SGD for smooth convex functions.
method Using method of exchangeable pairs to bound Wasserstein distance, we provide non-asymptotic results for SGD without replacement.
result SGD without replacement converges at a rate of O(1/K^2) for general smooth, strongly-convex functions.
Adaptive gradient methods converge faster with over-parameterization and line-search.
problem Training over-parameterized models using adaptive gradient methods.
method Simplified setting of smooth, convex losses with over-parameterized models, proving convergence rates and demonstrating improvements with line-search techniques.
result Adaptive gradient methods, particularly AMSGrad, converge faster with line-search techniques.
Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPscs.LG
TRPO adapts trust region methods for faster RL convergence.
problem Improving RL convergence rates in regularized MDPs.
method Adaptive scaling in TRPO for faster convergence rates.
result First RL result showing faster rates with regularization.
New method solves convex optimization faster than NAG.
problem Unconstrained smooth convex optimization problems.
method Accelerated quasi-Newton proximal extragradient (A-QPNE) method.
result Achieves a faster convergence rate of .
We investigate the learning rate of multiple kernel leaning (MKL) with elastic-net regularization, which consists of an -regularizer for inducing the sparsity and an -regularizer for controlling the smoothness. We focus on a sparse setting where the total number of kernels is large but the number of non…
FedLion improves Federated Learning by speeding up convergence and reducing communication costs.
problem Slow convergence and high communication costs in Federated Learning.
method Integrates Lion's adaptive approach into Federated Learning framework, using signed gradients.
result FedLion outperforms existing adaptive algorithms in convergence rate and communication efficiency.
AGDA and variance-reduced methods solve nonconvex-nonconcave minimax problems globally and faster.
problem Solving nonconvex-nonconcave minimax problems in machine learning.
method Global convergence of AGDA and variance-reduced algorithms.
result AGDA and variance-reduced methods achieve global convergence and faster rates.
learn2mix trains neural nets faster by adjusting class proportions dynamically.
problem Training neural nets efficiently with limited resources and imbalanced classes.
method Adaptive class proportion adjustment during training.
result Neural nets trained with learn2mix converge faster than static methods.
Improved private AdaGrad achieves faster convergence rates for convex functions.
problem Private empirical risk minimization with differential privacy.
method Noisy AdaGrad with knowledge of gradient subspace geometry.
result Faster convergence rates for convex functions, bypassing traditional bounds.
Paper speeds up IoT device detection and data decoding.
problem Efficiently detect and decode massive IoT devices in grant-free random access.
method Develops multi-armed bandit approaches for more efficient detection via coordinate descent.
result Proposed bandit based algorithms achieve faster convergence rates with lower time complexity.
Paper studies convergence rates from surrogate risk minimizers to Bayes optimal classifier.
problem Analyzing the convergence rates of surrogate risk minimizers to the Bayes optimal classifier.
method Introducing consistency intensity to characterize surrogate loss functions and using it to derive convergence rates.
result Empirical surrogate risk minimizers converge faster to the Bayes optimal classifier under certain conditions.
A new topology improves decentralized learning efficiency and accuracy.
problem Finding efficient decentralized learning topologies with fast consensus and low maximum degree.
method Proposed the Base- Graph topology for decentralized learning.
result The Base- Graph enables faster convergence and better communication efficiency than the exponential graph.
Improves understanding of stochastic NGVI convergence rates.
problem Lack of knowledge about non-asymptotic convergence rates in stochastic NGVI.
method Proved non-asymptotic convergence rates for conjugate likelihoods and showed implicit optimization for non-conjugate likelihoods.
result First non-asymptotic convergence rate for stochastic NGVI in conjugate likelihoods.
Proposes blockwise adaptive stepsize for faster training and better generalization in deep learning.
problem Widespread use of coordinate-wise adaptive methods like RMSprop and Adam leads to worse generalization than SGD.
method Splits network parameters into blocks and uses a blockwise adaptive stepsize, balancing adaptivity and generalization.
result Blockwise adaptive gradient descent converges faster and has lower generalization error than coordinate-wise adaptive methods.