PAGE optimizes nonconvex problems with optimal convergence rates.
problem Nonconvex optimization problems.
method PAGE algorithm for achieving optimal convergence rates.
result PAGE achieves optimal convergence rates for nonconvex optimization.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “optimal convergence rate”
The paper studies convergence rates of Tsallis entropic regularization in optimal transport.
problem Optimal transport with regularization.
method Γ-convergence and quantization/shadow arguments.
result Derives convergence rate of Tsallis entropic regularization.
Adaptive SGD with Line-Search and Polyak Stepsizes: Nonconvex Convergence and Accelerated Ratesmath.OC
Improved SGD methods converge faster for nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Adaptive SGD with line-search and Polyak stepsizes.
result Unified convergence rates for various nonconvex functions.
Averaged SGD achieves optimal convergence rate for neural networks in the NTK regime.
problem Convergence analysis of averaged stochastic gradient descent for neural networks.
method Analyzed convergence of averaged stochastic gradient descent for overparameterized two-layer neural networks.
result Achieved minimax optimal convergence rate with global convergence guarantee.
New factorial power constants improve optimization convergence rates.
problem Optimization convergence rates depend on various constants.
method Proposes using factorial powers for defining these constants.
result Factorial powers simplify or improve convergence rates of optimization methods.
Study on convergence rates for optimal transport with regularization.
problem Convergence analysis of divergence-regularized optimal transport.
method Novel methodology using quantization and martingale couplings.
result Sharp rates for various divergences and transport costs.
GAIL with neural networks converges to global optima and has a known rate.
problem Uncertainty about GAIL with neural networks achieving global optimality.
method Gradient-based alternating updates algorithm.
result Established sublinear convergence to globally optimal solution.
This work analyzes the convergence rate of unrolling for optimizing quadratic objectives.
problem The challenge of accurately computing Jacobians through optimization.
method Non-asymptotic convergence-rate analysis of unrolled differentiation for gradient descent and Chebyshev method.
result There is a trade-off between fast asymptotic convergence and immediate but slower convergence due to the learning rate.
Simpler, parameter-free AdaGrad and Adam variants with convergence guarantees.
problem Inefficiencies in ad-hoc learning rate tuning for optimization algorithms.
method Developed AdaGrad++ and Adam++ without predefined learning rates and proved their convergence.
result AdaGrad++ and Adam++ achieve comparable convergence rates to AdaGrad and Adam respectively.
The Sinkhorn-Knopp derivatives converge with linear rate.
problem Optimal transport problem with entropic regularization.
method Iterative proportional fitting procedure.
result Derivatives converge with linear rate.
The paper analyzes convergence rates of bilevel optimization algorithms and introduces a new stochastic algorithm.
problem Nonconvex-strongly-convex bilevel optimization problems in machine learning.
method Comprehensive convergence rate analysis for deterministic bilevel optimization using AID and ITD, and a novel stochastic algorithm stocBiO.
result Theoretical convergence rates for AID and ITD methods, and stocBiO's superior performance.
This paper analyzes OCBA algorithms' convergence rates for DEDS optimization.
problem Optimizing discrete-event dynamic systems with limited computing resources.
method Characterizes convergence rates of two OCBA algorithms under different performance measures.
result OCBA algorithms achieve optimal convergence rates under probability of correct selection and expected opportunity cost measures.
Kolmogorov-Arnold Networks achieve optimal convergence rates in nonparametric regression.
problem Nonparametric function approximation in multivariate settings.
method Structured additive and multiplicative KANs using B-splines.
result Achieve minimax-optimal convergence rate for Sobolev space functions.
Step decay schedules improve convergence in non-convex optimization.
problem Improving convergence in non-convex optimization problems.
method Analyzing convergence rates of step decay schedules in non-convex, convex, and strongly convex problems.
result Step decay schedules achieve convergence rates in various optimization scenarios.
The paper calculates how fast optimal investment strategies approach CRRA strategies in stochastic factor models.
problem Understanding convergence rates of optimal investment strategies in stochastic factor models.
method Analyzes optimal feedback functions in nonlinear and quadratic term structure models, considering decay of bond prices and power-like utility at high wealth levels.
result Convergence rates of optimal investment strategies to CRRA strategies are determined by bond price decay and power-like utility behavior.
VAV method optimizes learning rate for faster, stable SGD convergence.
problem Optimizing learning rate for efficient and stable machine learning models.
method Energy-based self-adaptive learning rate with auxiliary variable .
result VAV method achieves faster convergence and superior stability with larger learning rates.
CEI achieves convergence rates for constrained Bayesian optimization.
problem Constrained Bayesian optimization with theoretical convergence rates.
method Analyzing simple regret upper bound for CEI in RKHS and Gaussian process settings.
result CEI achieves convergence rates of and for squared exponential and Matérn kernels, respectively.
We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of \emph{consistency intensity} to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer…
Cubic-regularized Newton's method (CR) is a popular algorithm that guarantees to produce a second-order stationary solution for solving nonconvex optimization problems. However, existing understandings of the convergence rate of CR are conditioned on special types of geometrical properties of the objective function. In…
D-Adaptation automatically sets optimal learning rates without manual tuning.
problem Optimizing learning rates for efficient convergence in machine learning.
method D-Adaptation, which asymptotically achieves optimal learning rates without back-tracking or additional evaluations.
result D-Adaptation automatically matches hand-tuned learning rates across diverse problems.
Softmax policy gradient methods converge at rate with constants depending on problem and initialization.
problem Understanding convergence rates of softmax policy gradient methods in tabular settings.
method Analysis of softmax policy gradient and entropy regularized policy gradient methods, using Łojasiewicz inequality and lower bounds.
result Entropy regularization improves convergence rate from to .
Stochastic algorithm achieves sublinear convergence for bi-objective optimization.
problem Optimizing two conflicting functions using gradient or subgradient descent.
method Stochastic alternating algorithm with varying steps for each objective.
result Achieves sublinear convergence rate of O(1/T) under strong convexity.
Paper analyzes convergence rate of noisy Bayesian Optimization with Expected Improvement.
problem Theoretical convergence behaviors and rates of Expected Improvement (EI) in Bayesian optimization.
method Analyzes Expected Improvement (EI) under Gaussian process (GP) prior assumption, considering noisy observations.
result Established asymptotic error bound and rate for GP-EI with noisy observations.
Establishes geometric convergence of iterative optimization algorithms.
problem Analyzes convergence of iterative optimization algorithms under general assumptions.
method General framework for iterative optimization algorithms, proving asymptotic geometric convergence and providing convergence rates.
result Asymptotic geometric convergence of iterative optimization algorithms with exact rate.
Improved convergence rates for saddle-point optimization algorithms.
problem Understanding last-iterate convergence rates for saddle-point optimization algorithms in constrained settings.
method Expanding the understanding of last-iterate convergence for Optimistic Gradient Descent Ascent (OGDA) and Optimistic Multiplicative Weights Update (OMWU) in the constrained setting.
result Linear last-iterate convergence achieved with a universal constant learning rate for OMWU in bilinear games over the simplex.
Adam optimizer converges to zeros of a new vector field, not just gradient zeros.
problem Prove convergence rates for Adam optimizer in simple quadratic optimization problems.
method Introduced Adam vector field to analyze Adam optimizer's convergence.
result Established optimal convergence rates for Adam optimizer.
Optimizes convergence rate of stochastic proximal algorithms for composite convex problems.
problem Solving composite convex optimization problems with composite regularizers.
method Analyzed proximal stochastic gradient method and randomized incremental proximal method under relaxed variance assumptions.
result Proves convergence rate for last iterate of both algorithms under componentwise convexity and smoothness.
Making SGD Parameter-Freemath.OC
Develops a parameter-free SGD algorithm with optimal convergence rate.
problem Optimizing parameters in stochastic convex optimization.
method A novel parameter-free algorithm for SGD with high-probability guarantees and adaptive properties.
result Achieves optimal convergence rate with only a double-logarithmic factor increase compared to known-parameter settings.
MSGD outperforms SGD in overparametrized settings with faster convergence rates.
problem Optimization of non-convex functions with momentum.
method Momentum Stochastic Gradient Descent (MSGD) with rigorous analysis.
result MSGD converges exponentially faster than SGD in overparametrized settings.
Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projectionsmath.ST
The paper analyzes rates of convergence for optimal transport map estimators using barycentric projections.
problem Estimating optimal transport maps from data sampled according to two distributions.
method Comprehensive analysis of rates of convergence for plug-in estimators defined via barycentric projections.
result New stability estimate for barycentric projections under minimal smoothness assumptions.
We prove optimal bounds for the convergence rate of ordinal embedding (also known as non-metric multidimensional scaling) in the 1-dimensional case. The examples witnessing optimality of our bounds arise from a result in additive number theory on sets of integers with no three-term arithmetic progressions. We also carr…
Theory extends optimal learning rates without realizability assumption.
problem Agnostic binary classification without realizability assumption.
method Identifies tetrachotomy of optimal rates and combinatorial structures.
result Optimal universal rates for binary classification in agnostic setting.
Negative momentum accelerates convergence in minimax games but at a suboptimal rate.
problem The convergence rate of negative momentum in minimax games is suboptimal.
method Extending variational inequality formulation, connecting momentum method with Chebyshev polynomials.
result Negative momentum accelerates convergence locally but at a suboptimal rate.
The paper analyzes how hyperparameters affect SGD with momentum's convergence rate.
problem The role of hyperparameters in SGD with momentum's convergence rate.
method Theoretical analysis using a hyperparameters-dependent stochastic differential equation (hp-dependent SDE).
result The optimal linear rate of convergence depends on both the learning rate and the momentum coefficient.
New insights into optimizing Local SGD's outer optimizer for faster convergence.
problem Understanding the impact of outer optimizer and its hyperparameters in Local SGD.
method Analyzing convergence guarantees with new outer learning rates and momentum.
result Tuning the outer learning rate can improve convergence and handle inner learning rate ill-tuning.
Convergence rate of Bayesian tensor estimator: Optimal rate without restricted strong convexitystat.ML
In this paper, we investigate the statistical convergence rate of a Bayesian low-rank tensor estimator. Our problem setting is the regression problem where a tensor structure underlying the data is estimated. This problem setting occurs in many practical applications, such as collaborative filtering, multi-task learnin…
Paper improves a method for fast global and local convergence in optimization.
problem Slow global convergence in optimization methods with noisy Hessian estimates.
method Stochastic Newton Proximal Extragradient method using HPE framework.
result Faster global linear rate and superlinear convergence in fewer iterations.
Optimizing over-the-air convex optimization, analog schemes are nearly optimal at low SNR.
problem Optimizing over-the-air convex optimization with coded gradients.
method Analyzes coded gradients over an additive Gaussian noise channel, considers analog coding schemes.
result Analog coding schemes nearly match the optimal convergence rate at low SNR, but a slowdown is inevitable.
Improves understanding of stochastic NGVI convergence rates.
problem Lack of knowledge about non-asymptotic convergence rates in stochastic NGVI.
method Proved non-asymptotic convergence rates for conjugate likelihoods and showed implicit optimization for non-conjugate likelihoods.
result First non-asymptotic convergence rate for stochastic NGVI in conjugate likelihoods.
While classic work in convex-concave min-max optimization relies on average-iterate convergence results, the emergence of nonconvex applications such as training Generative Adversarial Networks has led to renewed interest in last-iterate convergence guarantees. Proving last-iterate convergence is challenging because ma…
New algorithm improves convergence for non-convex problems with boundaries.
problem Optimizing non-convex problems with constraints.
method Reflected Gradient Langevin Dynamics with probabilistic representation.
result Promising convergence rates, faster than existing methods.
Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Ratemath.OC
A new method for faster optimization in high dimensions.
problem Slow convergence in high-dimensional optimization problems.
method Subspace cubic regularized Newton method within Krylov subspace.
result Achieves a dimension-independent convergence rate of O(1/mk + 1/k^2).
Efficient global optimization is the problem of minimizing an unknown function f, using as few evaluations f(x) as possible. It can be considered as a continuum-armed bandit problem, with noiseless data and simple regret. Expected improvement is perhaps the most popular method for solving this problem; the algorithm pe…
New quasi-Newton method guarantees global superlinear convergence.
problem Global convergence and superlinear convergence of quasi-Newton methods.
method Hybrid proximal extragradient method with online learning for Hessian approximation.
result First globally convergent quasi-Newton method with explicit superlinear convergence rate.
Study shows convergence rates for BSDEs approximated by compound Poisson processes.
problem Analyzing convergence rates of BSDEs driven by Lévy processes.
method Approximating Lévy processes by compound Poisson processes and studying BSDEs.
result Optimal convergence rates derived for BSDEs in -norm and Wasserstein distance.
Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Timemath.OC
HF-opt uses Hamiltonian dynamics to optimize functions, achieving accelerated rates with randomized integration time.
problem Optimizing functions efficiently and accelerating convergence rates.
method Randomized Hamiltonian flow (RHF) with accelerated convergence rates.
result RHGD achieves accelerated convergence rates similar to Nesterov's AGD.
FastAdaBelief improves convergence rate of AdaBelief by exploiting strong convexity.
problem Improving convergence rate of AdaBelief without sacrificing generalization ability.
method Designing FastAdaBelief that adjusts step size considering strong convexity.
result Proves regret bound for FastAdaBelief.
Curvature-Dependant Global Convergence Rates for Optimization on Manifolds of Bounded Geometrymath.OC
Optimization rates improved for manifolds with bounded geometry.
problem Optimizing functions on manifolds with bounded geometry.
method Riemannian gradient descent and dynamic trivialization algorithm.
result Curvature-dependent convergence rates computed explicitly for common manifolds.