The extragradient method accelerates convergence in complex game dynamics.
problem Complex interactions in game dynamics cause simple methods to diverge, necessitating more sophisticated approaches.
method A polynomial-based analysis to identify three scenarios for accelerated convergence of the momentum extragradient method.
result The momentum extragradient method achieves faster convergence under specific eigenvalue conditions.
The extragradient method fails for hypomonotone variational inequalities.
problem The convergence of the extragradient method for hypomonotone variational inequalities.
method Application of the extragradient method to hypomonotone linear operators.
result The extragradient method diverges for hypomonotone variational inequalities.
This paper advances extragradient methods for solving inclusions under co-hypomonotonicity.
problem Solving inclusions with non-Lipschitz mappings.
method Unified and generalized extragradient methods under co-hypomonotonicity.
result Achieved O ( 1 / k ) \mathcal{O}(1/k) O ( 1/ k ) convergence rates for residual norms. Improved analysis of extragradient methods for structured VIPs.
problem Efficiently solving large-scale VIPs with weaker conditions.
method Single-call stochastic extragradient methods with expected residual condition.
result Convergence guarantees for quasi-strongly monotone and weak Minty VIPs.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
Paper improves a method for fast global and local convergence in optimization.
problem Slow global convergence in optimization methods with noisy Hessian estimates.
method Stochastic Newton Proximal Extragradient method using HPE framework.
result Faster global linear rate and superlinear convergence in fewer iterations.
Survey on extragradient methods for solving nonlinear equations and inclusions.
problem Approximating solutions of nonlinear equations and inclusions.
method Unified convergence analysis of extragradient and its variants.
result Sublinear convergence rates for different classes of algorithms.
Two accelerated extragradient methods converge at O ( 1 / k ) O(1/k) O ( 1/ k ) rate for co-hypomonotone inclusions.
problem Solving co-hypomonotone inclusions with sum of Lipschitz and multivalued operators.
method Developed two Nesterov's accelerated extragradient methods for co-hypomonotone inclusions.
result Achieve O ( 1 / k ) \mathcal{O}(1/k) O ( 1/ k ) last-iterate convergence rates on the residual norm. Paper develops fast method for computing optimal transport.
problem Efficient computation of optimal transport distance between distributions.
method Entropy-regularized extragradient method for first-order optimization.
result Achieves state-of-the-art runtime guarantees and good numerical performance.
Paper generalizes extragradient methods for solving equations and inclusions with improved convergence rates.
problem Solving equations and inclusions using extragradient methods.
method Unified and generalized extragradient methods for a broader class of algorithms, analyzing sublinear convergence rates.
result Unified and improved convergence results for various extragradient variants.
Paper proves Sion's theorem in geodesic spaces and develops a Riemannian extragradient method.
problem Understanding saddle points in nonconvex-nonconcave minimax problems.
method Geodesic metric space version of Sion's theorem and Riemannian extragradient method.
result Developed a Riemannian extragradient algorithm for smooth minimax problems.
Improved convergence for VIPs with SEG-RR, a variant of SEG with random reshuffling.
problem Solving variational inequality problems (VIPs) in machine learning.
method Stochastic Extragradient with Random Reshuffling (SEG-RR).
result SEG-RR achieves faster convergence rates than with-replacement variants for certain VIP classes.
Improved SEG method converges to Nash equilibrium in bilinear games.
problem Stochastic bilinear minimax optimization problem
method Stochastic ExtraGradient (SEG) method with constant step size, iteration averaging, and scheduled restarting.
result Provable convergence to Nash equilibrium under standard settings, optimal convergence rate in interpolation setting.
New method solves nonsmooth low-rank matrix optimization problems efficiently.
problem Nonsmooth and low-rank matrix optimization problems in statistics and machine learning.
method Low-rank Extragradient Method with warm-start initialization.
result The extragradient method converges to an optimal solution with rate O ( 1 / t ) O(1/t) O ( 1/ t ) and requires only two low-rank SVDs per iteration. We study the effect of the stochastic gradient noise on the training of generative adversarial networks (GANs) and show that it can prevent the convergence of standard game optimization methods, while the batch version converges. We address this issue with a novel stochastic variance-reduced extragradient (SVRE) optimi…
Last iterate of Extragradient algorithm converges slower than averaged iterates in saddle point problems.
problem Smooth convex-concave saddle point problems
method Analysis of Extragradient (EG) algorithm convergence rates
result The last iterate of EG converges at a rate of O(1/√T), compared to O(1/T) for averaged iterates
Paper proposes a quasi-Newton method for nonlinear equations with global convergence guarantees.
problem Solving smooth and monotone nonlinear equations efficiently and globally.
method Hybrid proximal extragradient framework combined with online learning for Jacobian approximation.
result First global convergence results showing quasi-Newton method's advantage over extragradient method.
New method solves convex optimization faster than NAG.
problem Unconstrained smooth convex optimization problems.
method Accelerated quasi-Newton proximal extragradient (A-QPNE) method.
result Achieves a faster convergence rate of O ( min { 1 k 2 , d log k k 2.5 } ) {O}\bigl(\min\{\frac{1}{k^2}, \frac{\sqrt{d\log k}}{k^{2.5}}\}\bigr) O ( min { k 2 1 , k 2.5 d l o g k } ) . In this paper we propose a primal-dual proximal extragradient algorithm to solve the generalized Dantzig selector (GDS) estimation problem, based on a new convex-concave saddle-point (SP) reformulation. Our new formulation makes it possible to adopt recent developments in saddle-point optimization, to achieve the optim…
ARBITER learns SPX-VIX term structures without arbitrage constraints.
problem Arbitrage-free modeling of SPX-VIX term structures.
method Risk-neutral neural operator mapping market states to operator outputs enforcing static arbitrage constraints.
result ARBITER outperforms other models in derivatives term structure evaluation metrics.
Riemannian algorithms converge at Euclidean rates for geodesically convex-concave problems.
problem Min-max optimization on Riemannian manifolds.
method RCEG method and RGDA for geodesically strongly-convex-concave problems.
result RCEG achieves linear convergence rate in geodesically strongly-convex-concave cases.
In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processi…
New algorithm solves structured nonconvex-nonconcave min-max problems.
problem Min-max optimization challenges in deep learning.
method Generalized extragradient algorithm for structured nonconvex-nonconcave problems.
result Algorithm converges to stationary points in Euclidean and ℓ p \ell_p ℓ p spaces. We consider a wide range of regularized stochastic minimization problems with two regularization terms, one of which is composed with a linear function. This optimization model abstracts a number of important applications in artificial intelligence and machine learning, such as fused Lasso, fused logistic regression, a…
New algorithms reduce variance in solving complex mathematical problems.
problem Solving convex-concave saddle point problems, variational inequalities, and inclusions.
method Stochastic variance reduction for extragradient, forward-backward-forward, and forward-reflected-backward methods.
result All proposed methods converge with complexities matching or improving deterministic counterparts.
Data-driven modeling increasingly requires to find a Nash equilibrium in multi-player games, e.g. when training GANs. In this paper, we analyse a new extra-gradient method for Nash equilibrium finding, that performs gradient extrapolations and updates on a random subset of players at each iteration. This approach prova…
Saddle-point optimization problems are an important class of optimization problems with applications to game theory, multi-agent reinforcement learning and machine learning. A majority of the rich literature available for saddle-point optimization has focused on the offline setting. In this paper, we study nonstationar…
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobians of standard gradient dynamics in the family. Shapes restricted to the real line represent well-understood classes of problems, like minimi…
This paper analyzes OGDA and EG methods for nonconvex minimax problems.
problem Theoretical guarantees of OGDA and EG methods in nonconvex settings.
method Unified analysis through single-call extra-gradient methods.
result Established convergence of OGDA and EG methods under NC-SC and NC-C settings.
Two new algorithms solve privacy-constrained SVI and SSP problems.
problem Privacy-constrained stochastic variational inequality and saddle-point problems.
method Proposed Noisy Stochastic Extragradient (NSEG) and Noisy Inexact Stochastic Proximal Point (NISPP) algorithms.
result Optimal risk bounds for weak gap function with sampling with replacement.
We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using variants of the gradient method (GD). Prime examples are extragradient (EG), the optimistic gradient method (OG) and consensus optimization (CO), which enjoy linear convergence in cas…
Optimal transportation, or computing the Wasserstein or ``earth mover's'' distance between two distributions, is a fundamental primitive which arises in many learning and statistical settings. We give an algorithm which solves this problem to additive ε ε ε with O ~ ( 1 / ε ) \tilde{O}(1/ε) O ~ ( 1/ ε ) parallel depth, and $\tilde{O}\left(n^2/ε\…
A new algorithm improves GAN training stability and performance.
problem Training GANs is difficult due to variance and rotational dynamics.
method Proposes Lookahead-Minmax algorithm for minmax optimization.
result Significant improvement in GAN performance and stability.
New quasi-Newton method guarantees global superlinear convergence.
problem Global convergence and superlinear convergence of quasi-Newton methods.
method Hybrid proximal extragradient method with online learning for Hessian approximation.
result First globally convergent quasi-Newton method with explicit superlinear convergence rate.
New insights into stochastic methods for solving variational inequalities.
problem Understanding convergence behaviors of stochastic algorithms in variational inequalities.
method Re-casting SEG/SGDA as Markov Chains to analyze their probabilistic structures.
result The average iterate is asymptotically normal with a unique invariant distribution for various VIPs.
A new method uses extrapolation to train deep learning models with larger batch sizes.
problem Persistent degradation in performance when increasing batch size for deep learning.
method Proposes extrapolation to stabilize optimization trajectory and improve generalization.
result Demonstrates scaling to larger batch sizes while maintaining or surpassing state-of-the-art accuracy.
New tensor recovery method improves efficiency under strict complementarity.
problem Efficiently recovering low-rank tensors using tensor nuclear norm.
method Developed strict complementarity condition for tensor nuclear norm ball and applied to gradient methods.
result Standard gradient methods achieve linear convergence and nearly linear runtime under strict complementarity.