Faster rates achieved for computing equilibria in convex-concave games.
problem Computing equilibria in convex-concave games efficiently.
method Optimistic prediction algorithms and no-regret techniques.
result Achieved O(1/T2) rate for equilibrium computation. OMWU shows last iterate convergence in convex-concave games.
problem Optimizing in constrained min-max optimization landscapes.
method OMWU (Optimistic Multiplicative-Weights Update) in the no-regret online learning framework.
result OMWU exhibits last iterate convergence for convex-concave games, generalizing previous results.
Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.
problem Understanding dynamics of zero-sum games with hidden structure.
method Gradient Descent Ascent applied to hidden zero-sum games with specific convex-concave structure.
result Gradient Descent Ascent converges to von-Neumann solution in strictly convex-concave hidden games.
APAC-Net solves high-dimensional stochastic MFGs using neural networks.
problem High-dimensional stochastic mean-field games.
method Alternating population and control neural networks, parameterizing value and density functions.
result Solves up to 100-dimensional MFG problems.
Gradient methods converge exponentially in concave network games.
problem Finding Nash equilibria in concave network zero-sum games.
method Gradient Ascent and Optimistic Gradient Ascent analyses.
result Exponential convergence rates in various game settings.
New algorithm AG-OG optimizes separable convex-concave problems efficiently.
problem Efficiently solving separable convex-concave minimax optimization problems.
method Leverages Nesterov acceleration and optimistic gradient on component and coupling parts of the problem.
result Achieves optimal convergence rate for various settings including bilinearly coupled problems.
Improved Frank-Wolfe algorithm solves saddle point problems efficiently.
problem Solving constrained smooth convex-concave saddle point problems.
method Extends Frank-Wolfe algorithm to use linear minimization oracles.
result First proof of convergence for FW-type saddle point solver over polytopes.
This work provides lower bounds for differentiable games and defines a new condition number.
problem Understanding the fundamental limits of convergence in differentiable games.
method The authors cast saddle-point and min-max problems as 2-player games and use tools from single-objective convex optimization to derive linear lower bounds for convex-concave games. They also introduce a new condition number for games.
result The authors provide linear lower bounds for differentiable games, including n-player games, and introduce a new condition number that captures the possibility of linear rates in games without strong convexity or concavity. This work finds mixed equilibria in machine learning problems using measures and simultaneous gradient ascent-descent.
problem Finding pure equilibria in machine learning problems is computationally hard.
method Entropic regularization, simultaneous gradient ascent-descent, and particle discretization in the Wasserstein metric.
result Global convergence towards the global equilibrium in mixed equilibria problems.
New algorithm solves non-convex, non-differentiable min-max games.
problem Limited theoretical understanding of non-smooth min-max games.
method Proximal gradient descent-ascent algorithm for convex-strongly convex games.
result Algorithm converges to ε-Nash equilibrium with polynomial gradient evaluations.
The paper explains how simple methods can converge to optimal solutions in complex neural games.
problem Finding optimal solutions in neural games with non-convex objectives.
method Theoretical framework using hidden convexity and overparameterization, with path-length bounds and PŁ conditions.
result Simple gradient methods can converge to Nash equilibria in non-convex min-max games under certain conditions.
New algorithm solves min-max optimization problems in a decentralized manner.
problem Solving min-max saddle point games in a decentralized and adaptive manner.
method Developed a decentralized adaptive momentum (DADAM3) algorithm for min-max optimization. result DADAM3 achieves non-asymptotic rates of convergence for finding Nash equilibrium points. This paper analyzes saddle points and minimax points in non-convex smooth games.
problem Understanding local optimal points in non-convex smooth games.
method Comprehensive analysis of local minimax points, including their optimality conditions and stability.
result Local saddle points are uniformly local minimax points under mild continuity assumptions.
New algorithm improves self-play reinforcement learning for competitive games.
problem Inefficient opponent selection in self-play reinforcement learning.
method Intelligently selects opponents based on adversarial rules derived from saddle point optimization.
result Algorithm converges to approximate equilibrium with high probability in convex-concave games.
Paper solves non-convex min-max games using gradient descent-ascent.
problem Solving saddle point games in non-convex settings.
method Iterative gradient descent-ascent algorithms for both players.
result First order stationary points found efficiently.
Improved FTPL algorithm reduces regret in predictable minimax games.
problem Online learning and minimax games with predictable loss sequences.
method Optimistic modification of FTPL with dual regularization view.
result Tighter regret bounds for predictable sequences, O(T−1/2) accuracy. New algorithms solve nonconvex-nonconcave minimax optimization problems.
problem Solving minimax optimization problems in machine learning.
method Two novel Newton-type algorithms for nonconvex-nonconcave minimax optimization.
result Proved local convergence at strict local minimax points.
Increasing iterate averaging improves convergence rates for saddle-point problems.
problem Solving saddle-point problems efficiently.
method Increasing iterate averaging schemes applied to various first-order methods.
result Increasing iterate averaging preserves the O(1/T) convergence rate with no additional assumptions or overhead. We define a class of L-convex-concave subsets of RPn, where L is a projective subspace of dimension l in RPn. These are sets whose sections by any (l+1)-dimensional space L' containing L are convex and concavely depend on L'. We introduce an L-duality for these sets, and prove that the L-dual to an L-…
New algorithms reduce variance in solving complex mathematical problems.
problem Solving convex-concave saddle point problems, variational inequalities, and inclusions.
method Stochastic variance reduction for extragradient, forward-backward-forward, and forward-reflected-backward methods.
result All proposed methods converge with complexities matching or improving deterministic counterparts.
New saddle network architectures preserve convex-concave geometry in optimization problems.
problem Optimization models with convex x and concave y components.
method Structured separable decomposition and saddle network architectures.
result Proven one-dimensional approximation theorem and high accuracy on various test functions.
Last iterate of Extragradient algorithm converges slower than averaged iterates in saddle point problems.
problem Smooth convex-concave saddle point problems
method Analysis of Extragradient (EG) algorithm convergence rates
result The last iterate of EG converges at a rate of O(1/√T), compared to O(1/T) for averaged iterates
PURE-CD algorithm proves complexity bounds for convex-concave problems.
problem Solving convex-concave min-max problems with bilinear coupling.
method Primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD).
result Complexity bounds match or improve existing results for dense and sparse problems.
ICCNLS models complex relationships as convex and concave components.
problem Complex input-output relationships with affine ambiguity.
method Sub-gradient constrained affine functions, global orthogonality constraints, L1, L2, and elastic net regularisation.
result Improved predictive accuracy and model simplicity compared to conventional methods.
Optimistic mirror descent improves convergence in saddle-point problems.
problem Training generative adversarial networks (GANs) with saddle-point problems.
method Analyzed mirror descent (MD) and optimistic mirror descent (OMD) in coherent non-monotone problems.
result Optimistic mirror descent converges in all coherent problems, improving upon vanilla MD.
Paper introduces ℓ-DER for regression tasks using morphological operators and convex-concave procedure.
problem Developing a universal approximator for regression tasks.
method Introduces ℓ-DER model, trains it using a convex-concave procedure (CCP) to minimize least-squares. result Outperforms other hybrid morphological models and state-of-the-art approaches.
This work analyzes and improves stochastic gradient methods for GAN training.
problem Understanding the training dynamics of GANs, particularly their convergence.
method Continuous-time analysis using differential equations, focusing on simGD and its variants.
result The methods converge under different assumptions, providing new insights into GAN training.
New method finds arbitrage opportunities in fluctuating asset bands.
problem Finding arbitrage opportunities in fluctuating asset bands.
method Formulate as maximizing volatility within a price band, using convex-concave optimization.
result Approximately solves non-convex optimization problem for moving-band arbitrage.
EMA outperforms MA in GAN training, reducing cycle amplitudes and improving stability.
problem Improving GAN training stability and effectiveness.
method Comparison of Moving Average (MA) and Exponential Moving Average (EMA) techniques.
result EMA converges to limit cycles with vanishing amplitude in simple bilinear games and enhances GAN training stability.
As has been observed by Morse \cite{Mo}, any generic vector field v on a compact smooth manifold X with boundary gives rise to a stratification of the boundary $\d X$ by compact submanifolds $\{\d_j^\pm X(v)\}_{1 \leq j \leq \dim(X)}$, where $\textup{codim}(\d_j^\pm X(v))= j$. Our main observation is that this stra…
Riemannian algorithms converge at Euclidean rates for geodesically convex-concave problems.
problem Min-max optimization on Riemannian manifolds.
method RCEG method and RGDA for geodesically strongly-convex-concave problems.
result RCEG achieves linear convergence rate in geodesically strongly-convex-concave cases.
Gradient method achieves linear convergence for saddle point problems without strong convexity.
problem Solving saddle point problems with non-strongly convex functions.
method Primal-dual gradient method with a novel analysis technique.
result Linear convergence achieved without strong convexity of f. Paper improves algorithms for convex-concave minimax optimization problems.
problem Minimizing convex-concave functions with strong convexity and concavity properties.
method Proposes a new algorithm with improved gradient complexity.
result Improves gradient complexity upper bound for minimax optimization.
We define a class of L-convex-concave subsets of RP3, where L is a projective line in RP3. These are sets whose sections by any plane containing L are convex and concavely depend on this plane. We prove a version of Arnold hypothesis for these sets, namely we prove that each such set conta…
Algorithm minimizes SP-Regret for online saddle point problem and related knapsack optimization.
problem Online saddle point problem and related online convex optimization with knapsacks.
method Proposed algorithms achieving sublinear SP-Regret in various settings.
result Achieved sublinear SP-Regret bounds for different problem settings.
GAT-GMM improves GANs' performance in learning Gaussian mixture models.
problem GANs struggle with multi-modal distributions like Gaussian mixtures.
method Proposes a minimax GAN framework using random linear generator and softmax-based quadratic discriminator.
result Gradient Descent Ascent method converges to an approximate minimax point.
New method shows last-iterate convergence in constrained min-max optimization.
problem Min-max optimization with constraints.
method Optimistic Multiplicative-Weights Update (OMWU) method.
result Last-iterate convergence to saddle points in constrained min-max optimization.
Enhances optimization algorithms to achieve faster convergence rates.
problem Minimizing smooth convex functions efficiently.
method Extends optimistic learning to achieve faster convergence rates.
result Achieves a rate of O(1/T2) for optimization. Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.
problem Efficiently solving constrained convex-concave min-max problems and monotone variational inequalities.
method Higher-order methods achieving iteration complexities of O(1/T^{ rac{p+1}{2}}) for p-th order derivatives.
result Achieved improved convergence rates for min-max and monotone variational inequalities.
New analysis shows convergence rate of 1/k for gradient and extra-gradient methods.
problem Finding saddle points in convex-concave problems.
method Interpreted as proximal point method approximations, showing iterates remain bounded.
result Primal dual gap converges at rate O(1/k).
New algorithms prove fast convergence in complex min-max problems.
problem Proving fast convergence in nonconvex min-max optimization.
method Hamiltonian Gradient Descent (HGD) and Consensus Optimization (CO) algorithms.
result HGD and CO achieve linear convergence in various settings.
A generalized optimistic method for saddle point problems with improved complexity.
problem Solving convex-concave saddle point problems efficiently.
method Proposes a generalized optimistic method that includes the optimistic gradient method as a special case, handling constrained saddle point problems with composite objective functions and arbitrary norms.
result Best-known global iteration complexity bounds for first-, second-, and higher-order methods.
Gradient descent GAN optimization is locally stable under certain conditions.
problem Understanding and stabilizing GAN optimization.
method Analysis of gradient descent GAN optimization, showing local asymptotic stability.
result Gradient descent GAN optimization is locally stable under proper conditions.
Researchers develop methods to recover agent behavior from sparse data using Gaussian processes.
problem Recovering agent behavior from limited, noisy data in potential mean field games.
method Two Gaussian process-based frameworks: inf-sup formulation and bilevel approach.
result Surrogate MFG models can accurately reproduce observed data, even when prior information is limited.
Optimistic method adapted for faster convex-concave min-max problems.
problem Solving convex-concave min-max optimization problems efficiently.
method Adaptive, line search-free second-order methods combining optimistic updates and second-order information.
result Achieves optimal convergence rate without line search or backtracking.
A new algorithm solves minimax problems without needing parameters.
problem Convex-concave minimax optimization problems in machine learning.
method Proposes a fully parameter-free LF-CR and FF-CR algorithms for solving these problems.
result The FF-CR algorithm achieves the best iteration complexity under gradient norm termination criterion.
Paper tackles partial label learning with self-guided retraining.
problem Dealing with partially labeled examples where each instance has a set of candidate labels.
method Unified formulation with constraints for joint training and pseudo-labeling; maximum infinity norm regularization for automatic differentiation; convex-concave optimization problem; upper-bound surrogate objective function.
result Significantly outperforms state-of-the-art partial label learning approaches.
A new algorithm speeds up multi-agent reinforcement learning.
problem Complex interactions between agents in multi-agent reinforcement learning.
method Double averaging scheme for decentralized convex-concave saddle-point problems.
result The algorithm converges to the optimal solution at a global geometric rate.