Adaptive momentum method solves non-convex min-max problems.
problem Non-convex min-max optimization problems in training generative adversarial networks.
method Proposes an adaptive momentum algorithm for non-convex min-max optimization.
result Establishes non-asymptotic convergence rates for the proposed algorithm.
New study shows min-max algorithms can converge to non-stationary points.
problem Challenges in min-max optimization due to periodic cycles and spurious attractors.
method Analyzed state-of-the-art algorithms and heuristics in non-convex/non-concave problems.
result Spurious attractors can prevent min-max algorithms from reaching true optima.
New algorithm solves non-convex, non-differentiable min-max games.
problem Limited theoretical understanding of non-smooth min-max games.
method Proximal gradient descent-ascent algorithm for convex-strongly convex games.
result Algorithm converges to ε-Nash equilibrium with polynomial gradient evaluations.
New algorithm solves structured nonconvex-nonconcave min-max problems.
problem Min-max optimization challenges in deep learning.
method Generalized extragradient algorithm for structured nonconvex-nonconcave problems.
result Algorithm converges to stationary points in Euclidean and ℓp spaces. Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.
problem Efficiently solving constrained convex-concave min-max problems and monotone variational inequalities.
method Higher-order methods achieving iteration complexities of O(1/T^{ rac{p+1}{2}}) for p-th order derivatives.
result Achieved improved convergence rates for min-max and monotone variational inequalities.
New algorithm solves min-max optimization problems in a decentralized manner.
problem Solving min-max saddle point games in a decentralized and adaptive manner.
method Developed a decentralized adaptive momentum (DADAM3) algorithm for min-max optimization. result DADAM3 achieves non-asymptotic rates of convergence for finding Nash equilibrium points. We reformulate LIPs as min-max problems for easier solution.
problem Recovering signals from few linear measurements.
method Proposed a min-max reformulation of LIPs.
result Saddle points characterize solutions to LIPs.
Study introduces statistical mechanics for min-max problems.
problem Understanding the properties of min-max problems in high dimensions.
method Statistical mechanical formalism for analyzing min-max problems.
result Derives the relationship between training data and generalization error.
In this paper, we consider first-order convergence theory and algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex in the variables of minimization and weakly concave in the variables of maximization. It has many important applications in mach…
General fuzzy min-max (GFMM) neural network is a generalization of fuzzy neural networks formed by hyperbox fuzzy sets for classification and clustering problems. Two principle algorithms are deployed to train this type of neural network, i.e., incremental learning and agglomerative learning. This paper presents a comp…
Proposes an efficient alternative to nonconvex-nonconcave min-max optimization.
problem Min-max optimization challenges in nonconvex-nonconcave settings.
method Introduces ε-greedy adversarial equilibrium model and proves its existence.
result Existence of ε-greedy adversarial equilibrium for smooth bounded functions.
Survey of advances in non-convex min-max optimization for applications.
problem Finding optimal solutions in non-convex, non-concave min-max problems.
method Selective review of theoretical and algorithmic advances.
result Exciting recent advances in solving non-convex min-max problems.
Bayesian optimization methods improved for min max optimization problems.
problem Min-max optimization for unknown functions.
method Extended Bayesian optimization to min-max problems with new acquisition functions.
result Improved acquisition functions lead to better solutions.
Study on local convergence of min-max algorithms to differential equilibria on Riemannian manifolds.
problem Solving zero-sum differential games on Riemannian manifolds.
method Analysis of two simultaneous min-max algorithms, τ-GDA and τ-SGA, to differential Stackelberg and Nash equilibria, with conditions for linear convergence and asymptotic approximation. result Established sufficient conditions for linear convergence of τ-GDA and demonstrated faster convergence of τ-SGA in some cases. We consider the problem of two-player zero-sum games. This problem is formulated as a min-max Markov game in the literature. The solution of this game, which is the min-max payoff, starting from a given state is called the min-max value of the state. In this work, we compute the solution of the two-player zero-sum game…
Riemannian algorithms converge at Euclidean rates for geodesically convex-concave problems.
problem Min-max optimization on Riemannian manifolds.
method RCEG method and RGDA for geodesically strongly-convex-concave problems.
result RCEG achieves linear convergence rate in geodesically strongly-convex-concave cases.
Study on convergence of Langevin dynamics for zero-sum games in probability distributions.
problem Analyzing convergence of Langevin dynamics for zero-sum games in probability distributions.
method Proved exponential and biased convergence guarantees for mean-field and finite-particle min-max Langevin dynamics.
result Explicit iteration complexity for finite-particle algorithms to approximate equilibrium distributions.
While classic work in convex-concave min-max optimization relies on average-iterate convergence results, the emergence of nonconvex applications such as training Generative Adversarial Networks has led to renewed interest in last-iterate convergence guarantees. Proving last-iterate convergence is challenging because ma…
This paper proposes an improved version of the current online learning algorithm for a general fuzzy min-max neural network (GFMM) to tackle existing issues concerning expansion and contraction steps as well as the way of dealing with unseen data located on decision boundaries. These drawbacks lower its classification …
Paper tackles fast convergence for non-convex strongly-concave min-max problems.
problem Non-convex strongly-concave min-max problems in deep learning.
method Proximal stage-based method with PL condition for faster convergence.
result Established fast convergence in primal objective gap and duality gap.
Paper tackles multi-block min-max optimization with applications in deep AUC maximization.
problem Multi-block min-max bilevel optimization with non-convex strongly-concave upper level and strongly convex lower level.
method Single-loop randomized stochastic algorithm for constant number of blocks per iteration.
result Sample complexity of O(1/ε^4) for finding ε-stationary point, matching optimal complexity.
New algorithms improve DRSL for large-scale problems.
problem Distributionally robust learning for real-world applications.
method Variance-reduced stochastic extra-gradient algorithms for min-max optimization.
result Provable faster convergence rates than existing approaches.
New algorithms solve stochastic variational inequalities without bounded variance assumption.
problem Solving stochastic variational inequalities without bounded variance assumption.
method Developed algorithms for two classes of problems: monotone and structured nonmonotone VIs.
result Oracle complexity of O(ε^-4) for solving VIs with unbounded domains and possibly unbounded variance.
Optimizes solving complex min-max problems with stochastic and nonconvex elements.
problem Min-max problems with stochastic and nonconvex elements.
method Combines conic nonexpansiveness, refined inexact Halpern iteration, and multilevel Monte Carlo estimator.
result Optimal or best-known complexity guarantees for $ρ< rac{1}{L}$, improving previous results.
Epoch gradient descent method (a.k.a. Epoch-GD) proposed by Hazan and Kale (2011) was deemed a breakthrough for stochastic strongly convex minimization, which achieves the optimal convergence rate of O(1/T) with T iterative updates for the {\it objective gap}. However, its extension to solving stochastic min-max pr…
The min-max kernel is a generalization of the popular resemblance kernel (which is designed for binary data). In this paper, we demonstrate, through an extensive classification study using kernel machines, that the min-max kernel often provides an effective measure of similarity for nonnegative data. As the min-max ker…
PURE-CD algorithm proves complexity bounds for convex-concave problems.
problem Solving convex-concave min-max problems with bilinear coupling.
method Primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD).
result Complexity bounds match or improve existing results for dense and sparse problems.
Study shows strong min-max principle for phase transitions.
problem Understanding nodal sets near minimal hypersurfaces.
method Analogous to White's principle, applies to Allen-Cahn energy.
result Strong min-max principle for phase transitions.
Equity-Transformer solves NP-hard min-max routing problems efficiently.
problem Min-max routing problems with multiple agents and large-scale applications.
method Sequential planning approach with Transformer and equitable workload distribution inductive biases.
result Significant runtime and cost reductions in min-max mTSP and min-max mPDP tasks.
Adam-type optimizers show one-sided convergence in GAN training, not reaching critical points.
problem Theoretical understanding of Adam-type optimizers in non-convex non-concave min-max optimization.
method Empirical and theoretical analysis of Adam-type algorithms' convergence in GAN training.
result Adam-type algorithms converge to one-sided first order stationary points under the one-sided MVI condition.
PAPAL algorithm finds mixed Nash equilibria in continuous games.
problem Finding mixed Nash equilibria in non-convex, non-concave games.
method Particle-based Primal-Dual Algorithm (PAPAL) for weakly entropy-regularized min-max optimization.
result PAPAL offers non-asymptotic convergence guarantees for ε-mixed Nash equilibrium. Upper bound for Morse index of min-max varifolds.
problem Bounding Morse index of varifolds.
method Proving upper bound for Morse index of min-max stationary integral varifolds.
result Upper bound for Morse index of min-max stationary integral varifolds.
Localized min-max method proves minimal hypersurface existence.
problem Existence of minimal hypersurfaces in complete manifolds.
method Localized min-max approach to prove existence.
result Existence of complete embedded minimal hypersurface with index at most one.
The paper solves min-max widths on a 3-sphere and strengthens multiplicity theorems.
problem Which min-max widths of the unit 3-sphere lie between 2π2 and 8π? method Homological min-max theory and stronger versions of multiplicity one theorems.
result Proves the 10th to 13th min-max widths of the unit 3-sphere lie between 2π2 and 8π. Paper proves finiteness and Morse index estimates for equivariant min-max hypersurfaces.
problem Existence and finiteness of G-invariant minimal hypersurfaces. method Equivariant min-max theory, compactness theorem, bumpy metrics theorem.
result Generalization of Morse index estimates to equivariant setting.
New algorithm converges to equilibrium in nonconvex-nonconcave optimization problems without dimension dependence.
problem Min-max optimization in nonconvex-nonconcave landscapes.
method Convergent algorithm with greedy max-player updates and proposal distribution for min-player.
result Algorithm converges to equilibrium in non-dependent iterations, suitable for GAN training.
In this paper, we compare 5 different nonlinear kernels: min-max, RBF, fRBF (folded RBF), acos, and acos-χ2, on a wide range of publicly available datasets. The proposed fRBF kernel performs very similarly to the RBF kernel. Both RBF and fRBF kernels require an important tuning parameter (γ). Interestingly, for a …
New proof of Smale conjecture for RP^3 and lens spaces using min-max theory.
problem Proving the Smale conjecture for specific spaces.
method Minimal surfaces and min-max theory.
result New proof of Smale conjecture for RP3 and lens spaces. Paper improves Morse index bound for hypersurfaces.
problem Improving Morse index bound for hypersurfaces.
method Construction of hierarchical deformations and restrictive min-max theory.
result Generalizes a result by X. Zhou for 3≤n+1≤7. New methods solve min-max problems on manifolds using Riemannian Hamiltonians.
problem Min-max optimization on Riemannian manifolds.
method Riemannian Hamiltonian methods (RHM) to minimize the Hamiltonian function.
result RHM leads to correct search directions and global optimality in min-max problems.
Bound on equivariant index for min-max surfaces.
problem Bounding the index of equivariant min-max surfaces.
method Equivariant min-max procedure with group action.
result Equivariant index bound by number of parameters.
We prove that in a closed manifold of dimension between 3 and 7 with a bumpy metric, the min-max minimal hypersurfaces associated with the volume spectrum introduced by Gromov, Guth, Marques-Neves, are two-sided and have multiplicity one. This confirms a conjecture by Marques-Neves. We prove that in a bumpy metric each…
New algorithms reduce variance in solving complex mathematical problems.
problem Solving convex-concave saddle point problems, variational inequalities, and inclusions.
method Stochastic variance reduction for extragradient, forward-backward-forward, and forward-reflected-backward methods.
result All proposed methods converge with complexities matching or improving deterministic counterparts.
The paper bounds the min-max width of embedded circles on spheres and manifolds.
problem Bounding the min-max width of embedded circles on spheres and manifolds.
method Inducing a sweepout by pairs of points in embedded circles from a given sweepout of the sphere by closed curves.
result Lower bounds for the Birkhoff min-max invariant of a Riemannian sphere in terms of the min-max width of its embedded circles.
Study confirms a 2-sphere metric with three geodesics of minimal length.
problem Understanding the systolic, width, and Gromov-Guth metrics on a 2-sphere.
method Classical min-max and hyperbolic geometry tools.
result Figure-eight geodesics achieve the systolic, width, and Gromov-Guth metrics on a 2-sphere.
Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…
In this paper, we study the shape of the min-max minimal hypersurface produced by Almgren-Pitts-Schoen-Simon \cite{AF62, AF65, P81, SS81} in a Riemannian manifold (Mn+1,g) of positive Ricci curvature for all dimensions. The min-max hypersurface has a singular set of Hausdorff codimension 7. We characterize the …
The min-max problem, also known as the saddle point problem, is a class of optimization problems which minimizes and maximizes two subsets of variables simultaneously. This class of problems can be used to formulate a wide range of signal processing and communication (SPCOM) problems. Despite its popularity, most exist…