This paper analyzes saddle points and minimax points in non-convex smooth games.
problem Understanding local optimal points in non-convex smooth games.
method Comprehensive analysis of local minimax points, including their optimality conditions and stability.
result Local saddle points are uniformly local minimax points under mild continuity assumptions.
Two new algorithms solve nonconvex-strongly concave problems efficiently.
problem Solving nonconvex-strongly concave minimax problems.
method Proposed MINIMAX-TR and MINIMAX-TRACE algorithms.
result Find ( ε , ε ) (ε, \sqrtε) ( ε , ε ) -second order stationary points within O ( ε − 1.5 ) \mathcal{O}(ε^{-1.5}) O ( ε − 1.5 ) iterations. MOSAIC detects change points in dynamic networks with low-rank and sparse changes.
problem Detecting change points in dynamic networks with specific structural properties.
method Eigen-decomposition-based test with screened signals and residual-based adjustment.
result MOSAIC achieves minimax-optimal detection and testing rates.
Paper proves Sion's theorem in geodesic spaces and develops a Riemannian extragradient method.
problem Understanding saddle points in nonconvex-nonconcave minimax problems.
method Geodesic metric space version of Sion's theorem and Riemannian extragradient method.
result Developed a Riemannian extragradient algorithm for smooth minimax problems.
New algorithms solve nonconvex-nonconcave minimax optimization problems.
problem Solving minimax optimization problems in machine learning.
method Two novel Newton-type algorithms for nonconvex-nonconcave minimax optimization.
result Proved local convergence at strict local minimax points.
This paper studies continuum-armed bandits under Besov smoothness conditions and derives minimax rates.
problem Optimizing an unknown function with limited evaluations.
method Studies continuum-armed bandits under Besov smoothness conditions and derives minimax rates.
result Minimax rates over Besov spaces are identical to those over the smallest Hölder space into which Besov spaces embed.
Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"w…
New methods solve complex optimization problems without strong convexity assumptions.
problem Complex bilevel optimization problems with minimax lower-level structures.
method Penalty-based first-order methods for bilevel minimax optimization.
result Achieves ε ε ε -KKT point with improved oracle complexity. Many tasks in modern machine learning can be formulated as finding equilibria in \emph{sequential} games. In particular, two-player zero-sum sequential games, also known as minimax optimization, have received growing interest. It is tempting to apply gradient descent to solve minimax optimization given its popularity a…
Paper proposes an algorithm to solve complex minimax problems efficiently.
problem Stochastic nonconvex-concave minimax problems in various fields.
method Accelerated first-order regularized momentum descent ascent algorithm (FORMDA).
result Achieves best-known complexity bound of i l d e O ( ε − 6.5 ) ilde{\mathcal{O}}(\varepsilon ^{-6.5}) i l d e O ( ε − 6.5 ) for single-loop algorithms. This paper tackles convex-submodular minimax problems in mixed continuous-discrete domains.
problem Convex-submodular minimax problems in mixed continuous-discrete domains.
method Introduces new notions of optimality and proposes iterative algorithms combining discrete and continuous optimization.
result Characterizes convergence rates, computational complexity, and quality of solutions for convex and monotone-submodular minimax problems.
Sampling without replacement speeds up optimization in minimax problems.
problem Optimizing minimax problems with faster convergence rates.
method Analysis of gradient descent ascent and proximal point method with two sampling strategies.
result Sampling without replacement leads to faster convergence rates in minimax optimization.
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
The paper analyzes PPM for nonconvex-nonconcave problems, identifying three regions with varying convergence guarantees.
problem Challenges in nonconvex-nonconcave minimax optimization.
method Classic proximal point method with insights from the Moreau envelope.
result Identification of three regions with varying convergence guarantees for PPM.
Paper analyzes minimax risks of personalized federated learning algorithms.
problem Statistical heterogeneity among clients in federated learning.
method Minimax analysis of FedAvg and local training approaches.
result Threshold for optimality between FedAvg and local training depends on data heterogeneity.
Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.
problem Nonconvex-nonconcave minimax problems with PL condition.
method Zeroth-order AGDA and VRAGDA algorithms.
result Iteration complexities for obtaining ε-stationary points.
New algorithms solve complex minimax problems efficiently.
problem Nonconvex-strongly concave minimax problems in machine learning.
method Gradient norm regularized trust-region (GRTR) and Levenberg-Marquardt (LMNegCur) algorithms.
result Proved iteration complexities matching best known results.
New method solves complex constrained optimization problems.
problem Constrained nonconvex-nonconcave minimax optimization problems.
method Inexact proximal gradient method using sequential convex programming.
result Established complexity guarantees for approximate stationary points.
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
Optimal nonparametric regression estimator adapts to unknown smoothness.
problem Nonparametric regression with unknown smoothness.
method Constructs an interpolating estimator that adapts to unknown smoothness.
result Minimax optimal rates achieved on Hölder classes.
New method tackles nonconvex-nonconcave problems with local KL condition.
problem Nonconvex-nonconcave minimax problems under varying KL conditions.
method Inexact proximal gradient method for KL-structured subproblems.
result Complexity guarantees for approximate stationary points.
Paper improves risk bounds for nonconvex-strongly-concave minimax problems.
problem Achieving sharper risk bounds for nonconvex-strongly-concave minimax problems.
method Using uniform localized convergence to derive high probability generalization error bounds.
result Derives n times faster excess primal risk bounds for popular algorithms.
Paper explores generalization of minimax learners, proposing a new metric.
problem Understanding how minimax learners perform on unseen data.
method Proposes a new metric, the primal gap, to study generalization of minimax learners.
result Derives generalization error bounds for the primal gap in nonconvex-concave settings.
Novel methods for accelerating optimization in complex bilevel and minimax problems.
problem Optimization challenges in bilevel and minimax problems, especially when strong convexity assumptions are not met.
method Accelerated fully first-order methods for Bilevel Optimization (BLO) and Minimax Optimization (NCSC).
result State-of-the-art complexity for finding approximate second-order stationary points in BLO and NCSC.
Study minimax estimation of stratified structure from i.i.d. samples.
problem Estimating stratified structure from i.i.d. samples of stratified mixtures of immersed manifolds.
method Ascending hierarchical co-detection of points belonging to different layers, identifying number of layers and their dimensions, assigning points to layers accurately, estimating tangent spaces optimally.
result Achieves optimal estimation of mixture components at their optimal dimension-specific rates adaptively.
The paper analyzes how optimization algorithms affect the generalization of minimax models.
problem The generalization performance of minimax models trained with different optimization algorithms.
method Analysis of gradient descent ascent (GDA) and proximal point method (PPM) algorithms under convex concave and non-convex non-concave settings.
result The PPM algorithm ensures a bounded excess risk in convex concave problems, while GDA's generalization depends on solving subproblems simultaneously.
State-of-the-art adversarial attacks are aimed at neural network classifiers. By default, neural networks use gradient descent to minimize their loss function. The gradient of a classifier's loss function is used by gradient-based adversarial attacks to generate adversarially perturbed images. We pose the question whet…
New algorithms improve convergence of minimax optimization.
problem Minimax optimization convergence issues in nonconvex problems.
method Established new convergence results for two single-loop algorithms.
result Improved convergence rates for minimax optimization.
A new method trains physics-constrained neural networks more efficiently.
problem Training machine learning tools with limited data and physical constraints.
method Dual-Dimer method for searching saddle points in nonconvex-nonconcave functions.
result The Dual-Dimer method improves training efficiency and convergence speed.
Randomization is minimax-optimal for variance in experimental design, even with structure.
problem Designing optimal randomized experiments for variance minimization.
method Analyzing permutation symmetric and non-symmetric sets of outcomes, proposing inference-constrained MSOD.
result Randomization is minimax-optimal for variance, even with structure, and requires uniformity constraints for Fisher's exact test.
This paper studies first order methods for solving smooth minimax optimization problems min x max y g ( x , y ) \min_x \max_y g(x,y) min x max y g ( x , y ) where g ( ⋅ , ⋅ ) g(\cdot,\cdot) g ( ⋅ , ⋅ ) is smooth and g ( x , ⋅ ) g(x,\cdot) g ( x , ⋅ ) is concave for each x x x . In terms of g ( ⋅ , y ) g(\cdot,y) g ( ⋅ , y ) , we consider two settings -- strongly convex and nonconvex -- and improve upon the best known rates in both. …
Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.
problem Nonconvex minimax problems with coupled linear constraints.
method Zeroth-order primal-dual alternating projected gradient (ZO-PDAPG) and zeroth-order regularized momentum primal-dual projected gradient (ZO-RMPDPG) algorithms.
result Iteration complexity guarantees for solving nonconvex-(strongly) concave minimax problems with coupled linear constraints.
New algorithms solve nonconvex-concave minimax problems without parameter knowledge.
problem Solving nonconvex-concave minimax problems efficiently.
method Three completely parameter-free single-loop algorithms.
result Achieve optimal iteration complexity for nonconvex-concave minimax problems.
Considering that the Seiberg-Witten functional satisfies the Palais-Smale Condition, up to gauge equivalence, the Minimax Principle can be applied on the moduli space to prove the existence of critical points, which correspond to solutions of the second-order SW-equations, up to gauge equivalence.
Paper reconciles minimax rates and optimal recovery rates for noisy observations.
problem Estimating a function from noisy observations.
method Develops NLA minimax rates for Besov classes in L q L_q L q -norms. result NLA minimax rates continuously depend on noise level and match optimal recovery rates as noise decreases.
GAT-GMM improves GANs' performance in learning Gaussian mixture models.
problem GANs struggle with multi-modal distributions like Gaussian mixtures.
method Proposes a minimax GAN framework using random linear generator and softmax-based quadratic discriminator.
result Gradient Descent Ascent method converges to an approximate minimax point.
A new algorithm solves minimax problems without needing parameters.
problem Convex-concave minimax optimization problems in machine learning.
method Proposes a fully parameter-free LF-CR and FF-CR algorithms for solving these problems.
result The FF-CR algorithm achieves the best iteration complexity under gradient norm termination criterion.
Minimax optimization plays a key role in adversarial training of machine learning algorithms, such as learning generative models, domain adaptation, privacy preservation, and robust learning. In this paper, we demonstrate the failure of alternating gradient descent in minimax optimization problems due to the discontinu…
We consider semi-supervised regression when the predictor variables are drawn from an unknown manifold. A simple two step approach to this problem is to: (i) estimate the manifold geodesic distance between any pair of points using both the labeled and unlabeled instances; and (ii) apply a k nearest neighbor regressor b…
Study sharp convergence rates of empirical UOT for spatio-temporal point processes.
problem Statistical analysis of UOT for spatio-temporal point processes.
method Empirical plug-in estimators for Kantorovich-Rubinstein distance between intensity measures.
result Sharp convergence rates of empirical UOT in terms of intrinsic dimensions of measures.
The paper develops adaptive confidence intervals for Efron's Gaussian two-groups model with unknown contamination.
problem Developing robust uncertainty quantification for Efron's Gaussian two-groups model with unknown contamination fraction.
method The approach involves Fourier-based certification procedures to find minimax-optimal adaptive confidence intervals.
result The minimax-optimal length of adaptive confidence intervals is polynomially worse than when contamination fraction is known.
New algorithms estimate Q-functions under partial coverage and realizability, improving offline RL guarantees.
problem Offline RL with limited exploration and assumptions about data coverage and Q-function realizability.
method Proposes minimax learning algorithms to estimate soft or vanilla Q-functions with L 2 L^2 L 2 -convergence guarantees. result PAC guarantees for offline RL under partial coverage and realizability conditions.
This paper analyzes neural networks for solving complex optimization problems.
problem Minimax optimization problems in infinite-dimensional function spaces.
method Mean-field analysis of stochastic gradient descent-ascent in neural networks.
result The algorithm converges to a stationary point at a sublinear rate.
This paper establishes minimax rates for online regression with arbitrary classes of functions and general losses. We show that below a certain threshold for the complexity of the function class, the minimax rates depend on both the curvature of the loss function and the sequential complexities of the class. Above this…
We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a uniq…
Novel graph-spanning algorithm detects changes in high-dimensional data.
problem Detecting changes in high-dimensional data across various distributions.
method Graph-spanning algorithm designed for offline and online data.
result Achieves high detection power with minimal separation rate.
DARLING tackles non-stationary RL with guarantees, improving dynamic regret.
problem Non-stationary reinforcement learning in unknown change points.
method Detection Augmented Reinforcement Learning (DARLING) for tabular and linear MDPs.
result DARLING matches minimax lower bounds in tabular and linear MDPs.
SDP achieves optimal error in noisy phase synchronization.
problem Phase synchronization with noisy measurements.
method SDP relaxation of Maximum Likelihood Estimation (MLE).
result Achieves error bound of ( 1 + o ( 1 ) ) σ 2 2 n p (1+o(1))\frac{σ^2}{2np} ( 1 + o ( 1 )) 2 n p σ 2 under normalized squared ℓ 2 \ell_2 ℓ 2 loss, matching minimax lower bound.