arXiv research

Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.

TiAda adapts adaptive gradient methods for nonconvex minimax optimization.

Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.

A new method solves a complex optimization problem efficiently.

NeAda solves nonconvex minimax optimization by balancing primal and dual variables adaptively.

New SGDA method speeds up nonconvex minimax optimization.

Paper analyzes nonconvex bandit problems with improved adaptive methods.

The paper analyzes PPM for nonconvex-nonconcave problems, identifying three regions with varying convergence guarantees.

A novel decentralized algorithm improves minimax optimization in federated learning.

New method solves complex constrained optimization problems.

New algorithms solve nonconvex-nonconcave minimax optimization problems.

The paper analyzes generalization bounds for NC-SC/NC-C stochastic minimax optimization.

New algorithms solve nonconvex-concave minimax problems without parameter knowledge.

New algorithms solve complex minimax problems without needing derivatives.

New algorithm solves minimax games with linear constraints.

New algorithm solves nonconvex-convex minimax problems efficiently.

Paper improves risk bounds for nonconvex-strongly-concave minimax problems.

Two new algorithms solve nonconvex-strongly concave problems efficiently.

DS-GDA solves nonconvex-nonconcave problems without regularity conditions.

Minimax optimization has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs), adversarial training and multi-agent reinforcement learning. As most of these applications involve continuous nonconvex-nonconcave formulations, a very basic question arises---"w…

Paper proposes an algorithm to solve complex minimax problems efficiently.

A federated minimax framework for heterogeneous clients.

Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.

This paper analyzes OGDA and EG methods for nonconvex minimax problems.

This paper studies first order methods for solving smooth minimax optimization problems $\min_x \max_y g(x,y)$ where $g(\cdot,\cdot)$ is smooth and $g(x,\cdot)$ is concave for each $x$. In terms of $g(\cdot,y)$, we consider two settings -- strongly convex and nonconvex -- and improve upon the best known rates in both. …

New method tackles nonconvex-nonconcave problems with local KL condition.

Paper analyzes complexity of solving nonconvex-strongly-concave problems.

Two new methods improve block-sparse signal recovery from noisy data.

In this paper, we study zeroth-order algorithms for minimax optimization problems that are nonconvex in one variable and strongly-concave in the other variable. Such minimax optimization problems have attracted significant attention lately due to their applications in modern machine learning tasks. We first consider a …

Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.

Nonconvex minimax problems appear frequently in emerging machine learning applications, such as generative adversarial networks and adversarial learning. Simple algorithms such as the gradient descent ascent (GDA) are the common practice for solving these nonconvex games and receive lots of empirical success. Yet, it i…

A fast sketching algorithm solves regularized least squares problems efficiently.

Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.

Paper analyzes convergence of GDA for nonconvex-nonconcave minimax problems.

New algorithm solves federated minimax optimization problems.

Paper proves Sion's theorem in geodesic spaces and develops a Riemannian extragradient method.

Improved FTPL algorithm reduces regret in predictable minimax games.

Sparse principal component analysis (PCA) involves nonconvex optimization for which the global solution is hard to obtain. To address this issue, one popular approach is convex relaxation. However, such an approach may produce suboptimal estimators due to the relaxation effect. To optimally estimate sparse principal su…

We consider compressed sensing formulated as a minimization problem of nonconvex sparse penalties, Smoothly Clipped Absolute deviation (SCAD) and Minimax Concave Penalty (MCP). The nonconvexity of these penalties is controlled by nonconvexity parameters, and L1 penalty is contained as a limit with respect to these para…

Method solves nonconvex constrained optimization problems with a new augmented Lagrangian approach.

We consider nonconvex-concave minimax optimization problems of the form $\min_{\bf x}\max_{\bf y\in{\mathcal Y}} f({\bf x},{\bf y})$, where $f$ is strongly-concave in $\bf y$ but possibly nonconvex in $\bf x$ and ${\mathcal Y}$ is a convex and compact set. We focus on the stochastic setting, where we can only access an…

A new decentralized method solves minimax problems with reduced communication and sample complexity.

This paper resolves a longstanding open question pertaining to the design of near-optimal first-order algorithms for smooth and strongly-convex-strongly-concave minimax problems. Current state-of-the-art first-order algorithms find an approximate Nash equilibrium using $\tilde{O}(κ_{\mathbf x}+κ_{\mathbf y})$ or $\tild…

We consider nonconvex-concave minimax problems, $\min_{\mathbf{x}} \max_{\mathbf{y} \in \mathcal{Y}} f(\mathbf{x}, \mathbf{y})$, where $f$ is nonconvex in $\mathbf{x}$ but concave in $\mathbf{y}$ and $\mathcal{Y}$ is a convex and bounded set. One of the most popular algorithms for solving this problem is the celebrated…

New algorithms solve complex minimax problems efficiently.

Paper proposes SMO for solving bilevel optimization problems efficiently.

Adversarial training, a special case of multi-objective optimization, is an increasingly prevalent machine learning technique: some of its most notable applications include GAN-based generative modeling and self-play techniques in reinforcement learning which have been applied to complex games such as Go or Poker. In p…

We propose a unified framework for estimating low-rank matrices through nonconvex optimization based on gradient descent algorithm. Our framework is quite general and can be applied to both noisy and noiseless observations. In the general case with noisy observations, we show that our algorithm is guaranteed to linearl…