arXiv research

We define a class of L-convex-concave subsets of $\Bbb{R}P^n$, where L is a projective subspace of dimension l in $\Bbb{R}P^n$. These are sets whose sections by any (l+1)-dimensional space L' containing L are convex and concavely depend on L'. We introduce an L-duality for these sets, and prove that the L-dual to an L-…

OMWU shows last iterate convergence in convex-concave games.

New saddle network architectures preserve convex-concave geometry in optimization problems.

PURE-CD algorithm proves complexity bounds for convex-concave problems.

ICCNLS models complex relationships as convex and concave components.

Paper introduces $\ell$-DER for regression tasks using morphological operators and convex-concave procedure.

In this paper we study the smooth convex-concave saddle point problem. Specifically, we analyze the last iterate convergence properties of the Extragradient (EG) algorithm. It is well known that the ergodic (averaged) iterates of EG converge at a rate of $O(1/T)$ (Nemirovski, 2004). In this paper, we show that the last…

New method finds arbitrage opportunities in fluctuating asset bands.

As has been observed by Morse \cite{Mo}, any generic vector field $v$ on a compact smooth manifold $X$ with boundary gives rise to a stratification of the boundary $\d X$ by compact submanifolds $\{\d_j^\pm X(v)\}_{1 \leq j \leq \dim(X)}$, where $\textup{codim}(\d_j^\pm X(v))= j$. Our main observation is that this stra…

Riemannian algorithms converge at Euclidean rates for geodesically convex-concave problems.

Paper improves algorithms for convex-concave minimax optimization problems.

We define a class of $L$-convex-concave subsets of $\mathbb{R}P^3$, where $L$ is a projective line in $\mathbb{R}P^3$. These are sets whose sections by any plane containing $L$ are convex and concavely depend on this plane. We prove a version of Arnold hypothesis for these sets, namely we prove that each such set conta…

While classic work in convex-concave min-max optimization relies on average-iterate convergence results, the emergence of nonconvex applications such as training Generative Adversarial Networks has led to renewed interest in last-iterate convergence guarantees. Proving last-iterate convergence is challenging because ma…

Improved algorithms for convex-concave min-max optimization and monotone variational inequalities.

We consider the convex-concave saddle point problem $\min_{x}\max_{y} f(x)+y^\top A x-g(y)$ where $f$ is smooth and convex and $g$ is smooth and strongly convex. We prove that if the coupling matrix $A$ has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if $f$ is not stron…

Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.

New algorithm AG-OG optimizes separable convex-concave problems efficiently.

A generalized optimistic method for saddle point problems with improved complexity.

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a uniq…

Optimistic method adapted for faster convex-concave min-max problems.

A new algorithm solves minimax problems without needing parameters.

We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstrained min-max problem. To do so, we first show that both OGDA and EG can be interpreted as approximate variants of the proximal point method…

Optimizes bond portfolios to avoid worst-case losses.

We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems. Our method shares the efficiency an…

We consider the use of no-regret algorithms to compute equilibria for particular classes of convex-concave games. While standard regret bounds would lead to convergence rates on the order of $O(T^{-1/2})$, recent work \citep{RS13,SALS15} has established $O(1/T)$ rates by taking advantage of a particular class of optimi…

APAC-Net solves high-dimensional stochastic MFGs using neural networks.

We consider variational inequalities coming from monotone operators, a setting that includes convex minimization and convex-concave saddle-point problems. We assume an access to potentially noisy unbiased values of the monotone operators and assess convergence through a compatible gap function which corresponds to the …

Partial label learning deals with the problem where each training instance is assigned a set of candidate labels, only one of which is correct. This paper provides the first attempt to leverage the idea of self-training for dealing with partially labeled examples. Specifically, we propose a unified formulation with pro…

We solve a complex optimization problem for Wasserstein barycenters using stochastic methods.

Despite the success of single-agent reinforcement learning, multi-agent reinforcement learning (MARL) remains challenging due to complex interactions between agents. Motivated by decentralized applications such as sensor networks, swarm robotics, and power grids, we study policy evaluation in MARL, where agents with jo…

This work analyzes how overparameterization aids GANs in reaching global saddle points.

Optimizes portfolios using CPT utility via convex optimization.

We reformulate LIPs as min-max problems for easier solution.

We present a distributionally robust formulation of a stochastic optimization problem for non-i.i.d vector autoregressive data. We use the Wasserstein distance to define robustness in the space of distributions and we show, using duality theory, that the problem is equivalent to a finite convex-concave saddle point pro…

We prove a priori interior $C^{2,α}$ estimates for solutions of fully nonlinear elliptic equations of twisted type. For example, our estimates apply to equations of the type convex + concave. These results are particularly well suited to equations arising from elliptic regularization. As application, we obtain a new pr…

This work finds mixed equilibria in machine learning problems using measures and simultaneous gradient ascent-descent.

This paper resolves a longstanding open question pertaining to the design of near-optimal first-order algorithms for smooth and strongly-convex-strongly-concave minimax problems. Current state-of-the-art first-order algorithms find an approximate Nash equilibrium using $\tilde{O}(κ_{\mathbf x}+κ_{\mathbf y})$ or $\tild…

New algorithms solve monotone inclusions and convex-concave minimax problems.

We consider the problem of decomposing a multivariate polynomial as the difference of two convex polynomials. We introduce algebraic techniques which reduce this task to linear, second order cone, and semidefinite programming. This allows us to optimize over subsets of valid difference of convex decompositions (dcds) a…

Paper optimizes hyperparameters for high-dimensional regression models.

Paper establishes lower bounds for finite-sum optimization problems using novel construction methods.

Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond. By necessity, most theoretical guarantees revolve around convex-concave (or even linear) problems; however, making theoretical inroads towards effici…

The paper extends convexity results for translating solitons in higher dimensions.

The paper analyzes how optimization algorithms affect the generalization of minimax models.

We study the online saddle point problem, an online learning problem where at each iteration a pair of actions need to be chosen without knowledge of the current and future (convex-concave) payoff functions. The objective is to minimize the gap between the cumulative payoffs and the saddle point value of the aggregate …

Efficiently maximizes AUC with deep nets, reducing communication rounds.

We consider an abstract compact orientable Cauchy-Riemann manifold endowed with a Cauchy-Riemann complex line bundle. We assume that the manifold satisfies condition Y(q) everywhere. In this paper we obtain a scaling upper-bound for the Szegö kernel on (0, q)-forms with values in the high tensor powers of the line bund…

We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation. Inspired by problems in stochastic programming, we show that two-stage procedures for non-linear IV regression can be reformulated as a convex-concave saddle-…