arXiv research

Paper studies convergence of Mean-Field GDA dynamics for MNE of continuous games.

Gradient-descent-ascent dynamics can exhibit various behaviors in non-convex non-concave games.

Study local convergence of GDA for training GANs with kernel-based discriminators.

Method identifies mixed Nash equilibria in high dimensions for training mixtures of GANs.

Paper analyzes adversarial dynamics in neural networks.

Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.

Motivated by applications in Optimization, Game Theory, and the training of Generative Adversarial Networks, the convergence properties of first order methods in min-max problems have received extensive study. It has been recognized that they may cycle, and there is no good understanding of their limit points when they…

Gradient descent-ascent converges to strict local minmax equilibria with a finite timescale separation.

New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.

This work finds mixed equilibria in machine learning problems using measures and simultaneous gradient ascent-descent.

Optimizes variational inference for dynamic network models.

Studying a softmax-attention model, we show that the learned query converges to the latent signal subspace spanned by the informative direction.

Improves posterior approximation speed for Dirichlet process mixture models.

We present a new algorithm based on an gradient ascent for a general Active Exploration bandit problem in the fixed confidence setting. This problem encompasses several well studied problems such that the Best Arm Identification or Thresholding Bandits. It consists of a new sampling rule based on an online lazy mirror …

Gradient ascent method successfully removes specific data points from neural networks without retraining.

Sequential coordinate ascent is more robust in high-dimensional linear regression.

Risk management in dynamic decision problems is a primary concern in many fields, including financial investment, autonomous driving, and healthcare. The mean-variance function is one of the most widely used objective functions in risk management due to its simplicity and interpretability. Existing algorithms for mean-…

The paper explains how simple methods can converge to optimal solutions in complex neural games.

Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised machine learning optimization problems such as SVM, due to their strong theoretical guarantees. While the closely related Dual Coordinate Ascent (DCA) method has been implemented in various software packages, it has so far lacked go…

Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning. This paper considers an extension of SDCA under the mini-batch setting that is often used in practice. Our main contribution is to introduce an accelerated mini-batch version of SDC…

New algorithm solves federated minimax optimization problems.

We introduce a proximal version of dual coordinate ascent method. We demonstrate how the derived algorithmic framework can be used for numerous regularized loss minimization problems, including $\ell_1$ regularization and structured output SVM. The convergence rates we obtain match, and sometimes improve, state-of-the-…

SoftAD improves classification accuracy with less fine-tuning and fewer computational costs.

Policy gradient aims to maximize expected return using gradient ascent.

This work uses a scalable approach to identify partially observed nonlinear systems.

Algorithm improves variational inference in Wasserstein distance.

Paper introduces SGA for barycenter optimization in optimal transport.

Lazy neural networks are vulnerable to adversarial attacks.

This paper analyzes neural networks for solving complex optimization problems.

Policy mirror ascent achieves Nash equilibrium in mean field games without a population generative model.

Paper proposes an algorithm to solve complex minimax problems efficiently.

In machine learning, Feature Selection (FS) is a major part of efficient algorithm. It fuels the algorithm and is the starting block for our prediction. In this paper, we present a new method, called Optimal Coordinate Ascent (OCA) that allows us selecting features among block and individual features. OCA relies on coo…

The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees. However, the available S-DCA formulation is limited to finit…

Random scan CAVI converges linearly under log-concave assumptions.

We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including …

New algorithm for solving minimax problems over distributions converges to Nash equilibrium.

Generative adversarial networks (GANs) are a widely used framework for learning generative models. Wasserstein GANs (WGANs), one of the most successful variants of GANs, require solving a minmax optimization problem to global optimality, but are in practice successfully trained using stochastic gradient descent-ascent.…

New taxonomy and improved solvers for discrete energy minimization.

Optimizes binary regression models with gradient ascent-descent methods.

This paper introduces a new method to evaluate multiple policies simultaneously.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

This paper introduces AdaSDCA: an adaptive variant of stochastic dual coordinate ascent (SDCA) for solving the regularized empirical risk minimization problems. Our modification consists in allowing the method adaptively change the probability distribution over the dual variables throughout the iterative process. AdaSD…

We introduce the notion of ascent sliceness of virtual knots. A representative of a virtual knot is an embedding $S^1 \hookrightarrow Σ_{g} \times I$, for $Σ_g$ a closed connected oriented surface of genus $g$; the virtual knot represented is slice if there exists a pair consisting of a disc $D$ and an oriented…

The paper connects two clustering methods by showing gradient ascent flow can move up the cluster tree.

Algorithm converges to Nash equilibria in competitive games.

Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.

This work reveals how label noise can cause a final ascent in neural network performance curves.

New algorithm for optimizing statistical utilities in bandits.