arXiv research

A graph clustering method that moves nodes to highest-degree neighbors.

Generative model captures hubs and dense communities in social networks.

Improves posterior approximation speed for Dirichlet process mixture models.

We present a new algorithm based on an gradient ascent for a general Active Exploration bandit problem in the fixed confidence setting. This problem encompasses several well studied problems such that the Best Arm Identification or Thresholding Bandits. It consists of a new sampling rule based on an online lazy mirror …

Gradient ascent method successfully removes specific data points from neural networks without retraining.

Sequential coordinate ascent is more robust in high-dimensional linear regression.

Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised machine learning optimization problems such as SVM, due to their strong theoretical guarantees. While the closely related Dual Coordinate Ascent (DCA) method has been implemented in various software packages, it has so far lacked go…

Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning. This paper considers an extension of SDCA under the mini-batch setting that is often used in practice. Our main contribution is to introduce an accelerated mini-batch version of SDC…

New algorithm solves federated minimax optimization problems.

We introduce a proximal version of dual coordinate ascent method. We demonstrate how the derived algorithmic framework can be used for numerous regularized loss minimization problems, including $\ell_1$ regularization and structured output SVM. The convergence rates we obtain match, and sometimes improve, state-of-the-…

SoftAD improves classification accuracy with less fine-tuning and fewer computational costs.

Algorithm improves variational inference in Wasserstein distance.

Paper introduces SGA for barycenter optimization in optimal transport.

Gradient descent-ascent converges to strict local minmax equilibria with a finite timescale separation.

Lazy neural networks are vulnerable to adversarial attacks.

Policy mirror ascent achieves Nash equilibrium in mean field games without a population generative model.

Paper proposes an algorithm to solve complex minimax problems efficiently.

In machine learning, Feature Selection (FS) is a major part of efficient algorithm. It fuels the algorithm and is the starting block for our prediction. In this paper, we present a new method, called Optimal Coordinate Ascent (OCA) that allows us selecting features among block and individual features. OCA relies on coo…

The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees. However, the available S-DCA formulation is limited to finit…

Random scan CAVI converges linearly under log-concave assumptions.

We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including …

Generative adversarial networks (GANs) are a widely used framework for learning generative models. Wasserstein GANs (WGANs), one of the most successful variants of GANs, require solving a minmax optimization problem to global optimality, but are in practice successfully trained using stochastic gradient descent-ascent.…

New taxonomy and improved solvers for discrete energy minimization.

Optimizes binary regression models with gradient ascent-descent methods.

New ODE models show saddle-point optimization methods converge differently, with last-iterate convergence for OGDA.

Paper studies convergence of Mean-Field GDA dynamics for MNE of continuous games.

SAVO actor improves reinforcement learning by avoiding local optima in complex Q-functions.

This paper introduces AdaSDCA: an adaptive variant of stochastic dual coordinate ascent (SDCA) for solving the regularized empirical risk minimization problems. Our modification consists in allowing the method adaptively change the probability distribution over the dual variables throughout the iterative process. AdaSD…

We introduce the notion of ascent sliceness of virtual knots. A representative of a virtual knot is an embedding $S^1 \hookrightarrow Σ_{g} \times I$, for $Σ_g$ a closed connected oriented surface of genus $g$; the virtual knot represented is slice if there exists a pair consisting of a disc $D$ and an oriented…

The paper connects two clustering methods by showing gradient ascent flow can move up the cluster tree.

Algorithm converges to Nash equilibria in competitive games.

Gradient Descent Ascent converges to von-Neumann solution in hidden zero-sum games.

Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.

This work reveals how label noise can cause a final ascent in neural network performance curves.

Study local convergence of GDA for training GANs with kernel-based discriminators.

New algorithm for optimizing statistical utilities in bandits.

New solver MPLP++ outperforms existing solvers for dense graph models.

In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a unified analysis as approximations of the classical proximal point method for solving…

We present a dual subspace ascent algorithm for support vector machine training that respects a budget constraint limiting the number of support vectors. Budget methods are effective for reducing the training time of kernel SVM while retaining high accuracy. To date, budget training is available only for primal (SGD-ba…

We propose a new stochastic dual coordinate ascent technique that can be applied to a wide range of regularized learning problems. Our method is based on Alternating Direction Multiplier Method (ADMM) to deal with complex regularization functions such as structured regularizations. Although the original ADMM is a batch…

RSGDA improves convergence rates for nonconvex-strongly concave optimization.

We consider nonconvex-concave minimax optimization problems of the form $\min_{\bf x}\max_{\bf y\in{\mathcal Y}} f({\bf x},{\bf y})$, where $f$ is strongly-concave in $\bf y$ but possibly nonconvex in $\bf x$ and ${\mathcal Y}$ is a convex and compact set. We focus on the stochastic setting, where we can only access an…

Alt-GDA outperforms Sim-GDA in minimax games with near-optimal local convergence.

We comment on the fact that gradient ascent for logistic regression has a connection with the perceptron learning algorithm. Logistic learning is the "soft" variant of perceptron learning.

Paper improves risk bounds for nonconvex-strongly-concave minimax problems.

New convergence guarantees for SGDA and SCO under expected co-coercivity.

A scalable method for BED with implicit models using approximate gradients.

Method infers parameters in complex diffusion processes.