arXiv research

New algorithms improve computation of optimal transport and Wasserstein barycenter.

We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into $N$ subproblems, and utilizes an augmented Lagrangian b…

A distributed algorithm for online multi-task learning reduces communication and runtime costs.

This paper proposes a communication-efficient distributed algorithm for high-dimensional data mining.

We consider empirical risk minimization of linear predictors with convex loss functions. Such problems can be reformulated as convex-concave saddle point problems, and thus are well suitable for primal-dual first-order algorithms. However, primal-dual algorithms often require explicit strongly convex regularization in …

Algorithm learns a single neuron robustly to shifts and adversarial noise.

Quantized Stochastic Primal-Dual Methods for Distributed Optimization

Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters (optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of…

A new method for distributed optimization reduces communication rounds without minibatches.

This work tackles resource allocation in asynchronous and stochastic systems.

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

Asymptotically optimal algorithm for contextual linear bandits.

In this paper, we propose a new primal-dual algorithm for minimizing $f(x) + g(x) + h(Ax)$, where $f$, $g$, and $h$ are proper lower semi-continuous convex functions, $f$ is differentiable with a Lipschitz continuous gradient, and $A$ is a bounded linear operator. The proposed algorithm has some famous primal-dual algo…

New algorithm achieves sublinear regret in CMDPs without error cancellations.

LEAD algorithm speeds up decentralized optimization with compression.

Accelerates machine learning algorithms for sparse data.

Paper proposes a voting-based MARL approach for IoT systems.

PURE-CD algorithm proves complexity bounds for convex-concave problems.

New algorithm samples constrained distributions efficiently.

Paper analyzes complexity of PSGLA for sampling log-concave distributions.

We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…

Improved first-order algorithm for entropy regularized OT with faster convergence.

PAPAL algorithm finds mixed Nash equilibria in continuous games.

New algorithm for federated learning with non-smooth regularizers.

We present a primal-dual algorithmic framework to obtain approximate solutions to a prototypical constrained convex optimization problem, and rigorously characterize how common structural assumptions affect the numerical efficiency. Our main analysis technique provides a fresh perspective on Nesterov's excessive gap te…

New algorithm speeds up large-scale statistical inference.

Unified algorithm solves convex optimization problems with optimal rates.

Oracle-efficient algorithm for offline RL with partial data coverage.

Efficient algorithm solves best subset selection problem.

New algorithms solve convex-concave problems faster than previous methods.

We develop a primal dual active set with continuation algorithm for solving the \ell^0-regularized least-squares problem that frequently arises in compressed sensing. The algorithm couples the the primal dual active set method with a continuation strategy on the regularization parameter. At each inner iteration, it fir…

We introduce Primal-Dual Wasserstein GAN, a new learning algorithm for building latent variable models of the data distribution based on the primal and the dual formulations of the optimal transport (OT) problem. We utilize the primal formulation to learn a flexible inference mechanism and to create an optimal approxim…

Communication remains the most significant bottleneck in the performance of distributed optimization algorithms for large-scale machine learning. In this paper, we propose a communication-efficient framework, CoCoA, that uses local computation in a primal-dual setting to dramatically reduce the amount of necessary comm…

Paper proposes a novel metric learning algorithm using Riemannian optimization.

Unified view connects CoCoA and ADMM for distributed ERM.

Paper solves optimal transport with neural nets, also detects anomalies.

Improved RL algorithm with linear MDPs for offline learning with partial data coverage.

Paper analyzes and compares ELF algorithms for federated learning.

Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.

A new algorithm reduces the complexity of solving optimal transport problems.

New approach shapes error distribution in long-term forecasting.

We relate the minimax game of generative adversarial networks (GANs) to finding the saddle points of the Lagrangian function for a convex optimization problem, where the discriminator outputs and the distribution of generator outputs play the roles of primal variables and dual variables, respectively. This formulation …

New algorithm solves complex minimax problems efficiently.

New framework guides resource usage to achieve sublinear regret in adversarial settings.

Drago optimizes DRO problems with faster convergence.

New algorithm solves composite optimization problems with unknown expectations.

Despite the success of single-agent reinforcement learning, multi-agent reinforcement learning (MARL) remains challenging due to complex interactions between agents. Motivated by decentralized applications such as sensor networks, swarm robotics, and power grids, we study policy evaluation in MARL, where agents with jo…

We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method…