arXiv research

This work reveals a primal-dual relationship between GANs and Autoencoders, improving their theoretical understanding.

We consider empirical risk minimization of linear predictors with convex loss functions. Such problems can be reformulated as convex-concave saddle point problems, and thus are well suitable for primal-dual first-order algorithms. However, primal-dual algorithms often require explicit strongly convex regularization in …

We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…

Unified view connects CoCoA and ADMM for distributed ERM.

PDCA algorithm learns policies for RL with constraints using a primal-dual approach.

New algorithm speeds up large-scale statistical inference.

New method accelerates convergence for entropy-regularized reinforcement learning problems.

New algorithms improve computation of optimal transport and Wasserstein barycenter.

PURE-CD algorithm proves complexity bounds for convex-concave problems.

New algorithm achieves sublinear regret in CMDPs without error cancellations.

Multi-task learning aims to learn multiple tasks jointly by exploiting their relatedness to improve the generalization performance for each task. Traditionally, to perform multi-task learning, one needs to centralize data from all the tasks to a single machine. However, in many real-world applications, data of differen…

Accelerates machine learning algorithms for sparse data.

Quantized Stochastic Primal-Dual Methods for Distributed Optimization

In this paper, we propose a new primal-dual algorithm for minimizing $f(x) + g(x) + h(Ax)$, where $f$, $g$, and $h$ are proper lower semi-continuous convex functions, $f$ is differentiable with a Lipschitz continuous gradient, and $A$ is a bounded linear operator. The proposed algorithm has some famous primal-dual algo…

We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into $N$ subproblems, and utilizes an augmented Lagrangian b…

Paper tackles robust classification and feature selection with a novel primal-dual method.

We consider the convex-concave saddle point problem $\min_{x}\max_{y} f(x)+y^\top A x-g(y)$ where $f$ is smooth and convex and $g$ is smooth and strongly convex. We prove that if the coupling matrix $A$ has full column rank, the vanilla primal-dual gradient method can achieve linear convergence even if $f$ is not stron…

Binarization of digital documents is the task of classifying each pixel in an image of the document as belonging to the background (parchment/paper) or foreground (text/ink). Historical documents are often subjected to degradations, that make the task challenging. In the current work a deep neural network architecture …

We present a primal-dual algorithmic framework to obtain approximate solutions to a prototypical constrained convex optimization problem, and rigorously characterize how common structural assumptions affect the numerical efficiency. Our main analysis technique provides a fresh perspective on Nesterov's excessive gap te…

New methods solve saddle point problems without line search.

Drago optimizes DRO problems with faster convergence.

We propose an computational framework for real-time risk assessment and prioritizing for random outcomes without prior information on probability distributions. The basic model is built based on satisficing measure (SM) which yields a single index for risk comparison. Since SM is a dual representation for a family of r…

Efficient algorithm solves best subset selection problem.

Improved first-order algorithm for entropy regularized OT with faster convergence.

Unified algorithm solves convex optimization problems with optimal rates.

New algorithms solve convex-concave problems faster than previous methods.

Paper solves optimal transport with neural nets, also detects anomalies.

We propose a doubly stochastic primal-dual coordinate optimization algorithm for empirical risk minimization, which can be formulated as a bilinear saddle-point problem. In each iteration, our method randomly samples a block of coordinates of the primal and dual solutions to update. The linear convergence of our method…

We develop a primal dual active set with continuation algorithm for solving the \ell^0-regularized least-squares problem that frequently arises in compressed sensing. The algorithm couples the the primal dual active set method with a continuation strategy on the regularization parameter. At each inner iteration, it fir…

New methods solve MI problems with locally Lipschitz operators, improving solution efficiency.

Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updat…

Derives a primal-dual MLSVD formulation for multilinear data.

Develops a regression approach for solving MDPs with general state and action spaces.

In the present paper, the primal-dual problem consisting of the investment risk minimization problem and the expected return maximization problem in the mean-variance model is discussed using replica analysis. As a natural extension of the investment risk minimization problem under only a budget constraint that we anal…

In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we…

Paper proposes a novel metric learning algorithm using Riemannian optimization.

New algorithm approximates continuous Wasserstein barycenters efficiently.

Improved RL algorithm with linear MDPs for offline learning with partial data coverage.

Despite the success of single-agent reinforcement learning, multi-agent reinforcement learning (MARL) remains challenging due to complex interactions between agents. Motivated by decentralized applications such as sensor networks, swarm robotics, and power grids, we study policy evaluation in MARL, where agents with jo…

We relate the minimax game of generative adversarial networks (GANs) to finding the saddle points of the Lagrangian function for a convex optimization problem, where the discriminator outputs and the distribution of generator outputs play the roles of primal variables and dual variables, respectively. This formulation …

We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method…

Random extrapolation speeds up coordinate descent for sparse and dense data.

New algorithm learns optimal resource allocation in wireless systems without models.

This paper improves inverse problem solving with weakly convex regularisers and proves convergence.

A distributed algorithm for online multi-task learning reduces communication and runtime costs.

Study efficient convergence of RL algorithm with function approximation.

Paper analyzes and compares ELF algorithms for federated learning.

We generalize the primal-dual methodology, which is popular in the pricing of early-exercise options, to a backward dynamic programming equation associated with time discretization schemes of (reflected) backward stochastic differential equations (BSDEs). Taking as an input some approximate solution of the backward dyn…